Accessing Pandas column using squared brackets vs using a dot (like an attribute)

Question

In both the bellow cases:

import pandas

d = {'col1': 2, 'col2': 2.5}
df = pandas.DataFrame(data=d, index=[0])

print(df['col2'])
print(df.col2)

Both methods can be used to index on a column and yield the same result, so is there any difference between them?

Answer 1

The "dot notation", i.e. df.col2 is the attribute access that's exposed as a convenience.

You may access an index on a Series, column on a DataFrame, and an item on a Panel directly as an attribute:

df['col2'] does the same: it returns a pd.Series of the column.

A few caveats about attribute access:

• you cannot add a column (df.new_col = x won't work, worse: it will silently actually create a new attribute rather than a column - think monkey-patching here)
• it won't work if you have spaces in the column name or if the column name is an integer.

Answer 2

They are the same as long you're accessing a single column with a simple name, but you can do more with the bracket notation. You can only use df.col if the column name is a valid Python identifier (e.g., does not contains spaces and other such stuff). Also, you may encounter surprises if your column name clashes with a pandas method name (like sum). With brackets you can select multiple columns (e.g., df[['col1', 'col2']]) or add a new column (df['newcol'] = ...), which can't be done with dot access.

The other question you linked to applies, but that is a much more general question. Python objects get to define how the . and [] operators apply to them. Pandas DataFrames have chosen to make them the same for this limited case of accessing single columns, with the caveats described above.

Answer 3

Short answer for differences:

• [] indexing (squared brackets access) has the full functionaly to operate on DataFrame column data.
• While attribute access (dot access) is mainly for convinience to access existing DataFrame column data, but occasionally has its limitations (e.g. special column names, creating a new column).

---

More explaination, Seires and DataFrame are core classes and data structures in pandas, and of course they are Python classes too, so there are some minor distinction when involving attribute access between pandas DataFrame and normal Python objects. But it's well documented and can be easily understood. Just few points to note:

1. In Python, users may dynamically add data attributes of their own to an instance object using attribute access.

   >>> class Dog(object):
   ...     pass
   >>> dog = Dog()
   >>> vars(dog)
   {}
   >>> superdog = Dog()
   >>> vars(superdog)
   {}
   >>> dog.legs = 'I can run.'
   >>> superdog.wings = 'I can fly.'
   >>> vars(dog)
   {'legs': 'I can run.'}
   >>> vars(superdog)
   {'wings': 'I can fly.'}
   

2. In pandas, index and column are closely related to the data structure, you may access an index on a Series, column on a DataFrame as an attribute.

   >>> import pandas as pd
   >>> import numpy as np
   >>> data = np.random.randint(low=0, high=10, size=(2,2))
   >>> df = pd.DataFrame(data, columns=['a', 'b'])
   >>> df
      a  b
   0  7  6
   1  5  8
   >>> vars(df)
   {'_is_copy': None, 
    '_data': BlockManager
       Items: Index(['a', 'b'], dtype='object')
       Axis 1: RangeIndex(start=0, stop=2, step=1)
       IntBlock: slice(0, 2, 1), 2 x 2, dtype: int64,
    '_item_cache': {}}
   

3. But, pandas attribute access is mainly a convinience for reading from and modifying an existing element of a Series or column of a DataFrame.

   >>> df.a
   0    7
   1    5
   Name: a, dtype: int64
   >>> df.b = [1, 1]
   >>> df
      a  b
   0  7  1
   1  5  1
   

4. And, the convinience is a tradeoff for full functionality. E.g. you can create a DataFrame object with column names ['space bar', '1', 'loc', 'min', 'index'], but you can't access them as an attribute, because they are either not a valid Python identifier 1, space bar or conflicts with an existing method name.

   >>> data = np.random.randint(0, 10, size=(2, 5))
   >>> df_special_col_names = pd.DataFrame(data, columns=['space bar', '1', 'loc', 'min', 'index'])
   >>> df_special_col_names
      space bar  1  loc  min  index
   0          4  4    4    8      9
   1          3  0    1    2      3
   

5. In these cases, the .loc, .iloc and [] indexing is the defined way to fullly access/operate index and columns of Series and DataFrame objects.

   >>> df_special_col_names['space bar']
   0    4
   1    3
   Name: space bar, dtype: int64
   >>> df_special_col_names.loc[:, 'min']
   0    8
   1    2
   Name: min, dtype: int64
   >>> df_special_col_names.iloc[:, 1]
   0    4
   1    0
   Name: 1, dtype: int64
   

6. Another important difference is when tyring to create a new column for DataFrame. As you can see, df.c = df.a + df.b just created an new attribute along side to the core data structure, so starting from version 0.21.0 and later, this behavior will raise a UserWarning (silent no more).

   >>> df
      a  b
   0  7  1
   1  5  1
   >>> df.c = df.a + df.b
   __main__:1: UserWarning: Pandas doesn't allow columns to be created via a new attribute name - see https://pandas.pydata.org/pandas-docs/stable/indexing.html#attribute-access
   >>> df['d'] = df.a + df.b
   >>> df
      a  b  d
   0  7  1  8
   1  5  1  6
   >>> df.c
   0    8
   1    6
   dtype: int64
   >>> vars(df)
   {'_is_copy': None, 
    '_data': 
       BlockManager
       Items: Index(['a', 'b', 'd'], dtype='object')
       Axis 1: RangeIndex(start=0, stop=2, step=1)
       IntBlock: slice(0, 2, 1), 2 x 2, dtype: int64
       IntBlock: slice(2, 3, 1), 1 x 2, dtype: int64, 
    '_item_cache': {},
    'c': 0    8
         1    6
         dtype: int64}
   

7. Finally, to create a new column for DataFrame, never use attribute access, the correct way is to use either [] or .loc indexing:

   >>> df
      a  b
   0  7  6
   1  5  8
   >>> df['c'] = df.a + df.b 
   >>> # OR
   >>> df.loc[:, 'c'] = df.a + df.b
   >>> df # c is an new added column
      a  b   c
   0  7  6  13
   1  5  8  13
   

Answer 4

. notation is very useful when working interactively and for exploration. However, for code clarity and to avoid crazy shit to happen, you definitely should use [] notation. An example why you should use [] when creating a new column.

df = pd.DataFrame(data={'A':[1, 2, 3],
                    'B':[4,5,6]})

# this has no effect
df.D = 11
df

    A  B
 0  1  4
 1  2  5
 2  3  6
# but this works
df['D'] = 11
df

Out[19]: 
   A  B   D
0  1  4  11
1  2  5  11  
2  3  6  11

Answer 5

If you had a dataframe like this (I am not recommending these column names)...

df = pd.DataFrame({'min':[1,2], 'max': ['a','a'], 'class': [1975, 1981], 'sum': [3,4]})

print(df)

   min max  class  sum
0    1   a   1975    3
1    2   a   1981    4

It all looks OK and there are no errors. You can even access the columns via df['min'] etc...

print(df['min'])

0    1
1    2
Name: min, dtype: int64

However, if you tried with df.<column_name> you would get problems:

print(df.min)

<bound method NDFrame._add_numeric_operations.<locals>.min of    min max  class  sum
0    1   a   1975    3
1    2   a   1981    4>

print(df.max)

<bound method NDFrame._add_numeric_operations.<locals>.max of    min max  class  sum
0    1   a   1975    3
1    2   a   1981    4>

print(df.class)

  File "<ipython-input-31-3472b02a328e>", line 1
    print(df.class)
             ^
SyntaxError: invalid syntax
    
print(df.sum)

<bound method NDFrame._add_numeric_operations.<locals>.sum of    min max  class  sum
0    1   a   1975    3
1    2   a   1981    4>