Select Pandas rows based on list index

Question

I have a dataframe df:

20060930  10.103       NaN     10.103   7.981
20061231  15.915       NaN     15.915  12.686
20070331   3.196       NaN      3.196   2.710
20070630   7.907       NaN      7.907   6.459

Then I want to select rows with certain sequence numbers which indicated in a list, suppose here is [1,3], then left:

20061231  15.915       NaN     15.915  12.686
20070630   7.907       NaN      7.907   6.459

How or what function can do that?

score 200 · Answer 1 · edited Mar 12 '21 at 12:13

200

ind_list = [1, 3]
df.ix[ind_list]

should do the trick! When I index with data frames I always use the .ix() method. Its so much easier and more flexible...

UPDATE This is no longer the accepted method for indexing. The ix method is deprecated. Use .iloc for integer based indexing and .loc for label based indexing. See below example:

ind_list = [1, 3]
df.iloc[ind_list]

edited Mar 12 '21 at 12:13

tpk

1,228
4
16
32

answered Oct 03 '13 at 09:43

Woody Pride

12,263
8
46
60

34

This is now deprecated, .iloc should be used for positional indexing – t_warsop May 09 '19 at 12:56
8

This didn't work for me, I had to use `df.iloc[[1,3],:]` – MogaGennis May 25 '21 at 11:10
The solution in the update doesn't work either, I have the latest working solution as of March 2021 [here](https://stackoverflow.com/a/66579467/14022582) – user42 Jun 07 '21 at 14:31

score 135 · Answer 2 · edited Apr 17 '20 at 01:22

135

you can also use iloc:

df.iloc[[1,3],:]

This will not work if the indexes in your dataframe do not correspond to the order of the rows due to prior computations. In that case use:

df.index.isin([1,3])

... as suggested in other responses.

edited Apr 17 '20 at 01:22

Community

1
1

answered Oct 10 '13 at 12:17

yemu

22,689
10
30
29

score 94 · Answer 3 · edited Jun 20 '20 at 09:12

94

Another way (although it is a longer code) but it is faster than the above codes. Check it using %timeit function:

df[df.index.isin([1,3])]

PS: You figure out the reason

edited Jun 20 '20 at 09:12

Community

1
1

answered Jan 08 '19 at 11:14

Amruth Lakkavaram

1,257
8
11

6

use `df.index.get_level_values(0).isin` for multiindex – CiaranWelsh Oct 14 '19 at 13:29
Your comparison is invalid, because it should compare with `df.iloc` instead of `df.loc`. – Tom Price Jun 04 '22 at 17:27

score 15 · Answer 4 · answered Mar 11 '21 at 09:13

15

If index_list contains your desired indices, you can get the dataframe with the desired rows by doing

index_list = [1,2,3,4,5,6]
df.loc[df.index[index_list]]

This is based on the latest documentation as of March 2021.

answered Mar 11 '21 at 09:13

user42

568
4
16

1

This is a great answer. The advantage of this method is that you can use the full power of df.loc. For example you can select the column you want with df.loc[df.index[index_list], "my_column"] and even set values with df.loc[df.index[index_list], "my_column"] = "my_value" – Gabriel Apr 05 '21 at 14:19

score 5 · Answer 5 · edited Jun 20 '20 at 09:12

For large datasets, it is memory efficient to read only selected rows via the skiprows parameter.

Example

pred = lambda x: x not in [1, 3]
pd.read_csv("data.csv", skiprows=pred, index_col=0, names=...)

This will now return a DataFrame from a file that skips all rows except 1 and 3.

Details

From the docs:

skiprows : list-like or integer or callable, default None

...

If callable, the callable function will be evaluated against the row indices, returning True if the row should be skipped and False otherwise. An example of a valid callable argument would be lambda x: x in [0, 2]

This feature works in version pandas 0.20.0+. See also the corresponding issue and a related post.

score 2 · Answer 6 · answered Nov 05 '20 at 03:05

2

There are many ways of solving this problem, and the ones listed above are the most commonly used ways of achieving the solution. I want to add two more ways, just in case someone is looking for an alternative.

index_list = [1,3]

df.take(pos)

#or

df.query('index in @index_list')

answered Nov 05 '20 at 03:05

Loochie

2,066
10
18

this is the correct answer if you have say a named index like: `pd.DataFrame({'num_legs': [2, 4, 8, 0, 6, 10], 'num_wings': [2, 0, 0, 0, 4, 0], 'num_specimen_seen': [10, 2, 1, 8, 3, 0], 'do_I_like_it': [0, 1, 1, 1, 0, 0]}, index=['falcon', 'dog', 'spider', 'fish', 'dragonfly', 'limulus'])` – user27221 Mar 11 '21 at 16:15
@user27221 could you please take your DataFrame, transpose it, and then explain me how to select `num_legs` based on `num_wings == 0` and `do_I_like_it == 1`? – vault Sep 08 '21 at 11:16

score 0 · Answer 7 · answered May 27 '22 at 10:29

What you are trying to do is to filter your dataframe by index. The best way to do that in pandas at the moment is the following:

Single Index

desired_index_list = [1,3]
df[df.index.isin(desired_index_list)]

Multiindex

desired_index_list = [1,3]
index_level_to_filter = 0
df[df.index.get_level_values(index_level_to_filter).isin(desired_index_list)]

Select Pandas rows based on list index

7 Answers7

Linked

Related