Extracting specific columns in numpy array

Question

This is an easy question but say I have an MxN matrix. All I want to do is extract specific columns and store them in another numpy array but I get invalid syntax errors. Here is the code:

extractedData = data[[:,1],[:,9]].

It seems like the above line should suffice but I guess not. I looked around but couldn't find anything syntax wise regarding this specific scenario.

score 347 · Accepted Answer · edited Aug 23 '20 at 10:39

347

I assume you wanted columns 1 and 9?

To select multiple columns at once, use

X = data[:, [1, 9]]

To select one at a time, use

x, y = data[:, 1], data[:, 9]

With names:

data[:, ['Column Name1','Column Name2']]

You can get the names from data.dtype.names…

edited Aug 23 '20 at 10:39

cs95

330,695
80
606
657

answered Dec 05 '11 at 14:24

Fred Foo

342,876
71
713
819

How to do that with column names? – Zelphir Kaltstahl Nov 20 '16 at 21:02
9

data[:, ['Column Name1','Column Name2']] – code-assassin Dec 06 '16 at 13:37
is it a view or a copy? my bottleneck is on this line I search way to optimize – Fractale Oct 02 '18 at 03:04
2

could it be that this function is not working anymore? – PV8 Jul 11 '19 at 09:07
What is this syntax called? – Burrito Jul 03 '20 at 14:51
If you get an IndexError saying only "integers, slices (`:`), ellipsis ..." etc are valid, try dropping the colon and just using the list of column names (e.g. `data[['Column Name1', 'Column Name2']]`) – Jeremiah England Aug 10 '20 at 20:19
If you want to extract all columns from `a` to `b`, try `X = data[:, range(a, b + 1)]` – Daniel Aug 23 '21 at 20:16

score 34 · Answer 2 · answered Dec 05 '11 at 14:26

34

Assuming you want to get columns 1 and 9 with that code snippet, it should be:

extractedData = data[:,[1,9]]

answered Dec 05 '11 at 14:26

Michael J. Barber

23,704
8
64
85

score 15 · Answer 3 · answered Jun 01 '15 at 11:14

if you want to extract only some columns:

idx_IN_columns = [1, 9]
extractedData = data[:,idx_IN_columns]

if you want to exclude specific columns:

idx_OUT_columns = [1, 9]
idx_IN_columns = [i for i in xrange(np.shape(data)[1]) if i not in idx_OUT_columns]
extractedData = data[:,idx_IN_columns]

yanhh · Answer 4 · 2018-07-28T01:25:55.013

Just:

>>> m = np.matrix(np.random.random((5, 5)))
>>> m
matrix([[0.91074101, 0.65999332, 0.69774588, 0.007355  , 0.33025395],
        [0.11078742, 0.67463754, 0.43158254, 0.95367876, 0.85926405],
        [0.98665185, 0.86431513, 0.12153138, 0.73006437, 0.13404811],
        [0.24602225, 0.66139215, 0.08400288, 0.56769924, 0.47974697],
        [0.25345299, 0.76385882, 0.11002419, 0.2509888 , 0.06312359]])
>>> m[:,[1, 2]]
matrix([[0.65999332, 0.69774588],
        [0.67463754, 0.43158254],
        [0.86431513, 0.12153138],
        [0.66139215, 0.08400288],
        [0.76385882, 0.11002419]])

The columns need not to be in order:

>>> m[:,[2, 1, 3]]
matrix([[0.69774588, 0.65999332, 0.007355  ],
        [0.43158254, 0.67463754, 0.95367876],
        [0.12153138, 0.86431513, 0.73006437],
        [0.08400288, 0.66139215, 0.56769924],
        [0.11002419, 0.76385882, 0.2509888 ]])

Daksh · Answer 5 · 2018-03-05T16:20:51.823

11

One thing I would like to point out is, if the number of columns you want to extract is 1 the resulting matrix would not be a Mx1 Matrix as you might expect but instead an array containing the elements of the column you extracted.

To convert it to Matrix the reshape(M,1) method should be used on the resulting array.

edited Mar 05 '18 at 16:20

answered Oct 21 '17 at 14:53

Daksh

976
10
22

2

Also you can achieve this by using a colon, for example `data[:, 8:9]`. This takes the eight column but does not remove the extra dimension. – Jan Kukacka Feb 05 '18 at 16:19
1

data[:,8] will also pick the 8th column and return a Mx1 Matrix – StefanMK May 15 '20 at 09:30

score 3 · Answer 6 · answered Feb 05 '18 at 16:25

One more thing you should pay attention to when selecting columns from N-D array using a list like this:

data[:,:,[1,9]]

If you are removing a dimension (by selecting only one row, for example), the resulting array will be (for some reason) permuted. So:

print data.shape            # gives [10,20,30]
selection = data[1,:,[1,9]]
print selection.shape       # gives [2,20] instead of [20,2]!!

score 2 · Answer 7 · edited Aug 15 '20 at 09:32

2

You can use the following:

extracted_data = data.ix[:,['Column1','Column2']]

edited Aug 15 '20 at 09:32

galoget

614
9
15

answered Sep 04 '17 at 09:34

Rahul

21
1

4

A good answer will always have an explanation of what was done and why it was done in such a manner, not only for the OP but for future visitors to SO. Please add some description to make others understand. – Rucha Bhatt Joshi Sep 04 '17 at 09:48

score 0 · Answer 8 · answered Jul 11 '19 at 09:09

0

I think the solution here is not working with an update of the python version anymore, one way to do it with a new python function for it is:

extracted_data = data[['Column Name1','Column Name2']].to_numpy()

which gives you the desired outcome.

The documentation you can find here: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_numpy.html#pandas.DataFrame.to_numpy

answered Jul 11 '19 at 09:09

PV8

5,145
4
35
64

the question starts with a numpy array, not a dataframe – TMrtSmith Nov 08 '19 at 15:07

score -2 · Answer 9 · answered Aug 02 '17 at 17:18

-2

you can also use extractedData=data([:,1],[:,9])

answered Aug 02 '17 at 17:18

Pranav Mahajan

21
1

Extracting specific columns in numpy array

9 Answers9

Linked

Related