6

I know ValueError question has been asked many times. I am still struggling to find an answer because I am using inverse_transform in my code.

Say I have an array a 

a.shape
> (100,20)

and another array b

b.shape
> (100,3)

When I did a np.concatenate, 

hat = np.concatenate((a, b), axis=1)

Now shape of hat is 

hat.shape    
(100,23)

After this, I tried to do this,

inversed_hat = scaler.inverse_transform(hat)

When I do this, I am getting an error:

ValueError: operands could not be broadcast together with shapes (100,23) (25,) (100,23)

Is this broadcast error in inverse_transform? Any suggestion will be helpful. Thanks in advance!

o-90
  • 15,749
  • 10
  • 40
  • 61

2 Answers2

6

Although you didn't specify, I'm assuming you are using inverse_transform() from scikit learn's StandardScaler. You need to fit the data first.

import numpy as np
from sklearn.preprocessing import MinMaxScaler


In [1]: arr_a = np.random.randn(5*3).reshape((5, 3))

In [2]: arr_b = np.random.randn(5*2).reshape((5, 2))

In [3]: arr = np.concatenate((arr_a, arr_b), axis=1)

In [4]: scaler = MinMaxScaler(feature_range=(0, 1)).fit(arr)

In [5]: scaler.inverse_transform(arr)
Out[5]:
array([[ 0.19981115,  0.34855509, -1.02999482, -1.61848816, -0.26005923],
       [-0.81813499,  0.09873672,  1.53824716, -0.61643731, -0.70210801],
       [-0.45077786,  0.31584348,  0.98219019, -1.51364126,  0.69791054],
       [ 0.43664741, -0.16763207, -0.26148908, -2.13395823,  0.48079204],
       [-0.37367434, -0.16067958, -3.20451107, -0.76465428,  1.09761543]])

In [6]: new_arr = scaler.inverse_transform(arr)

In [7]: new_arr.shape == arr.shape
Out[7]: True
o-90
  • 15,749
  • 10
  • 40
  • 61
  • thank you for your response, I know, I should have mentioned, I used `MinMaxScaler`. For example: `scaler = MinMaxScaler(feature_range=(0, 1))`. –  Aug 23 '17 at 19:01
  • I tried your answer, it works when I have `fit`, but I have `fit_transform` it gives an error `AttributeError: 'numpy.ndarray' object has no attribute 'inverse_transform'`. I used `fit_transform`. Do you know why this is happening? I am searching about this. –  Aug 23 '17 at 19:12
  • Yes, `fit_transform()` returns a dataset, `fit()` will produce an object from with which you can call other methods. If you read the [docs](http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html) you can see `fit()` has no return type, where as `fit_transform()` returns a numpy array. – o-90 Aug 23 '17 at 19:14
  • thank you! That is helpful! I am having a similar issue like this https://datascience.stackexchange.com/questions/22488/value-error-operands-could-not-be-broadcast-together-with-shapes-lstm and this one https://tutel.me/c/programming/questions/42997228/lstmkeras+error+valueerror+nonbroadcastable+output+operand+with+shape+677041+doesn39t+match+the+broadcast+shape+6770412 –  Aug 23 '17 at 19:17
  • @Jesse Ya in both of those questions you need to be doing something like `scaler = MinMaxScaler().fit(dataset)` and then to scale your dataset, do `scaled_data = scaler.transform(dataset)` and then at the end when you are trying to do and inverse_transform, do `scaler.inverse_transform(inv_yhat)` – o-90 Aug 23 '17 at 19:23
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/152667/discussion-between-gobrewers14-and-jesse). – o-90 Aug 23 '17 at 19:23
  • Why would you use an array with the same shape as the input for the inverse transformation? It should of the number of features selected. This does not make sense at all. Inverse transform should project back from PCA space to original space... – Soren Feb 25 '21 at 22:09
-1

It seems you are using pre-fit scaler object of sklearn.preprocessing. If it's true, according to me data that you have used for fitting is of dimension (x,25) whereas your data shape is of (x,23) dimension and thats the reason you are getting this issue.

vipin bansal
  • 780
  • 9
  • 10