3

While using:

with open("data_file.pickle", "rb") as pfile:
     raw_data = pickle.load(pfile)

I get the error:

AttributeError: Can't get attribute '_unpickle_block' on <module 'pandas._libs.internals' from '/opt/conda/lib/python3.8/site-packages/pandas/_libs/internals.cpython-38-x86_64-linux-gnu.so'>

Another answer to a similar question suggests checking the version of pickle I am using. It is the same on my machine, where I developed the code and on server, where I am running the code. I have searched everywhere with no answers. Please help.

No-Time-To-Day
  • 33
  • 1
  • 5

2 Answers2

2

I don't think the problem is pickle module but Pandas version. Your file was probably created with an older version of Pandas. Now you use a newer version, pickle can't "deserialize" the object because the API change.

Try to downgrade your Pandas version and reload file. You can also try to use pd.read_pickle.

Corralien
  • 70,617
  • 7
  • 16
  • 36
  • Hi, thanks for the reply. This seems to be the problem. I downgraded pandas on my machine to 1.1.5, which is on the server, now I have error in the initial read from the raw dataset itself. I was wondering if there is any way to `pickle.read` using newer version of pandas and `pickle.dump` using 1.1.5. I tried `pd.read_pickle`, it throws up the same error. – No-Time-To-Day Feb 12 '22 at 09:45
  • I don't know if this is the way to proceed. However, this is what I did: I read data using `pandas v1.4.0` and serialized it as hdf5 file. Then downgraded pandas to `pandas v1.1.5` serialized again using pickle. @Corralien was right, the issue was with pandas and not pickle. – No-Time-To-Day Feb 12 '22 at 10:21
0

In my case I had to upgrade instead of downgrade the Pandas version. Just make sure they match. Some tips for future readers:

Ask the version with:

import pandas as pd
pd.__version__

And change the version with (replace with your own version)

%pip install pandas==1.4.1
Tessa I
  • 21
  • 2