1

I want to open all parquet files existing in a specific folder and draw a scatter plot with it. I used following code: 

for file in glob.glob("*.parquet"):
    with pd.read_parquet(file, columns=["cordx", "cordy"]) as df:
        make some scaterplot

Files can be found as I let python print the file sucessfully. But i recive AttributeError: __enter__. And read_parquet is the corredct way to open those files. When I dont use columns=[ ] I get pyarrow.lib.ArrowIOError: Arrow error: Out of memory: malloc of size 9771487328 failed error because the files are very big. So its necessary to load only those columns.

Amir
  • 371
  • 6
  • 19

1 Answers1

3

This error tells you that pandas.read_parquet does not have a context manager implemented which means that you cannot call it using with because __enter__ and __exit__ methods are missing. Look in here How to read a Parquet file into Pandas DataFrame?

Colonder
  • 1,436
  • 1
  • 19
  • 38