classification
Title: python crashes when pandas reading parquet
Type: crash Stage: resolved
Components: IO, Windows Versions: Python 3.6
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: brand17, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2021-06-19 10:57 by brand17, last changed 2021-06-19 11:21 by steve.dower. This issue is now closed.

Files
File name Uploaded Description Edit
message.png brand17, 2021-06-19 10:57
Messages (3)
msg396124 - (view) Author: Andrey Brykin (brand17) Date: 2021-06-19 10:57
Python crashes when running this code (I attached the error message screenshot):

import pandas as pd
d = {'col1': [[0.] * 25] * 2560}
df = pd.DataFrame(data=d)
df.to_parquet('data.parquet')
for j in range(15):
  table = pd.read_parquet('data.parquet')

There is no error when running from python 3.9.5 with the same pandas version. So it doesn't relate to pandas. 

The error happening with the exact list size of 2560 - no error with 2561 or 2559. Another dimension is also important: there is no error if it is exceeding 25.

I am running on Windows 10 2004.
msg396125 - (view) Author: Andrey Brykin (brand17) Date: 2021-06-19 10:59
pandas version is 1.1.5
msg396127 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-06-19 11:21
Python 3.6 is no longer supported by us, so unless this occurs with a newer version (which apparently it does not), we're not going to be able to look into it.

If this is critical to you, you probably want to get in touch with Anaconda or ActiveState to see if they will provide you with support (or someone else, not sure who else is offering support like this right now... Quansight maybe?).
History
Date User Action Args
2021-06-19 11:21:57steve.dowersetstatus: open -> closed
resolution: out of date
messages: + msg396127

stage: resolved
2021-06-19 10:59:25brand17setmessages: + msg396125
2021-06-19 10:57:49brand17create