I am having a complex nested json which I am flattening using the flatten_json library and some other logic. My requirement is to create a pandas DataFrame from this so that I can load it into Snowflake table.
The final flattened json has normal key-value pairs and also keys having an array of values. It looks like this:
new_flatten_json={
"a":"1",
"b":"2",
"c":[
"0.5",
"0.7"
],
"d":[
"100",
"200",
"300",
"400"
],
"e":"3"
}
I am trying to create a dataframe like this:
df=pd.DataFrame.from_records(new_flatten_json) and I am getting an error: ValueError: All arrays must be of the same length
So I tried to introduce an index, df=pd.DataFrame.from_records(new_flatten_json, index=[0])
but now the error is: ValueError: could not broadcast input array from shape (2,) into shape (1,)
It would be great if someone helps me out on this issue.