I want to convert this data frame in format .tsv into a .nc file.
I am looking at these questions: Convert time series data from csv to netCDF python and convert csv to netcdf
but I am having some troubles. here the code I am using:
import pandas as pd
import numpy as np
import netCDF4 as nc
import xarray as xr
df = pd.read_csv("I:/Research/data.tsv", sep="\t")
#create netCDF file:
fn = 'I:/Research/New_data.nc'
ds = nc.Dataset(fn, 'w', format='NETCDF4')
depth = ds.createDimension('depth', df.depth.size)
lat = ds.createDimension('lat', df.latitude.size)
lon = ds.createDimension('lon', df.longitude.size)
lons_out = df.longitude.tolist()
lats_out = df.latitude.tolist()
depth_out = df.depth.tolist()
d15N_out = df.d15N.tolist()
lats = ds.createVariable('latitude',np.dtype('float32').char,('lat',))
lons = ds.createVariable('longitude',np.dtype('float32').char,('lon',))
depths = ds.createVariable('depth',np.dtype('float32').char,('depth',))
no3_d15N = ds.createVariable('no3_d15N',np.dtype('float32').char,('lat', 'lon', 'depth'))
no3_d15N.units = 'permill (vs Air N^2)'
lats[:] = lats_out
lons[:] = lons_out
depths[:] = depth_out
no3_d15N[:] = d15N_out
As soon as I run no3_d15N[:] = d15N_out I obtain this memory error:
MemoryError: Unable to allocate 4.25 EiB for an array with shape (1069813, 1069813, 1069813) and data type float32
Am I doing something wrong? How can I implement these values without that this memory error occurring?
