I have two folders of .tif files (41 and 49 rasters, respectively). The rasters are multi-band (14) and of the same location (same extent, boundaries, etc), just on different days. I would like to extract all band data for a pre-determined pixel (I can use x,y coords or index location) and then make a table out of the data. The table should then appear with band numbers as column headers and each date as a new row.
As of now, I've tried loops that technically work, but after the 12-15th tif the memory/RAM of my IDE is full and the program crashes. I'm saving the info in a dictionary to later convert to a csv, but I'm not sure if this is the best method (or if I should immediately write data to a table). Could anyone offer advice on how to optimize my code, or an alternative method?
My two "most successful" codes are as follows:
c_dir = 'path'
pixel = 1265219
pixel_dict = {}
count = 0 # set so I can see how far the loops get before crashing
for image in os.listdir(c_dir):
if image.endswith('.tif'):
if image not in pixel_dict.keys(): # an attempt to re-run after storage full; not working
count += 1
image_arr = gdal_array.LoadFile(os.path.join(c_dir,image))
newimage = pd.DataFrame(image_arr.reshape([14,-1]).T)
band_data = newimage.loc[pixel][0:12] # pulls only band data, not every attribute
pixel_dict[image] = band_data
print(image, count)
And a second slightly different method:
c_dir = 'path'
pixel = 1265219
pixel_dict = {}
count = 0
for image in os.listdir(c_dir):
if image.endswith('.tif'):
with rio.open(os.path.join(c_dir, image)) as src:
count += 1
arr = src.read()
newimage = pd.DataFrame(arr.reshape([14,-1]).T)
band_data = newimage.loc[pixel][0:12]
pixel_dict[image] = band_data
print(image, count)
