4

I am currently trying to save a list of numpy arrays into a single file, an example of such a list can be of the form below

import numpy as np
np_list = []
for i in range(10):
    if i % 2 == 0:
        np_list.append(np.random.randn(64))
    else:
        np_list.append(np.random.randn(32, 64))

I can combine all of them using into a single file using savez by iterating through list but is there any other way? I am trying to save weights returned by the function model.get_weights(), which is a list of ndarray and after retrieving the weights from the saved file I intend to load those weights into another model using model.set_weights(np_list). Therefore the format of the list must remain the same. Let me know if anyone has an elegant way of doing this.

DVK
  • 405
  • 1
  • 6
  • 25
  • the answer here seems to do the trick [NumPy save some arrays at once](https://stackoverflow.com/a/35133517/7537870) – DVK Feb 13 '20 at 02:07
  • Does this answer your question? [NumPy save some arrays at once](https://stackoverflow.com/questions/35133317/numpy-save-some-arrays-at-once) – AMC Feb 13 '20 at 02:26

1 Answers1

11

I would go with np.save and np.load because it's platform-independent, faster than savetxt and works with lists of arrays, for example:

import numpy as np

a = [
    np.arange(100),
    np.arange(200)
]
np.save('a.npy', a, allow_pickle=True)
b = np.load('a.npy', allow_pickle=True)

This is the documentation for np.save and np.load. And in this answer you can find a better discussion How to save and load numpy.array() data properly?

marcos
  • 4,253
  • 1
  • 9
  • 20
  • This does not work for a list of numpy arrays – DVK Feb 13 '20 at 02:09
  • @DVK you can set `allow_pickle=True` and it works. – marcos Feb 13 '20 at 02:11
  • Careful with this approach. `save` will do `np.array(a)` before saving. So `b` will be an array, not a list. And as others have found out, making an array from a list of arrays has its hidden dangers. Don't do it naively. – hpaulj Feb 13 '20 at 02:27
  • @hpaulj oh, that's True! Thanks for the insight. – marcos Feb 13 '20 at 02:28