11

I want to know what is storage size limitation of an array in Firestore. I want to create millions of index in array and want to store data as a JSON object. Array will look like:

   [{id:1,name:'shakti',userdata:2122},
   {id:0.55,name:'shakti',userdata:2122},
   {id:1.58,name:'shakti',userdata:2122},
   {id:2.58,name:'shakti',userdata:2122},
   {id:1.5,name:'shakti',userdata:2122}];

I went through the documentation but did not get any proper guide there.

Alex Mamo
  • 112,184
  • 14
  • 139
  • 167
Shakti S
  • 301
  • 3
  • 13

2 Answers2

23

Edit:

For Android, there is a library named FirestoreDocument-Android, that will help you to check against the maximum of 1 MiB (1,048,576 bytes) quota.

There is no specific limitation when it comes to Firestore arrays. The limitation comes in case of documents because in Firestore the documents have limits. So there are some limits when it comes to how much data you can put into a document. According to the official documentation regarding usage and limits:

Maximum size for a document: 1 MiB (1,048,576 bytes)

As you can see, you are limited to 1 MiB total of data in a single document. When we are talking about storing text, you can store pretty much but as your array getts bigger, be careful about this limitation.

So, as long as you store data within this limitation, there will be no problem. If you think that you might get over this limitation, I recommend you to store that data in a collection. In this case there are no limitations. You can store as many documents as you want.

Alex Mamo
  • 112,184
  • 14
  • 139
  • 167
  • what will happen if the document reaches its 1MB quota after adding , lets say, 30000 items to the array? Will the app will crash if I update array again or simply the array update will not work but app will run smoothly? – Jayesh Babu Sep 21 '19 at 08:12
  • 3
    @JayeshBabu In that case, it won't be possible to write over the limit. There will be no crash. – Alex Mamo Sep 21 '19 at 08:30
  • Hi @AlexMamo, let's say that my array has 30000 items but my document has 100kb. This will be ok right? – Regis Dec 03 '19 at 02:18
  • 1
    @RegisZanandrea As long as the total size of the document is less than 1 MiB, yes it will be all right. – Alex Mamo Dec 03 '19 at 08:34
  • doesn't the 20k field/index limit come into play here with the array length? – Marko K Jan 01 '21 at 11:36
  • @MarkoK I'm not sure I understand your question, but please post a new question using its own [MCVE](https://stackoverflow.com/help/mcve), so I and other Firebase developers can help you. – Alex Mamo Jan 01 '21 at 14:37
  • @AlexMamo not a question, rather a half-baked answer based on my fairly limited Firebase knowledge. I was actually misinformed, turns out the limit is now 40k indexed fields, but the point remains – as far as I know, you wouldn't be able to hold arrays longer than 40k (unique?) elements, even if the document remains <1MB. All array elements get an index for array-contains, which should count towards that 40k index limit. _Edit: seeing as though you work at Google, I suppose I'm wrong, but luckily I'm not on the hunt for large Firestore arrays myself._ – Marko K Jan 03 '21 at 16:49
  • @MarkoK, you can mark it *Single-field index exemptions*. [docs](https://firebase.google.com/docs/firestore/query-data/index-overview#single-field_index_exemptions) – Kamushek Jun 01 '21 at 14:07
23

Each Firestore document can contain 1,048,576 bytes of data, a limit which includes not only the number of characters in each field name but in the name of the document itself. It's, therefore, impossible for a single document to contain an array with millions of items because there are barely a million available bytes in the document. A string array named fruits with two items "kiwi" and "orange" consumes 19 bytes and if each item in that array was a string about that long, the array would max out at about a quarter-million items.

At this point, you could distribute the array, which would simply be other arrays in other documents that spread the load (like a distributed counter described in Firestore documentation). You could randomly choose an array/document before writing to it or keep a counter that determines which array/document you write to next.

But at this point, you may be better off rethinking your data architecture because Firestore is purpose built for large collections with small documents. And—as far as the writing of this answer—there is no known limit to the size of a collection.

Whatever you choose, just be aware that Firestore charges ($) per document read and write, so fetching an array with 1,000 items will cost you 1 read, whereas fetching 1,000 documents will cost you 1,000 reads.

fake girlfriends
  • 10,377
  • 2
  • 38
  • 42
  • Correct. Just in case anyone is wondering: In this example each letter is a byte and then firestore charges 1 extra byte per word. See: https://firebase.google.com/docs/firestore/storage-size – Andres Silva Jul 20 '20 at 15:01