0

I'm doing a bulk indexing in AWS OpenSearch within my node application and it's failing because the chunk size are too big. Below is the error

Request size exceeded 104857600 bytes

So I need to calculate the actual chunk size in bytes. So I tried with below 2 ways and not sure which one is correct.

// with Buffer
const str = 'tree'
const obj = [{a:'sas'},{a:'e'}]
console.log(Buffer.from(str).length); ---> 4
console.log(Buffer.from(obj).length); ---> 2

// with object-sizeof
var sizeof = require('object-sizeof')
const str = 'tree'
const obj = [{a:'sas'},{a:'e'}]
console.log(sizeof(str)); ---> 8
console.log(sizeof(obj)); ---> 12

It seems 'Buffer' takes 1 Byte for a char while 'object-sizeof' takes 2 bytes for a char. So what is correct here? And which is the correct way to check the byte size of actual object array? Thanks in advance

ahkam
  • 475
  • 3
  • 9

1 Answers1

0

According to answers on How many bytes in a JavaScript string? you can use Buffer.byteLength(string, 'utf8') to get the byte size of a string.

Anyway, the size of a char in javascript is 2 bytes, so i guess it's the same with typescript/node.js : https://developer.mozilla.org/en-US/docs/Web/API/DOMString/Binary

liguepk
  • 61
  • 5