node encode and decode utf-16 buffer

Question

im working on a javascript/nodejs application that needs to talk with a C++ tcp/udp socket. It seems like I get from the old C++ clients an utf16 buffer. I didn't found a solution right now to convert it to a readable string and the other direction seems to be the same problem.

Is there a easy way for this two directions?

Nice greetings

Answer 1

If you have a UTF-16-encoded buffer, you can convert it to a UTF-8 string like this:

let string = buffer.toString('utf16le');

To read these from a stream, it's easiest to use convert to string at the very end:

let chunks = [];
stream.on('data', chunk => chunks.push(chunk))
      .on('end',  ()    => {
        let buffer = Buffer.concat(chunks);
        let string = buffer.toString('utf16le');
        ...
      });

To convert a JS string to UTF-16:

let buffer = Buffer.from(string, 'utf16le')

Answer 2

Single Buffer

If you have a single Buffer you can use its toString method that will convert all or part of the binary contents to a string using a specific encoding. It defaults to utf8 if you don't provide a parameter, but I've explicitly set the encoding in this example.

var req = http.request(reqOptions, function(res) { ...

res.on('data', function(chunk) {
    var textChunk = chunk.toString('utf8');
    // process utf8 text chunk
});

});

Streamed Buffers

If you have streamed buffers like in the question above where the first byte of a multi-byte UTF8-character may be contained in the first Buffer (chunk) and the second byte in the second Buffer then you should use a StringDecoder. :

var StringDecoder = require('string_decoder').StringDecoder;

var req = http.request(reqOptions, function(res) { ... var decoder = new StringDecoder('utf8');

res.on('data', function(chunk) {
    var textChunk = decoder.write(chunk);
    // process utf8 text chunk
});

}); If you have a single Buffer you can use its toString method that will convert all or part of the binary contents to a string using a specific encoding. It defaults to utf8 if you don't provide a parameter, but I've explicitly set the encoding in this example.

var req = http.request(reqOptions, function(res) { ...

res.on('data', function(chunk) {
    var textChunk = chunk.toString('utf8');
    // process utf8 text chunk
});

});