How do I receive arbitrary length data using a UdpSocket?

Question

I am writing an application which sends and receives packages using UDP. However, the documentation of recv_from states:

If a message is too long to fit in the supplied buffer, excess bytes may be discarded.

Is there any way to receive all bytes and write them into a vector? Do I really have to allocate an array with the maximum packet length (which, as far as I know, is 65,507 bytes for IPv4) in order to be sure to receive all data? That seems a bit much for me.

This is a problem for C as well: http://www.microhowto.info/howto/listen_for_and_receive_udp_datagrams_in_c.html — Neikos, Jan 17 '18 at 08:44
Temporarily getting 65k of uninitialized memory from stack shouldn't be a big deal. — ArtemGr, Jan 17 '18 at 09:01
@ArtemGr it probably isn't, but why would I allocate that much if I could just grow only if I need the space? because most packets are probably only a few bytes long — Lukor, Jan 17 '18 at 12:27
@Lukor Because extra system calls are much more expensive than getting some space from the stack. — ArtemGr, Jan 17 '18 at 12:34

Shepmaster · Accepted Answer · 2018-01-17T15:15:50.483

Check out the next method in the docs, UdpSocket::peek_from (emphasis mine):

Receives a single datagram message on the socket, without removing it from the queue.

You can use this to read a fixed amount of data, such as the leading 4 bytes containing the length of the entire packet. You can use byteorder to get the length as a number then allocate exactly the right amount of space and call recv_from.

Now, is this a good idea?

As ArtemGr states:

Because extra system calls are much more expensive than getting some space from the stack.

And from the linked question:

Obviously at some point you will start wondering if doubling the number of system calls to save memory is worth it. I think it isn't.

With the recent Spectre / Meltdown events, now's a pretty good time to be be reminded to avoid extra syscalls.

You could, as suggested, just allocate a "big enough" array ahead of time. You'll need to track how many bytes you've actually read vs allocated though. I recommend something like arrayvec to make it easier.

You could instead implement a pool of pre-allocated buffers on the heap. When you read from the socket, you use a buffer or create a new one. When you are done with the buffer, you put it back in the pool for reuse. That way, you incur the memory allocation once and are only passing around small Vecs on the stack.

How do I receive arbitrary length data using a UdpSocket?

1 Answers1