boost::asio read - return after all data where read from socket, without waiting for EOF

Question

I'm quite new to boost::asio, I faced one problem I don't really know how to fix, could you please help me.

In general I'm trying to implement proxy based on boost::asio. I'm using async_read_some function to read response from server, something like that:

        _ssocket.async_read_some(boost::asio::buffer(_sbuffer),
                boost::bind(&connection::handle_server_read_body_some,
                        shared_from_this(),
                        boost::asio::placeholders::error,
                        boost::asio::placeholders::bytes_transferred
            ));

Everything is fine, it reads some bunch of data and call handler. The problem is at the moment when I'm caling async_read_some function - and there is no more data to read from socket. So handler is not called for about ~15 seconds - till EOF will be rased. (So server socket disconnected). I've tried different read functions, and all of them returns only when 1 or mote bytes where read or there was some error.

The thing is that sometimes I don't know how many bytes I need to read - so I just need to read everything what is present. I tried to use

boost::asio::socket_base::bytes_readable

or

 _ssocket.available(err)

To figgure out how many bytes avaliable on socket, but the thing is that those function returns number of bytes which could be read without blocking, so I can't base my implementation on that, even from tests I see that sometimes bytes_readable returns 0 - and next call of async_read_some on the same socket - reads bunch of data.

My question is - is there any way to get imidiate return (in case of synchronous call) / handler call (in case of async) when there is no more data to read from socket? Because currently it just hang for 15 sec till EOF.

I will appriciate any advice or tips you can give me.

I think you're up against the fundamentals of TCP/IP here. How does the socket know when there is no more data to read except when the remote socket tells it (i.e. EOF)? — Nick, May 16 '13 at 12:39
With streaming sockets (like TCP) it's impossible to know, because it's, well, streaming. A network stream like a TCP connection is much like a stream in real life, data flows from one end to the other without end, and the only way to end the stream is to cut the connection. If you want message boundaries then either add them to your protocol (or use a protocol including message boundaries), or use UDP. — Some programmer dude, May 16 '13 at 12:41
Hm...but do you think that 10-15 second needed to tell that? I mean, yep, I'm agree with you and expect that remote socket suppose to notify in case no more data present, but this delay is a bit too long, dont you think so? — miks131, May 16 '13 at 12:41
Maybe the remove end doesn't close it connections until after 10-15 seconds? And depending on the protocol, it might even be your responsibility send something back or to close the connection, and the other just sits there waiting for your reply or disconnect, and when nothing happens it just times out and does the disconnection itself. — Some programmer dude, May 16 '13 at 12:44
@JoachimPileborg I'm working with HTTP. Maybe it does not, checking it now. What I don't really like is that sometimes I don't know length of body, and when I'm using boost::asio::async_read with 16k buffer it does not call handler till connection will be closed - and it means I can't write data back to browser. Does it mean that I should use async_read only in case I know how much bytes do I need to read, and async_read_until "\r\n\r\n" when I dont know exact size of the message? — miks131, May 16 '13 at 13:10
For HTTP, you might want to read the header using [`async_read_until`](http://www.boost.org/doc/libs/1_53_0/doc/html/boost_asio/overview/core/line_based.html), and in the header you should have a [`Content-length`](http://stackoverflow.com/questions/2773396/whats-the-content-length-field-in-http-header) header field telling you how long the body is. — Some programmer dude, May 16 '13 at 13:21

Andriy Tylychko · Accepted Answer · 2013-05-16T13:57:01.317

There's nothing wrong with your usage of Boost.Asio. The problem is that you need to know how to deal with HTTP messages. Basically, you need to detect message type and parse it to get known its length. Server disconnection is not always the case because HTTP supports KEEP-ALIVE (the same connection is used for multiple messages). Please read following quote from RFC 2616:

http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html

4.4 Message Length

The transfer-length of a message is the length of the message-body as it appears in the message; that is, after any transfer-codings have been applied. When a message-body is included with a message, the transfer-length of that body is determined by one of the following (in order of precedence):

1.Any response message which "MUST NOT" include a message-body (such as the 1xx, 204, and 304 responses and any response to a HEAD request) is always terminated by the first empty line after the header fields, regardless of the entity-header fields present in the message.

2.If a Transfer-Encoding header field (section 14.41) is present and has any value other than "identity", then the transfer-length is defined by use of the "chunked" transfer-coding (section 3.6), unless the message is terminated by closing the connection.

3.If a Content-Length header field (section 14.13) is present, its decimal value in OCTETs represents both the entity-length and the transfer-length. The Content-Length header field MUST NOT be sent if these two lengths are different (i.e., if a Transfer-Encoding
 header field is present). If a message is received with both a
 Transfer-Encoding header field and a Content-Length header field,
 the latter MUST be ignored.
4.If the message uses the media type "multipart/byteranges", and the transfer-length is not otherwise specified, then this self- delimiting media type defines the transfer-length. This media type MUST NOT be used unless the sender knows that the recipient can parse it; the presence in a request of a Range header with multiple byte- range specifiers from a 1.1 client implies that the client can parse multipart/byteranges responses.
   A range header might be forwarded by a 1.0 proxy that does not
   understand multipart/byteranges; in this case the server MUST
   delimit the message using methods defined in items 1,3 or 5 of
   this section.
5.By the server closing the connection. (Closing the connection cannot be used to indicate the end of a request body, since that would leave no possibility for the server to send back a response.)

For compatibility with HTTP/1.0 applications, HTTP/1.1 requests containing a message-body MUST include a valid Content-Length header field unless the server is known to be HTTP/1.1 compliant. If a request contains a message-body and a Content-Length is not given, the server SHOULD respond with 400 (bad request) if it cannot determine the length of the message, or with 411 (length required) if it wishes to insist on receiving a valid Content-Length.

All HTTP/1.1 applications that receive entities MUST accept the "chunked" transfer-coding (section 3.6), thus allowing this mechanism to be used for messages when the message length cannot be determined in advance.

Messages MUST NOT include both a Content-Length header field and a non-identity transfer-coding. If the message does include a non- identity transfer-coding, the Content-Length MUST be ignored.

When a Content-Length is given in a message where a message-body is allowed, its field value MUST exactly match the number of OCTETs in the message-body. HTTP/1.1 user agents MUST notify the user when an invalid length is received and detected.

If Https connections are going via proxy.Then first time connect will establish a connection between client and server.Then all data will go in encrytped format .In that case how to read content.As the connection will be kept alive until whole data is exchanged. — asim, Jul 28 '14 at 17:58
@user1276728: HTTPS: [Technically, it is not a protocol in and of itself; rather, it is the result of simply layering the Hypertext Transfer Protocol (HTTP) on top of the SSL/TLS protocol](http://en.wikipedia.org/wiki/HTTP_Secure). Hence, no difference here, data you get from SSL socket is already decrypted and you can treat it as HTTP packet. — Andriy Tylychko, Jul 29 '14 at 10:43
Above i mentioned that via proxy i am reading content.So proxy will be acting as a tunnel between client and server in case of HTTPS. So whatever encrypted content i will get from client i have to transfer to server.My question is how to read encrypted content without knowing any header. — asim, Jul 29 '14 at 17:24
@asim I think you could use timers as a worst-case fallback. — akaltar, Jul 08 '16 at 17:59

boost::asio read - return after all data where read from socket, without waiting for EOF

1 Answers1