Make Apache encode or replace quotes instead of escaping them?

Question

Format Notes For security reasons, starting with version 2.0.46, non-printable and other special characters in %r, %i and %o are escaped using \xhh sequences, where hh stands for the hexadecimal representation of the raw byte. Exceptions from this rule are " and \, which are escaped by prepending a backslash, and all whitespace characters, which are written in their C-style notation (\n, \t, etc). In versions prior to 2.0.46, no escaping was performed on these strings so you had to be quite careful when dealing with raw log files.

This is a problem for Analog which is still the handiest analyser I use.

I get

.... "GET /somerequest?q=\"quoted string\"&someparm=bla"

in the logfile and it is of course flagged as corrupt since Analog expects

.... "GET /somerequest?q=%22quoted string%22&someparm=bla"

or similar.

I realise I can pre-process using something like

perl -p -i.bak -e 's/\\"/%22/g' logfile

But I'd rather not have to add this step to these files which are 50-90MB zipped per day

Thanks for any pointers

This is presumably just in the log files? – MrWhite Jun 06 '13 at 16:28 — MrWhite, Jun 06 '13 at 16:28
Yes. Only in The log files – mplungjan Jun 06 '13 at 16:53 — mplungjan, Jun 06 '13 at 16:53
I haven't used Analog since the 90's wow, #throwback – Anagio Sep 26 '13 at 16:47 — Anagio, Sep 26 '13 at 16:47
It is still amazing - very impressed how it runs – mplungjan Sep 26 '13 at 18:08 — mplungjan, Sep 26 '13 at 18:08

score 1 · Answer 1 · edited May 23 '17 at 12:37

You need to tie in and reformat log entries in real time (Rather than parsing in batch later). Essentially, you want to pipe the log to another process before writing.

You could write a short script to do this. For your reference, use this guide to learn about piping APache logs: https://httpd.apache.org/docs/2.0/logs.html#piped .

Note, my answer was based on help given by this stackoverflow post.

The other option would be to tie into the Apache logging hook, but I would recommend the pipe as the simpler option.

Make Apache encode or replace quotes instead of escaping them?

1 Answers1