Bash delete everything before the first blank on every line

Question

I have file of logs that all start with the timestamp, followed by the log level and then the message and I want a script that gets rid of the timestamp.

That is, I want a script that for every line of a file would turn:

21:22:34.571 DEBUG - some message

into

DEBUG - some message

I haven't used bash much so any advice would be appreciated.

When do dates appear in the log file — never, or only before the first message posted after midnight, or every hour, or ...? It matters because if the date lines are not preceded by a time stamp (and they probably shouldn't be), then you need to leave those alone. Mechanisms using `cut` or things like `sed 's/^[^ ]* //'` become less appropriate — you need to be more stringent in what you're matching. I'll observe that leaving the date out is probably not a good idea, but that's your problem, not mine. — Jonathan Leffler, Sep 28 '16 at 04:30
I'm trying to compare two sets of logs to try to figure out what's different between them, pretty hard to do that when it says every line is different due to the timestamp. — annedroiid, Sep 28 '16 at 04:33

score 2 · Answer 1 · answered Sep 28 '16 at 04:19

2

You can try either sed or cut depending on the input data:

sed -e 's/^[0-9]\{2\}:[0-9]\{2\}:[0-9]\{2\}.[0-9]\{3\}//' <data_file_name>


cut -c 13- <data_file_name>

answered Sep 28 '16 at 04:19

GMichael

2,666
1
19
30

Comment (wry amusement rather than anything more serious): Using `` in a shell script is not a good idea — it looks like bungled I/O redirection. Use a simple name: `data_file_name` or `log.file` or something appropriate, commenting "Assuming the data is in `log.file`" or something similar. Using `"$@"` can also be good; it indicates 'process all the file name arguments to the script, or standard input if there are none'. – Jonathan Leffler Sep 28 '16 at 04:24
1

Also, it might be simpler to use `sed 's/^[^ ]* //'` — delete all the non-blanks up to and including the first blank. Your script doesn't actually remove the blank, either. – Jonathan Leffler Sep 28 '16 at 04:25
@JonathanLeffler You're right in both cases. The question was about an idea. I gave two ideas. Nothing more – GMichael Sep 28 '16 at 04:33

score 2 · Accepted Answer · answered Sep 28 '16 at 04:30

2

If you could use awk:

awk '$1="";1' data_file_name

Else, use the shell (very very slow):

#!/bin/bash
while read -r line; do
    printf '%s\n' "${line#* }"
done <"data_file_name"

answered Sep 28 '16 at 04:30

done

6,370
27
49

Note that the `awk` script leaves leading blanks on the output lines. The shell code is clean in that respect. – Jonathan Leffler Sep 28 '16 at 04:40
@JonathanLeffler Worse, the `awk` code, by assigning to `$1`, causes `$0` to be reconstituted from all the parameters by interposing `OFS` between them. It clobbers the white-space separation between the fields. – Kaz Sep 28 '16 at 04:59
@Kaz And there is [this answer with 236 up votes that does exactly the same](http://stackoverflow.com/a/2961994/6843677). If you want to actually [keep the spaces use this (probably option 4)](http://stackoverflow.com/a/32774373/6843677), but that seems like too much for this simple need. – done Sep 28 '16 at 05:25

score 0 · Answer 3 · answered Sep 28 '16 at 04:50

grep can be used as well by simply extracting everything after space

$ cat ip.txt 
21:22:34.571 DEBUG - some message
21:23:34.571 DEBUG - some other message

This will leave a leading blank

$ grep -o ' .*' ip.txt 
 DEBUG - some message
 DEBUG - some other message

This won't

$ grep -oP ' \K.*' ip.txt 
DEBUG - some message
DEBUG - some other message

Bash delete everything before the first blank on every line

3 Answers3