No duplicated lines in bash history

Question

Supposedly, adding export HISTCONTROL=ignoreboth to .bashrc would do the trick, but what I see in practice is there still exists duplicated lines in my .bash_history file.

You can see my .bashrc below.

export HISTFILESIZE=100000
export HISTSIZE=100000 
export HISTIGNORE="ls*:rm*:cd*:CD*:ps*:exit*:reset*:clear*:synaptic*:mkdir*:cat*"
export HISTCONTROL=ignoreboth
shopt -s histappend
export PROMPT_COMMAND="history -a; history -c; history -r; $PROMPT_COMMAND"

Does anybody know why I'm still getting duplicated lines?

http://askubuntu.com/questions/15926/how-to-avoid-duplicate-entries-in-bash-history — Rinzwind, May 15 '13 at 10:23
I saw that question, but it does not seems to be the same since I'm already doing what it suggests — Psicofrenia, May 15 '13 at 10:24
@Psicofrenia It is export HISTCONTROL=ignoreboth:erasedups, or you write export HISTCONTROL=ignoreboth that's incorrect. — Radu Rădeanu, May 15 '13 at 10:26
Sorry but ... export HISTCONTROL=ignoreboth and export HISTCONTROL=ignoreboth:erasedups are 2 diff. things — Rinzwind, May 15 '13 at 10:26
I changed to export HISTCONTROL=ignoreboth:erasedups and it still duplicating :( — Psicofrenia, May 15 '13 at 10:28
HISTCONTROL=ignoreboth ignores only duplicate consecutive lines. You'll still get duplicate non-consecutive lines. Is that your problem? — Eric Carvalho, May 15 '13 at 10:38
Ah, maybe that's the problem. What I want is to NEVER have the same line twice in .bash_history, being consecutive or note. That means I can use only export HISTCONTROL=erasedups ? — Psicofrenia, May 15 '13 at 10:42
From man bash: "A value of erasedups causes all previous lines matching the current line to be removed from the history list before that line is saved." Did you test erasedups? — Eric Carvalho, May 15 '13 at 11:14

score 4 · Answer 1 · edited Apr 13 '17 at 12:37

4

HISTCONTROL only affects new hist lines.
What I think the author what is a way to edit the existing hist file.

nl ~/.bash_history | sort -k 2 | uniq -f 1 | sort -n | cut -f 2 > temp_file
followed by
temp_file > ~/.bash_history

This should preserve line order, while removing dups from the existing hist file.
May leave behind ghost time-stamps, if you have turned on time-stamps.
If you have that issue, comment, and I'll whip up a second clean up for those.

edited Apr 13 '17 at 12:37

Community

1

answered May 15 '13 at 11:59

demure

2,282
1
16
19

1

Great one-liners! Thanks. In my case, "temp_file > ~/.bash_history" didn't work, but "cat temp_file > ~/.bash_history". Also, you forgot a last "rm temp_file" :) – Eleno Dec 31 '13 at 18:09

score 0 · Answer 2 · answered May 15 '13 at 10:44

0

"erasedups" should do the trick, did you source .bashrc after your change?

Can you please echo $HISTCONTROL and tell us the output? Also: be sure you actually want to do that on .bashrc and not in .bash_profile instead...

Basically, what I'm suspecting is that you simply don't have HISTCONTROL on your environment.

answered May 15 '13 at 10:44

Marcos Marado

1

I changed to export HISTCONTROL=erasedups, opened a new tab, and used an inexistent command kaka. Cat the .bash_history before kaka was kaka kaka kaka, since I tried it 3 times before the change, after it became kaka kaka kaka kada – Psicofrenia May 15 '13 at 10:46
Btw, if I echo HISTCONTROL I get erasedups – Psicofrenia May 15 '13 at 10:49

No duplicated lines in bash history

2 Answers2