6

I appologize for asking this, but I really am really bad with regex...

Can someone help me transform the headers of my fasta files from this:

>Agaricus_chiangmaiensis|JF514531|SH174817.07FU|reps|k__Fungi;p__Basidiomycota;c__Agaricomycetes;o__Agaricales;f__Agaricaceae;g__Agaricus;s__Agaricus_chiangmaiensis

to this:

>JF514531|SH174817.07FU;tax=d:Fungi,p:Basidiomycota,c:Agaricomycetes,o:Agaricales,f:Agaricaceae,g:Agaricus,s:Agaricus_chiangmaiensis_SH174817.07FU;

seems like I could use the command 

sed 's/__/:/g;s/;/,/g;s/.*|/|/g'

to substitute : for __, and , for ;. I'm not entirely sure how to proceed from there though...

Thank you

terdon
  • 10,071
  • 5
  • 22
  • 48
Graham
  • 63
  • 5
  • Could you [edit] your question and explain the transformation you need please? You don't only want to replace __ with : and ; with ,. You also seem to be removing some text. Do you always want to remove whatever is before the first |? And what happened to the reps? And why is k__Fungi becoming tax=d:Fungi instead of k:Fungi? We need to know exactly what you are trying to do in order to give an answer that works for you. – terdon Jan 19 '19 at 13:11
  • Looks like you figured it out without more clarification - I'll try to be more concise when asking in the future - – Graham Jan 22 '19 at 17:35

2 Answers2

5

If awk is good as well:

$ awk -v FS="|" -v OFS="|" '/>/{
    gsub(";",",", $5); 
    gsub("__",":",$5); 
    print ">"$2,$3";tax="$5"_"$3";";
    next;
}1' file.fa
>JF514531|SH174817.07FU;tax=k:Fungi,p:Basidiomycota,c:Agaricomycetes,o:Agaricales,f:Agaricaceae,g:Agaricus,s:Agaricus_chiangmaiensis_SH174817.07FU;

This relies on the order of the parts separated by |.

I don't understand why k__Fungi becomes d:Fungi, so I ignored it for now.

Some explanation for what awkis doing.

  • | is defined as a field (column) separator for the input and output
  • If the line contains a > the code between { and } is executed.
  • the first gsub replaces ; with , in the 5th field
  • the second gsub replaces __ with : in the 5th field
  • then we print the new header line. The $2, $3 and $5 get replaced by the values of the second, third and 5th field of the input. , get replaced by the defined output field separator
  • with next we tell awk to go to the next input line and repeat the code
  • The 1 is in equivalent to print $0 for printing the whole input line. In our code this is only executed if the line is no header line.
finswimmer
  • 1,342
  • 7
  • 15
3

It seems like you want to do four things:

  1. Remove all text up to and including the first |.
  2. Now that that has been removed, take the string between what is now (after step 1) the first | and what is now the second |, and append it to the end of the header with a _.
  3. Replace all __ with : and all ; with ,.
  4. Change the (fixed?) string reps|k with tax=d.

If so, and assuming that you only have one occurrence of reps| in each header, you can do:

$ sed 's/>[^|]*|/>/; 
       s/reps|k/tax=d/; 
       s/__/:/g; 
       s/;/,/g; 
       s/|\([^|]*\)\(|.*\)/|\1\2_\1;/; ' file.fa 
>JF514531|SH174817.07FU;tax=d:Fungi,p:Basidiomycota,c:Agaricomycetes,o:Agaricales,f:Agaricaceae,g:Agaricus,s:Agaricus_chiangmaiensis_SH174817.07FU;

This will:

  • s/>[^|]*|/>/ : replace the longest stretch of non-| characters after a > and until the first | with > alone.
  • s/reps|k/tax=d/; : replace the first occurrence of the string reps|k with tax=.
  • s/__/:/g; s/;/,/g : replace all (that's what the /g means) occurrences of __ with : and of ; with ,.

  • s/|\([^|]*\)\(|.*\)/|\1\2_\1;/ : the \(foo\) construct is called a capturing group and will capture foo and save it as \1, which can be used on the right hand side of the replacement operator. The next captured group will be \2 then \3 etc. Here, with |\([^|]*\) we are capturing the longest string of non| found after the 1st | (this happens after the first operation, so the 1st | is the one that was 3rd in the original) and then everything after the next | (|.*) until the end of the line. We then replace everything matched with a |, then whatever was matched by the first capture group (\1), then the second (\2), then we add a _ and repeat the \1. Finally, if you want to make the changes to the original input file, use -i:

    sed -i.bak 's/>[^|]*|/>/; s/reps|k/tax=d/; s/__/:/g; s/;/,/g; s/|\([^|]*\)\(|.*\)/|\1\2_\1;/; ' file.fa

    That will save the original file as file.fa.bak. use -i alone without any argument if you don't need the backup.

terdon
  • 10,071
  • 5
  • 22
  • 48
  • This works really well, except for two things: the command changes reps to tax for every other header, and there is still a | in front of tax instead of ;. – Graham Jan 22 '19 at 18:27