2

I'm currently launching a shell file on Gromacs (Linux) with 60 different commands in it. When launching the file the commands are processed sequentially one by one.

I want to launch all of them in parallel and in background.
How can I achieve that?

Here some lines of my shell file:

gmx rms -s md.tpr -n index_analysis.ndx -f md.xtc -o rmsd_fc_chain_B.xvg -tu ns << eof
29
29
eof

gmx rms -s md.tpr -n index_analysis.ndx -f md.xtc -o rmsd_fc_chain_C.xvg -tu ns << 
eof
32
32
eof

gmx rms -s md.tpr -n index_analysis.ndx -f md.xtc -o rmsd_fc_chain_D.xvg -tu ns << 
eof
35
35
eof
hek2mgl
  • 143,113
  • 25
  • 227
  • 253
philDDD
  • 23
  • 2

3 Answers3

6

If you wanted to have the full benefit of a tool explicitly written for this type of thing, you could use GNU Parallel. You would probably change your shell script (let's call it commands) to look like this:

printf "29\n29\n" | gmx rms -s md.tpr -n index_analysis.ndx -f md.xtc -o rmsd_fc_chain_B.xvg -tu ns
printf "32\n32\n" | gmx rms -s md.tpr -n index_analysis.ndx -f md.xtc -o rmsd_fc_chain_C.xvg -tu ns

Then you could run:

parallel -j 8 < commands

if you wanted to run 8 at a time, or -j 32 if you wanted to run 32 at a time. You can test what it would do, without actually doing anything, by using --dry-run:

parallel --dry-run < cmds
printf "29\n29\n" | gmx rms -s md.tpr -n index_analysis.ndx -f md.xtc -o rmsd_fc_chain_B.xvg -tu ns
printf "32\n32\n" | gmx rms -s md.tpr -n index_analysis.ndx -f md.xtc -o rmsd_fc_chain_C.xvg -tu ns

You can also get an Estimated Time of Arrival with --eta, and also spread the jobs over any other computers on your network that you can ssh into. You get the idea - very flexible.

Mark Setchell
  • 169,892
  • 24
  • 238
  • 370
1

run them in the background using &, e.g. 

sleep 60 &
sleep 60 &
sleep 60 &

will run 3 times sleep in background (and they are executed in parallel). See e.g. http://www.cyberciti.biz/faq/linux-command-line-run-in-background/ or executing shell command in background from script

For the combination in use with the here_doc : Running script with HERE_DOC method in background . This should work (only one command shown):

gmx rms -s md.tpr -n index_analysis.ndx -f md.xtc -o rmsd_fc_chain_B.xvg -tu ns << eof &
29
29
eof
Community
  • 1
  • 1
Raphael Roth
  • 25,362
  • 13
  • 78
  • 128
0

Based on the command you have written you might want to run something like (test all A..Z with all 1..100):

gmx rms -s md.tpr -n index_analysis.ndx -f md.xtc -o rmsd_fc_chain_{A..Z}.xvg -tu ns << 
eof
{1..100}
eof

Using GNU Parallel you would do this:

doit() {
  printf "$2\n$2\n" |
  gmx rms -s md.tpr -n index_analysis.ndx -f md.xtc -o rmsd_fc_chain_${1}.xvg -tu ns
}
export -f doit
parallel doit ::: {A..Z} ::: {1..100}

Learn more about GNU Parallel by walking through the tutorial: https://www.gnu.org/software/parallel/parallel_tutorial.html

Ole Tange
  • 29,397
  • 5
  • 75
  • 92