Optimal way to run a python script multiple time with different argument values

Question

I have a python script that takes as input ~20 arguments. I want to run this script multiple times with different values for the arguments each time. At the moment I use a basic bash script like the following (with more parameters and more different values for each parameter)

for com_adv_par18 in 0.288 0.289
do
  for com_adv_par19 in 0.288 0.289
  do
    for com_adv_par20 in 0.288 0.289
    do
    python alpha2.py $com_adv_par18 $com_adv_par19 $com_adv_par20
    done
  done
 done

I am worrying though that this is not the most optimal way to do it. Both coding and computing time wise . Could you propose any alternative method to insert the parameters and run the program more efficiently?

Thanks in advance.

Why don't you just import the script and run it's `main` function (whatever it's called) repeatedly with the given parameters? Note that your bash loop is working sequentially. — MisterMiyagi, Jul 06 '16 at 08:44

score 2 · Accepted Answer · answered Jul 06 '16 at 08:56

The answer to your question depends on a lot of things - a significant factor is the length of time each execution takes.

If you can refactor the alpha2.py script so that you can import it, then you could use a python wrapper script along these lines:

from alpha2 import do_something
from itertools import product

# define argument lists here, e.g. list1 = [0.288, 0.289], etc.

for args in product(list1, list2, list3):
    do_something(*args)

Each execution will still be sequential but the advantage of this approach is that you don't suffer the overhead of loading a new python instance for every combination of parameters.

Thanks, this seems exactly what I want. Each execution doesn't take much time so it is ok to run it sequential. The total execution time has decreased dramatically. Also the input arguments are easily manipulated. — Ilias Pan, Jul 06 '16 at 10:54

score 0 · Answer 2 · edited May 23 '17 at 12:08

0

Why not use another python script to process args and call the initial script as you want? See such threads as Run a python script from another python script, passing in args

edited May 23 '17 at 12:08

Community

1
1

answered Jul 06 '16 at 08:36

Tttt1228

389
4
11

Optimal way to run a python script multiple time with different argument values

2 Answers2