xargs with multiple arguments

Question

I have a source input, input.txt

a.txt
b.txt
c.txt

I want to feed these input into a program as the following:

my-program --file=a.txt --file=b.txt --file=c.txt

So I try to use xargs, but with no luck.

cat input.txt | xargs -i echo "my-program --file"{}

It gives

my-program --file=a.txt
my-program --file=b.txt
my-program --file=c.txt

But I want

my-program --file=a.txt --file=b.txt --file=c.txt

Any idea?

Answer 1

Don't listen to all of them. :) Just look at this example:

echo argument1 argument2 argument3 | xargs -l bash -c 'echo this is first:$0 second:$1 third:$2'

Output will be:

this is first:argument1 second:argument2 third:argument3

Answer 2

None of the solutions given so far deals correctly with file names containing space. Some even fail if the file names contain ' or ". If your input files are generated by users, you should be prepared for surprising file names.

GNU Parallel deals nicely with these file names and gives you (at least) 3 different solutions. If your program takes 3 and only 3 arguments then this will work:

(echo a1.txt; echo b1.txt; echo c1.txt;
 echo a2.txt; echo b2.txt; echo c2.txt;) |
parallel -N 3 my-program --file={1} --file={2} --file={3}

Or:

(echo a1.txt; echo b1.txt; echo c1.txt;
 echo a2.txt; echo b2.txt; echo c2.txt;) |
parallel -X -N 3 my-program --file={}

If, however, your program takes as many arguments as will fit on the command line:

(echo a1.txt; echo b1.txt; echo c1.txt;
 echo d1.txt; echo e1.txt; echo f1.txt;) |
parallel -X my-program --file={}

Watch the intro video to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ

Answer 3

How about:

echo $'a.txt\nb.txt\nc.txt' | xargs -n 3 sh -c '
   echo my-program --file="$1" --file="$2" --file="$3"
' argv0

Answer 4

It's simpler if you use two xargs invocations: 1st to transform each line into --file=..., 2nd to actually do the xargs thing ->

$ cat input.txt | xargs -I@ echo --file=@ | xargs echo my-program
my-program --file=a.txt --file=b.txt --file=c.txt

Answer 5

You can use sed to prefix --file= to each line and then call xargs:

sed -e 's/^/--file=/' input.txt | xargs my-program

Answer 6

Here is a solution using sed for three arguments, but is limited in that it applies the same transform to each argument:

cat input.txt | sed 's/^/--file=/g' | xargs -n3 my-program

Here's a method that will work for two args, but allows more flexibility:

cat input.txt | xargs -n 2 | xargs -I{} sh -c 'V="{}"; my-program -file=${V% *} -file=${V#* }'

Answer 7

I stumbled on a similar problem and found a solution which I think is nicer and cleaner than those presented so far.

The syntax for xargs that I have ended with would be (for your example):

xargs -I X echo --file=X

with a full command line being:

my-program $(cat input.txt | xargs -I X echo --file=X)

which will work as if

my-program --file=a.txt --file=b.txt --file=c.txt

was done (providing input.txt contains data from your example).

---

Actually, in my case I needed to first find the files and also needed them sorted so my command line looks like this:

my-program $(find base/path -name "some*pattern" -print0 | sort -z | xargs -0 -I X echo --files=X)

Few details that might not be clear (they were not for me):

• some*pattern must be quoted since otherwise shell would expand it before passing to find.
• -print0, then -z and finally -0 use null-separation to ensure proper handling of files with spaces or other wired names.

Note however that I didn't test it deeply yet. Though it seems to be working.

Answer 8

xargs doesn't work that way. Try:

  myprogram $(sed -e 's/^/--file=/' input.txt)

Answer 9

It's because echo prints a newline. Try something like

echo my-program `xargs --arg-file input.txt -i echo -n " --file "{}`

Answer 10

Actually, it's relatively easy:

... | sed 's/^/--prefix=/g' | xargs echo | xargs -I PARAMS your_cmd PARAMS

The sed 's/^/--prefix=/g' is optional, in case you need to prefix each param with some --prefix=.

The xargs echo turns the list of param lines (one param in each line) into a list of params in a single line and the xargs -I PARAMS your_cmd PARAMS allows you to run a command, placing the params where ever you want.

So cat input.txt | sed 's/^/--file=/g' | xargs echo | xargs -I PARAMS my-program PARAMS does what you need (assuming all lines within input.txt are simple and qualify as a single param value each).

Answer 11

There is another nice way of doing this, if you do not know the number of files upront:

my-program $(find . -name '*.txt' -printf "--file=%p ")

Answer 12

I was looking for a solution for this exact problem and came to the conclution of coding a script in the midle.

to transform the standard output for the next example use the -n '\n' delimeter

example:

 user@mybox:~$ echo "file1.txt file2.txt" | xargs -n1 ScriptInTheMiddle.sh

 inside the ScriptInTheMidle.sh:
 !#/bin/bash
 var1=`echo $1 | cut -d ' ' -f1 `
 var2=`echo $1 | cut -d ' ' -f2 `
 myprogram  "--file1="$var1 "--file2="$var2 

For this solution to work you need to have a space between those arguments file1.txt and file2.txt, or whatever delimeter you choose, one more thing, inside the script make sure you check -f1 and -f2 as they mean "take the first word and take the second word" depending on the first delimeter's position found (delimeters could be ' ' ';' '.' whatever you wish between single quotes . Add as many parameters as you wish.

Problem solved using xargs, cut , and some bash scripting.

Cheers!

if you wanna pass by I have some useful tips http://hongouru.blogspot.com

Answer 13

Nobody has mentioned echoing out from a loop yet, so I'll put that in for completeness sake (it would be my second approach, the sed one being the first):

for line in $(< input.txt) ; do echo --file=$line ; done | xargs echo my-program