Bash Parallel


Jobs in GNU Linux can be parallelized using GNU parallel. A job can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. A job can also be a command that reads from a pipe.


  1. parallel [options] [command [arguments]] < list_of_arguments >


-j nRun n jobs in parallel
-kKeep same order
-XMultiple arguments with context replace
--colsep regexpSplit input on regexp for positional replacements
{} {.} {/} {/.} {#}Replacement strings
{3} {3.} {3/} {3/.}Positional replacement strings
-S sshloginExample:
--trc {}.barShorthand for --transfer --return {}.bar --cleanup
--onallRun the given command with argument on all sshlogins
--nonallRun the given command with no arguments on all sshlogins
--pipeSplit stdin (standard input) to multiple jobs.
--recend strRecord end separator for --pipe.
--recstart strRecord start separator for --pipe.