Using GNU parallel (YouTube tutorials)(youtube.com)

2 pointsby uranium2356 anni fa2 comments

http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

2 comments

We've briefly used parallel for an internal tool when the site went "multisite" (i.e. multiple parallel instances that sometimes need to all do the same thing, such as post-deployment tasks).

We currently have a patch to remove it, and instead use regular task backgrounding and the `wait` shell builtin.

The way output is buffered was not particularly conducive to what we wanted to achieve.

MachineMartin6 anni fa

THE way? It sounds as if GNU Parallel only buffers in one way.

GNU Parallel can handle output in a few ways:

  * --ungroup: With -u output GNU Parallel does not touch the output and output may mix
  * --line-buffer: This buffers a single line from each process.
  * --group: This buffers in files in $TMPDIR. $TMPDIR can point a RAM disk, if you want to buffer in RAM.
  * --sqlworker: This buffers in memory before saving output to an SQL server.

If you are using '&' + 'wait', it pretty much has the same effect as `--ungroup`.