yes 1000000 | parallel -j10 seq | pv >/dev/null
yes 1000000 | parallel -kj10 "echo 'This is double spaced '{#}; seq {}" | pv >/dev/null parallel -k --tag --argsep -- {} echo ::: 1 -- parallel-*
parallel --tmux ... cat hosts.txt | parallel --quote --timeout=10 ssh {} 'echo {} $(md5sum ~/.config/file)'
parallel --slf hosts.txt --timeout=10 --nonall --tag md5sum .config/file some command | xargs -P other command | third command
* does not buffer stderr
* does not check if the disk is full for a period of time during a task (thus risking incomplete output)
* does not clean up, if killed
* does not work correctly if task_with_stdout is a composed command
I included the example:
to give you some fixed "goalposts" to aim for: Provide a solution that gives the same output byte for byte.
Also you do not seem to get the point about the amount of data. I regularly have output from a single job that is bigger than RAM, but rarely have output from a single job that would fill /tmp. However, the total combined output from all the jobs will often take up more space than /tmp.
In numbers: RAM=32 GB, /tmp=400 GB, a single job=33 GB, number of jobs=1000, jobs in parallel=8.
In other words: Running all jobs and saving the outputs into files before outputting data will not be useful for me. If you want to use FIFOs I really cannot see how you can deal with output that is bigger than RAM, unless you mix output from different jobs - which again would not be useful to me. But prove me wrong by spending 5 minutes on building the solution.
As for your example:
How would you design this, if output from different jobs are not allowed to mix?
If they are allowed to mix paralel gives you:
none of these use space in /tmp.
I sit back with the feeling you are willing to spend hours complaining, but not 5 minutes on proving your assertion that it can be done "easy-ish".
Prove me wrong: Spend 5 minutes on the task you believed was "easy-ish".
If it cannot be done in 5 minutes, be brave enough to admit you were wrong.