Bash Process substitution Stream a file through multiple programs at once

Help us to keep this website almost Ad Free! It takes only 10 seconds of your time:
> Step 1: Go view our video on YouTube: EF Core Bulk Extensions
> Step 2: And Like the video. BONUS: You can also share it!

Example

This counts the number of lines in a big file with wc -l while simultaneously compressing it with gzip. Both run concurrently.

tee >(wc -l >&2) < bigfile | gzip > bigfile.gz

Normally tee writes its input to one or more files (and stdout). We can write to commands instead of files with tee >(command).

Here the command wc -l >&2 counts the lines read from tee (which in turn is reading from bigfile). (The line count is sent to stderr (>&2) to avoid mixing with the input to gzip.) The stdout of tee is simultaneously fed into gzip.



Got any Bash Question?