24.10 zloop: Run a Command on Compressed Files

The good thing about compressing files (24.7) is that it saves disk space. The bad thing is that if there are lots of compressed files you want to access separately, typing all those gzcat (or zcat) commands can get tedious and waste time.

I wrote a script named zloop that takes a command you want to run and a list of compressed files. It runs gzcat on each file, separately, and pipes each gzcat output to the command you gave. Because gzcat also understands compress format, it can handle .Z files too. The script shows the command line it ran and the output (if any) of the command. If the command returned nonzero status (44.7), zloop prints a warning.

% ls
185.gz  187.gz  189.gz  191.gz  193.gz  195.gz  197.gz
186.gz  188.gz  190.gz  192.gz  194.gz  196.gz  198.gz
% zloop 'egrep "^Subject:.*group"' *.gz

==== zloop: zcat 185.gz | egrep "^Subject:.*group" ====
Subject: List of Active Newsgroups

==== zloop: zcat 186.gz | egrep "^Subject:.*group" ====
Subject: Alternative Newsgroup Hierarchies

==== zloop: zcat 187.gz | egrep "^Subject:.*group" ====
zloop: note: that command returned 1 (non-zero) status:
   '/usr/local/bin/gzcat 187.gz | egrep "^Subject:.*group"'

==== zloop: zcat 188.gz | egrep "^Subject:.*group" ====
Subject: Checkgroups message (with INET groups)
Subject: Checkgroups message (without INET groups)
Subject: Monthly checkgroups posting
   ...

zloop is sort of verbose for a UNIX command - but you can make it quieter by editing the script. The status messages are sent to standard error. So, if you want to send zloop output through a pager like more, tell the shell to merge standard output and standard error:

|& 2>&1	.\ % zloop 'egrep "^Subject:.*group"' *.gz |& more csh $ zloop 'egrep "^Subject:.*group"' *.gz 2>&1 | more sh

With a plain pipe (|), status messages and command output can be jumbled (13.4).

In case it isn't clear: when you redirect the output of zloop, you're redirecting the output of all the commands that zloop runs - i.e., typing this command:

tr -2	% zloop 'tr "[A-Z]" "[a-z]" | pr -2' *.gz > toprint

is like typing these commands by hand:

(	( gzcat file1.gz | tr "[A-Z]" "[a-z]" | pr -2 gzcat file2.gz | tr "[A-Z]" "[a-z]" | pr -2 gzcat file3.gz | tr "[A-Z]" "[a-z]" | pr -2 ) > toprint