View Source: PerlOneLiners - Waikato Linux Users Group

Edit PageHistory Diff Info LikePages
Many oneliners rely on the magic of [Perl]'s <tt>-i</tt> switch, which means when files supplied on the commandline are opened they are edited in place, and if an extension was passed, a backup copy with that extension will be made. <tt>-e</tt> specifies the [Perl] code to run. See perlrun(1) for any other switches used here - in particular, <tt>-n</tt> and <tt>-p</tt> make powerful allies for <tt>-i</tt>.

The unsurpassed power of [Perl]'s RegularExpression flavour contributes a great deal to the usefulness of nearly every oneliner, so you will also want to read the perlretut(1) and perlre(1) manpages to learn about it.

!! Perl pie

<verbatim>
perl -pi -e 's/foo/bar/' file
</verbatim>

Does an inplace [SED] on the file.  [GNU] sed(1) v4 also supports this with <tt>-i</tt> and will probably be quicker if you only need a simple query and replacement.  However, [Perl]'s [RegularExpression]s are more powerful and easier on the hands than the [POSIX] variety offered by [SED].  With [GNU] sed(1), you can use the <tt>-r</tt> switch to get an extended RegularExpression syntax which also requires fewer backslashes than the [POSIX] flavour.

!! Removing empty lines from a file

<verbatim>
perl -ni.bak -e'/\S/ && print' file1 file2
</verbatim>

In [Shell]:

<verbatim>
for FILE in file1 file2 ; do mv "$F"{,.bak} ; grep '[^ ]' "$F.bak" > "$F" ; done
</verbatim>

!! Collapse consecutive blank lines to a single one

<verbatim>
perl -00 -pi.bak -e1 file1 file2
</verbatim>

Note the use of <tt>1</tt> as a no-op piece of Perl code. In this case, the <tt>-00</tt> and <tt>-p</tt> switches already do all the work, so only a dummy needs to be supplied.

!! Binary dump of a string

<verbatim>
perl -e 'printf "%08b\n", $_ for unpack "C*", shift' 'My String'
</verbatim>

!! Replace literal "\n" and "\t" in a file with newlines and tabs

<verbatim>
perl -pe 's!\\n!\n!g; s!\\t!\t!g' $file
</verbatim>

Note that you can use any punctuation as the separator in an <tt>s///</tt> command, and if you have backslashes or even need literal slashes in your pattern then doing this can increase clarity.

!! Convert data from rows to columns
This assumes that each of the input rows is exactly the same length (in terms of number of items), and assumes they are separated by spaces. This is useful if you have data in tabular form, but need it to be in columns instead (eg. you want to use it as input to GnuPlot).

<verbatim>
perl -e '@rows=();
 while ($l=<>) {@line=split(/ \s+/,$l);  push @rows, [@line ]}
 for $i (0 .. @{$rows[0]}) {
   for $row (@rows) {print $row->[$i] . "\t"}
   print "\n"
 }'
</verbatim>

Alternatively you can let [Perl] do the drudgework work for you. In the following, <tt>-n</tt> implies the <tt>while(<>){}</tt> loop and the <tt>-a -F''regex''</tt> imply the <tt>split</tt> (the result is stored in the predefined <tt>@F</tt> array). Anyone who is at all familiar with [AWK] should follow along easily.

<verbatim>
perl -aF'\s+' -ne'push @rows, [ @F ]; END {
    for $i ( 0 .. $#{ $rows[0] } ) {
        for $cols ( @rows ) { print $cols->[ $i ] . "\t" }
        print "\n"
    }
}'
</verbatim>

Both of these will read whitespace-separated tabular data from stdin(3) or from the files passed, and will write to tab-separated tabular data to stdout(3).

!! List all currently running processes

This is useful if you suspect that ps(1) is not reliable, whether due to a RootKit or some other cause. It prints the process ID and command line of every running process on the system (except some "special" kernel processes that lie about/don't have command lines).

<verbatim>
perl -0777 -pe 'BEGIN { chdir "/proc"; @ARGV = sort { $a <=> $b } glob("*/cmdline") }
    $ARGV =~ m!^(\d+)/!; print "$1\t"; s/\0/ /g; $_ .= "\n";'
</verbatim>

It runs an implicit loop over the <tt>/proc/*/cmdline</tt> files, by priming <tt>@ARGV</tt> with a list of files sorted numerically (which needs to be done explicitly using <tt><=></tt> -- the default sort is [ASCII]betical) and then employing the <tt>-p</tt> switch. <tt>-0777</tt> forces files to be slurped wholesale. Per file, the digits that lead the filename are printed, followed by a tab. Since a null separates the arguments in these files, all of them are replaced by spaces to make the output printable. Finally, a newline is appended. The print call implicit in the <tt>-p</tt> switch then takes care of outputting the massaged command line.


!! List all currently running processes, but nicer

See above for what this does. The ''how'' is different, though.

<verbatim>
perl -MFile::Slurp -0le 'for(sort { $a <=> $b } grep !/\D/, read_dir "/proc")
    { @ARGV = "/proc/$_/cmdline"; printf " %6g %s\n", $_, join(" ", <>); }'
</verbatim>

This time the loop is explicit. Again, there are two parts to the program -- selecting files and doing [I/O] on them.

To read the directory, a convenience function is pulled in from the File::Slurp module, loaded using the <tt>-M</tt> switch. The module has been part of the core distribution since Perl 5.8.0. Reading a directory manually is straightforward but the code would be longer and clumsier.

Selecting the files is pretty simple, if a little obtuse: it's done by reading the contents of <tt>/proc</tt>, then using <tt>grep !/D/</tt> to reject any entries that contain non-digit characters from the list. The results are then sorted numerically, which needs to be done explicitly using the <tt><=></tt> operator because the default sort is [ASCII]betical. Each entry is then interpolated into a full path and stuck into <tt>@ARGV</tt> one by one, from where the <tt><></tt> "diamond operator" will pick it up, auto-open it and read it for us, even autoreporting any errors in a nicely verbose format.

Producing human-readable output is a little more involved, using switches to abbreviate a bit of magic. The <tt>-0</tt> switch sets the <tt>$/</tt> variable: here, because the switch is not followed by a digit, it sets the variable to a null character. This means that null characters will be regarded as line separators on input. The <tt>-l</tt> switch has two effects, of which only one is relevant to us: it automatically removes line terminators from lines read using the diamond operator. (The other is to set the <tt>$\</tt> variable, which we aren't interested in or affected by.)

    Note that the <tt>-0</tt> and <tt>-l</tt> switches are order sensitive both in syntax and semantics. We order them for syntax here, because they can both accept an octal number as input, but we don't to pass one to either of them (particularly, <tt>-0</tt> will be mistaken for a digit parameter to <tt>-l</tt> if we turn them around).

Together, these switches effectively mean that we get null terminated lines from files, with the nulls removed on input. So we get the command line arguments listed in a <tt>/proc/*/cmdline</tt> file as nice list of separate strings. And because <tt>join()</tt> expects a list, <tt><></tt> returns all "lines" (ie command line arguments) at once, which <tt>join()</tt> then dutifully puts together with spaces between.

The printf(3) is straightforward.
4 pages link to PerlOneLiners:
Last edited on Friday, May 26, 2006 10:29:28 am by CraigBox
Edit PageHistory Diff Info LikePages