|
|
| Author |
Message |
cross
|
|
Unix tool o' the day.
| 
Sep 18 06:28 UTC 2010 |
Got a favorite Unix tool? Post about it here.
|
| 22 responses total. |
cross
|
|
response 1 of 22:
|
Sep 18 06:59 UTC 2010 |
Early versions of the Unix system were marked by an emphasis on what
came to be known as the tools philosophy: write small programs that
act as a filters, doing one thing and doing it well, and combining
these programs in pipelines in the shell.
It seems in recent years that that philosophy has fallen by the
wayside. As near as I can tell, this started with the introduction of
perl, which was a Swiss Army knife of a tool, capable of doing many
things (some very well, some, well, not so well). However, the basic
Unix tools still ship with almost every Unix (or derivitive) system,
and knowing how to use them effectively is worthwhile. Indeed, it is
possible to combine tools in surprising ways, creating complex yet
elegant systems entirely in shell: often programs that would take tens
or hundreds of lines of C or even Perl can be written in one or two
lines of shell. Often, new filters are prototyped this way.
In this item, talk about your favorite Unix tools: what they do, a
basic introduction to how to use them, and why you like them.
|
cross
|
|
response 2 of 22:
|
Sep 18 07:07 UTC 2010 |
To start, here's a list of some of the basic tools for manipulating
files:
ls - LiSt the files in the current directory (default)
Or as specified by command line arguments.
mv - Move a file to a new name; this is also used for renaming.
cp - Create a new copy of a file or files.
rm - Remove a file or files.
cat - ConCATenate files to the standard output. For each argument,
print its contents on stdout. Defaults to copying the standard
input stream to the standard output stream.
cd - Usually built into the shell, and sometimes called 'chdir'.
This changes the current working directory for the current
process to the specified directory. Defaults to changing
to the user's home directory.
chmod - CHange file MODe. Change the permissions of a file.
chown - CHange file OWNer. Change what user owns a file.
chgrp - CHange file GRouP. Change what group owns a file.
These, and a handful of others, are the basic tools for working with
files. Most of them take many and varied options, some of greater or
lesser utility. Indeed, a general proliferation of options to basic
Unix tools prompted Brian Kernighan and Rob Pike to write
the, "Program Design in the UNIX Environment" paper and given a
presentation at the 1983 Summer USENIX conference called, "UNIX Style,
or cat -v Considered Harmful."
|
tod
|
|
response 3 of 22:
|
Sep 18 20:39 UTC 2010 |
vi
The best editor out there. (Shut your EMACS moufs!) ;)
|
cross
|
|
response 4 of 22:
|
Sep 19 08:55 UTC 2010 |
Hey! There's already a text editor holy war item!
Today's tool of the day is: look(1).
Look takes a sorted file (which defaults to the system dictionary
file) and a prefix, and prints all the strings in the file that start
with that prefix. It does this using a binary search algorithm (hence
the reason the file be sorted. One of the preconditions of a binary
search is that the data be ordered in some meaningful way; since in
general look is looking things up lexiographically, it expects the
data to be sorted that way), and can be quite speedy. It's not so
much of a filter, but can be used at the beginning of a pipeline as a
data source.
|
remmers
|
|
response 5 of 22:
|
Sep 21 22:15 UTC 2010 |
I don't know if ssh is commonly considered a "tool" in the same way
that ls and cat are, but I know that it and its relative scp can be
used in pipelines and other Unixy constructs for processing remote
data locally, without establishing a remote "session" in the usual
sense. A few (probably silly) examples off the top of my
head of things I could do at a local shell prompt in a terminal
window on my laptop. (What the examples actually do is left as
an exercise for the reader.)
(1) ssh grex.org last | grep '\.msu\.edu ' | wc -l
(2) ssh grex.org cat '/bbs/agora50/_*'|grep '^,U.*,cross$'|wc -l
(3) for f in `ssh grex.org ls /bbs/agora33/_*`
do scp grex.org:$f .
done
Yes, I'm sure there are better ways of doing all the above (example 3
is particularly hideous), but I'm just illustrating a point here.
|
cross
|
|
response 6 of 22:
|
Sep 22 12:03 UTC 2010 |
resp:5 Nice. Certainly, example (3) could be rewritten as:
scp 'grex.org:/bbs/agora33/_*' .
I'm pretty sure that's what John meant when he said it was
particularly hideous.
Btw: I don't know if you're using Bash or some such shell, but it may
not be a bad idea to quote the "*" in the back-tick'ed part of the for
loop in (3).
An often handy flag for ssh is '-C', which compresses the data
stream. That can speed up things like this, at the expense of a
little more CPU and RAM usage on either end of the connection.
If you don't mind running grep on Grex instead of the local end of
your connection, you could substitute 'grep' for 'cat' in (1). There
used to be people on USENET who complained about "useless uses of cat"
and there was an award. However, note how this differs: the pipeline
is spread across the network, and there may be legitimate reasons that
one does not want to run grep on Grex ("cat" is a much, much simpler
program). There are legitimate reasons to use cat in pipelines; this
could be one of them.
I think that John's examples are very much in the spirit of Unix tool
usage; 'ssh' certainly counts in this case, particularly for its
ability to spread computation across the network. That's an oft-
neglected capability, usually overlooked in favor of people who simply
want to use it interactively. I claim that using, e.g., 'scp' is
certainly more "Unixy" than, say, the sftp or ftp commands.
|
cross
|
|
response 7 of 22:
|
Sep 22 12:33 UTC 2010 |
Today's Unix tool o' the day is: ed. The Standard Editor.
And that's what it is: ed is a line oriented text editor. One uses it
by running 'ed file' and then typing in commands from the ed command
set. See the ed(1) manual page, or the book, "The Unix Programming
Environment" for information on how to use ed.
Ed is not a graphical editor; it does not display a full screen-full
of information at a time. In fact, it won't display any part of the
file unless you tell it to by running special ed commands; ed was
designed for use on printing teletype terminals (ie, terminals that
output to an actual typewriter-like device, instead of a graphic
screen). Its diagnostics are horrible in their simplicity: usually,
if you do something ed does not understand, you get a line with the
single character '?' on it, all by itself. Some have called it
the, "What you get is what you get" or WYGIWYG text editor.
Ed is probably the first Unix text editor. I say "probably" because
it's certainly been rewritten several times, and they tried several
variants out at Bell Labs. It is descended from the QED editor that
was used on the Multics system and CTSS. Ed was implemented mostly by
Ken Thompson in the early 1970s, and went on to inspire the Sam and
acme text editors written by Rob Pike. It is called "the Standard
Editor" because it was the only traditional text editor that came with
early Unix distributions.
So, ed is not graphical, has horrible diagnostics, an arcane command
set, and was designed for 1960s standards, why on earth would anyone
want to learn it now?
Because it's tremendously useful as an editing tool for in-place
files. Given that it reads commands from standard input, it can also
be used as a target in a pipeline: one can write scripts that generate
ed commands that are then piped into ed running on a file; this is
handy for batch-mode editing without using temporary files (ed might
make a temporary file, but the script-writer won't have to worry about
that).
Have you got 1000 files in a directory that all have to have the
word "UNIX" changed to "Unix Operating System" in them? Here's a
simple way:
for file in *.html
do
(echo 'g/UNIX/s/UNIX/Unix Operating System/g'; echo w; echo q) |
ed $file
done
Done. Note, some versions of ed(1) support chaining some commands
together, and some support combining the 'w' and 'q' commands into a
single 'wq', but the original ed does not.
|