File and Archiving CommandsArchiving - tar
The standard UNIX archiving utility. Originally a
Tape ARchiving program, it has
developed into a general purpose package that can handle
all manner of archiving with all types of destination
devices, ranging from tape drives to regular files to even
stdout (see Example 3-4). GNU
tar has been patched to accept various compression
filters, such as tar czvf archive_name.tar.gz
*, which recursively archives and gzips all files in a directory
tree except dotfiles
in the current working directory ($PWD).
Some useful tar options:
-c create (a new
archive) -x extract (files from
existing archive) --delete delete (files
from existing archive) | This option will not work on magnetic tape
devices. |
-r append (files to
existing archive) -A append
(tar files to
existing archive) -t list (contents of
existing archive) -u update archive -d compare archive with
specified filesystem -z gzip the archive (compress or uncompress, depending on whether
combined with the -c or
-x) option -j
bzip2 the
archive
| It may be difficult to recover data from a
corrupted gzipped tar
archive. When archiving important files, make multiple
backups. |
- shar
Shell archiving utility. The files in a shell archive
are concatenated without compression, and the resultant
archive is essentially a shell script, complete with
#!/bin/sh header, and containing all
the necessary unarchiving commands. Shar archives
still show up in Internet newsgroups, but otherwise
shar has been pretty well replaced by
tar/gzip. The
unshar command unpacks
shar archives. - ar
Creation and manipulation utility for archives, mainly
used for binary object file libraries. - rpm
The Red Hat Package Manager, or
rpm utility provides a wrapper for
source or binary archives. It includes commands for
installing and checking the integrity of packages, among
other things. A simple rpm -i package_name.rpm
usually suffices to install a package, though there are many
more options available. | rpm -qf identifies which package a
file originates from. bash$ rpm -qf /bin/ls
coreutils-5.2.1-31
|
|
| rpm -qa gives a
complete list of all installed rpm packages
on a given system. An rpm -qa package_name
lists only the package(s) corresponding to
package_name. bash$ rpm -qa
redhat-logos-1.1.3-1
glibc-2.2.4-13
cracklib-2.7-12
dosfstools-2.7-1
gdbm-1.8.0-10
ksymoops-2.4.1-1
mktemp-1.5-11
perl-5.6.0-17
reiserfs-utils-3.x.0j-2
...
bash$ rpm -qa docbook-utils
docbook-utils-0.6.9-2
bash$ rpm -qa docbook | grep docbook
docbook-dtd31-sgml-1.0-10
docbook-style-dsssl-1.64-3
docbook-dtd30-sgml-1.0-10
docbook-dtd40-sgml-1.0-11
docbook-utils-pdf-0.6.9-2
docbook-dtd41-sgml-1.0-10
docbook-utils-0.6.9-2
|
|
- cpio
This specialized archiving copy command
(copy
input and output)
is rarely seen any more, having been supplanted by
tar/gzip. It still
has its uses, such as moving a directory tree. Example 12-27. Using cpio to move a directory tree #!/bin/bash
# Copying a directory tree using 'cpio.'
# Advantages of using 'cpio':
# Speed of copying. It's faster than 'tar' with pipes.
# Well suited for copying special files (named pipes, etc.)
#+ that 'cp' may choke on.
ARGS=2
E_BADARGS=65
if [ $# -ne "$ARGS" ]
then
echo "Usage: `basename $0` source destination"
exit $E_BADARGS
fi
source=$1
destination=$2
find "$source" -depth | cpio -admvp "$destination"
# ^^^^^ ^^^^^
# Read the 'find' and 'cpio' man page to decipher these options.
# Exercise:
# --------
# Add code to check the exit status ($?) of the 'find | cpio' pipe
#+ and output appropriate error messages if anything went wrong.
exit 0 |
- rpm2cpio
This command extracts a
cpio archive from an rpm one. Example 12-28. Unpacking an rpm archive #!/bin/bash
# de-rpm.sh: Unpack an 'rpm' archive
: ${1?"Usage: `basename $0` target-file"}
# Must specify 'rpm' archive name as an argument.
TEMPFILE=$$.cpio # Tempfile with "unique" name.
# $$ is process ID of script.
rpm2cpio < $1 > $TEMPFILE # Converts rpm archive into cpio archive.
cpio --make-directories -F $TEMPFILE -i # Unpacks cpio archive.
rm -f $TEMPFILE # Deletes cpio archive.
exit 0
# Exercise:
# Add check for whether 1) "target-file" exists and
#+ 2) it is really an rpm archive.
# Hint: parse output of 'file' command. |
Compression - gzip
The standard GNU/UNIX compression utility, replacing
the inferior and proprietary
compress. The corresponding decompression
command is gunzip, which is the equivalent of
gzip -d. The zcat filter decompresses a
gzipped file to
stdout, as possible input to a pipe or
redirection. This is, in effect, a cat
command that works on compressed files (including files
processed with the older compress
utility). The zcat command is equivalent to
gzip -dc. | On some commercial UNIX systems, zcat
is a synonym for uncompress -c,
and will not work on gzipped
files. |
See also Example 7-7. - bzip2
An alternate compression utility, usually more efficient
(but slower) than gzip, especially on
large files. The corresponding decompression command is
bunzip2. | Newer versions of tar have been patched with
bzip2 support. |
- compress, uncompress
This is an older, proprietary compression
utility found in commercial UNIX distributions. The
more efficient gzip has largely
replaced it. Linux distributions generally include a
compress workalike for compatibility,
although gunzip can unarchive files
treated with compress. | The znew command transforms
compressed files into
gzipped ones. |
- sq
Yet another compression utility, a filter that works
only on sorted ASCII word lists. It uses the standard
invocation syntax for a filter, sq < input-file >
output-file. Fast, but not nearly as efficient
as gzip. The corresponding
uncompression filter is unsq, invoked
like sq. | The output of sq may be
piped to gzip for further
compression. |
- zip, unzip
Cross-platform file archiving and compression utility
compatible with DOS pkzip.exe.
"Zipped" archives seem to be a more
acceptable medium of exchange on the Internet than
"tarballs". - unarc, unarj, unrar
These Linux utilities permit unpacking archives
compressed with the DOS arc.exe,
arj.exe, and
rar.exe programs.
File Information - file
A utility for identifying file types. The command
file file-name will return a
file specification for file-name,
such as ascii text or
data. It references
the magic numbers
found in /usr/share/magic,
/etc/magic, or
/usr/lib/magic, depending on the
Linux/UNIX distribution. The -f option causes
file to run in batch mode, to read from
a designated file a list of filenames to analyze. The
-z option, when used on a compressed
target file, forces an attempt to analyze the uncompressed
file type. bash$ file test.tar.gz
test.tar.gz: gzip compressed data, deflated, last modified: Sun Sep 16 13:34:51 2001, os: Unix
bash file -z test.tar.gz
test.tar.gz: GNU tar archive (gzip compressed data, deflated, last modified: Sun Sep 16 13:34:51 2001, os: Unix)
|
# Find sh and Bash scripts in a given directory:
DIRECTORY=/usrlocal/bin
KEYWORD=Bourne
# Bourne and Bourne-Again shell scripts
file $DIRECTORY/* | fgrep $KEYWORD
# Output:
# /usr/local/bin/burn-cd: Bourne-Again shell script text executable
# /usr/local/bin/burnit: Bourne-Again shell script text executable
# /usr/local/bin/cassette.sh: Bourne shell script text executable
# /usr/local/bin/copy-cd: Bourne-Again shell script text executable
# . . . |
Example 12-29. Stripping comments from C program files #!/bin/bash
# strip-comment.sh: Strips out the comments (/* COMMENT */) in a C program.
E_NOARGS=0
E_ARGERROR=66
E_WRONG_FILE_TYPE=67
if [ $# -eq "$E_NOARGS" ]
then
echo "Usage: `basename $0` C-program-file" >&2 # Error message to stderr.
exit $E_ARGERROR
fi
# Test for correct file type.
type=`file $1 | awk '{ print $2, $3, $4, $5 }'`
# "file $1" echoes file type . . .
# Then awk removes the first field of this, the filename . . .
# Then the result is fed into the variable "type".
correct_type="ASCII C program text"
if [ "$type" != "$correct_type" ]
then
echo
echo "This script works on C program files only."
echo
exit $E_WRONG_FILE_TYPE
fi
# Rather cryptic sed script:
#--------
sed '
/^\/\*/d
/.*\/\*/d
' $1
#--------
# Easy to understand if you take several hours to learn sed fundamentals.
# Need to add one more line to the sed script to deal with
#+ case where line of code has a comment following it on same line.
# This is left as a non-trivial exercise.
# Also, the above code deletes lines with a "*/" or "/*",
#+ not a desirable result.
exit 0
# ----------------------------------------------------------------
# Code below this line will not execute because of 'exit 0' above.
# Stephane Chazelas suggests the following alternative:
usage() {
echo "Usage: `basename $0` C-program-file" >&2
exit 1
}
WEIRD=`echo -n -e '\377'` # or WEIRD=$'\377'
[[ $# -eq 1 ]] || usage
case `file "$1"` in
*"C program text"*) sed -e "s%/\*%${WEIRD}%g;s%\*/%${WEIRD}%g" "$1" \
| tr '\377\n' '\n\377' \
| sed -ne 'p;n' \
| tr -d '\n' | tr '\377' '\n';;
*) usage;;
esac
# This is still fooled by things like:
# printf("/*");
# or
# /* /* buggy embedded comment */
#
# To handle all special cases (comments in strings, comments in string
#+ where there is a \", \\" ...) the only way is to write a C parser
#+ (using lex or yacc perhaps?).
exit 0 |
- which
which command-xxx gives the full path
to "command-xxx". This is useful for finding
out whether a particular command or utility is installed
on the system. $bash which rm
- whereis
Similar to which, above,
whereis command-xxx gives the
full path to "command-xxx", but also to its
manpage. $bash whereis rm
rm: /bin/rm /usr/share/man/man1/rm.1.bz2 |
- whatis
whatis filexxx looks up
"filexxx" in the
whatis database. This is useful
for identifying system commands and important configuration
files. Consider it a simplified man
command. $bash whatis whatis
whatis (1) - search the whatis database for complete words |
Example 12-30. Exploring /usr/X11R6/bin #!/bin/bash
# What are all those mysterious binaries in /usr/X11R6/bin?
DIRECTORY="/usr/X11R6/bin"
# Try also "/bin", "/usr/bin", "/usr/local/bin", etc.
for file in $DIRECTORY/*
do
whatis `basename $file` # Echoes info about the binary.
done
exit 0
# You may wish to redirect output of this script, like so:
# ./what.sh >>whatis.db
# or view it a page at a time on stdout,
# ./what.sh | less |
See also Example 10-3. - vdir
Show a detailed directory listing. The effect is similar to
ls -l. This is one of the GNU fileutils. bash$ vdir
total 10
-rw-r--r-- 1 bozo bozo 4034 Jul 18 22:04 data1.xrolo
-rw-r--r-- 1 bozo bozo 4602 May 25 13:58 data1.xrolo.bak
-rw-r--r-- 1 bozo bozo 877 Dec 17 2000 employment.xrolo
bash ls -l
total 10
-rw-r--r-- 1 bozo bozo 4034 Jul 18 22:04 data1.xrolo
-rw-r--r-- 1 bozo bozo 4602 May 25 13:58 data1.xrolo.bak
-rw-r--r-- 1 bozo bozo 877 Dec 17 2000 employment.xrolo
|
- locate, slocate
The locate command searches for files using a
database stored for just that purpose. The
slocate command is the secure version of
locate (which may be aliased to
slocate). $bash locate hickson
/usr/lib/xephem/catalogs/hickson.edb |
- readlink
Disclose the file that a symbolic link points to. bash$ readlink /usr/bin/awk
../../bin/gawk
|
- strings
Use the strings command to find
printable strings in a binary or data file. It will list
sequences of printable characters found in the target
file. This might be handy for a quick 'n dirty examination
of a core dump or for looking at an unknown graphic image
file (strings image-file | more might
show something like JFIF,
which would identify the file as a jpeg
graphic). In a script, you would probably
parse the output of strings
with grep or sed. See Example 10-7
and Example 10-9. Example 12-31. An "improved" strings
command #!/bin/bash
# wstrings.sh: "word-strings" (enhanced "strings" command)
#
# This script filters the output of "strings" by checking it
#+ against a standard word list file.
# This effectively eliminates gibberish and noise,
#+ and outputs only recognized words.
# ===========================================================
# Standard Check for Script Argument(s)
ARGS=1
E_BADARGS=65
E_NOFILE=66
if [ $# -ne $ARGS ]
then
echo "Usage: `basename $0` filename"
exit $E_BADARGS
fi
if [ ! -f "$1" ] # Check if file exists.
then
echo "File \"$1\" does not exist."
exit $E_NOFILE
fi
# ===========================================================
MINSTRLEN=3 # Minimum string length.
WORDFILE=/usr/share/dict/linux.words # Dictionary file.
# May specify a different
#+ word list file
#+ of one-word-per-line format.
wlist=`strings "$1" | tr A-Z a-z | tr '[:space:]' Z | \
tr -cs '[:alpha:]' Z | tr -s '\173-\377' Z | tr Z ' '`
# Translate output of 'strings' command with multiple passes of 'tr'.
# "tr A-Z a-z" converts to lowercase.
# "tr '[:space:]'" converts whitespace characters to Z's.
# "tr -cs '[:alpha:]' Z" converts non-alphabetic characters to Z's,
#+ and squeezes multiple consecutive Z's.
# "tr -s '\173-\377' Z" converts all characters past 'z' to Z's
#+ and squeezes multiple consecutive Z's,
#+ which gets rid of all the weird characters that the previous
#+ translation failed to deal with.
# Finally, "tr Z ' '" converts all those Z's to whitespace,
#+ which will be seen as word separators in the loop below.
# ****************************************************************
# Note the technique of feeding the output of 'tr' back to itself,
#+ but with different arguments and/or options on each pass.
# ****************************************************************
for word in $wlist # Important:
# $wlist must not be quoted here.
# "$wlist" does not work.
# Why not?
do
strlen=${#word} # String length.
if [ "$strlen" -lt "$MINSTRLEN" ] # Skip over short strings.
then
continue
fi
grep -Fw $word "$WORDFILE" # Match whole words only.
# ^^^ # "Fixed strings" and
#+ "whole words" options.
done
exit $? |
Comparison - diff, patch
diff: flexible file comparison
utility. It compares the target files line-by-line
sequentially. In some applications, such as comparing
word dictionaries, it may be helpful to filter the
files through sort
and uniq before piping them
to diff. diff file-1
file-2 outputs the lines in the files that
differ, with carets showing which file each particular
line belongs to. The --side-by-side option to
diff outputs each compared file, line by
line, in separate columns, with non-matching lines marked. The
-c and -u options likewise
make the output of the command easier to interpret. There are available various fancy frontends for
diff, such as spiff,
wdiff, xdiff, and
mgdiff. | The diff command returns an exit
status of 0 if the compared files are identical, and 1 if
they differ. This permits use of diff
in a test construct within a shell script (see
below). |
A common use for diff is generating
difference files to be used with patch
The -e option outputs files suitable
for ed or ex
scripts. patch: flexible versioning
utility. Given a difference file generated by
diff, patch can
upgrade a previous version of a package to a newer version.
It is much more convenient to distribute a relatively
small "diff" file than the entire body of a
newly revised package. Kernel "patches" have
become the preferred method of distributing the frequent
releases of the Linux kernel. patch -p1 <patch-file
# Takes all the changes listed in 'patch-file'
# and applies them to the files referenced therein.
# This upgrades to a newer version of the package. |
Patching the kernel: cd /usr/src
gzip -cd patchXX.gz | patch -p0
# Upgrading kernel source using 'patch'.
# From the Linux kernel docs "README",
# by anonymous author (Alan Cox?). |
| The diff command can also
recursively compare directories (for the filenames
present). bash$ diff -r ~/notes1 ~/notes2
Only in /home/bozo/notes1: file02
Only in /home/bozo/notes1: file03
Only in /home/bozo/notes2: file04
|
|
| Use zdiff to compare
gzipped files. |
- diff3
An extended version of diff that compares
three files at a time. This command returns an exit value
of 0 upon successful execution, but unfortunately this gives
no information about the results of the comparison. bash$ diff3 file-1 file-2 file-3
====
1:1c
This is line 1 of "file-1".
2:1c
This is line 1 of "file-2".
3:1c
This is line 1 of "file-3"
|
- sdiff
Compare and/or edit two files in order to merge
them into an output file. Because of its interactive nature,
this command would find little use in a script. - cmp
The cmp command is a simpler version of
diff, above. Whereas diff
reports the differences between two files,
cmp merely shows at what point they
differ. | Like diff, cmp
returns an exit status of 0 if the compared files are
identical, and 1 if they differ. This permits use in a test
construct within a shell script. |
Example 12-32. Using cmp to compare two files
within a script. #!/bin/bash
ARGS=2 # Two args to script expected.
E_BADARGS=65
E_UNREADABLE=66
if [ $# -ne "$ARGS" ]
then
echo "Usage: `basename $0` file1 file2"
exit $E_BADARGS
fi
if [[ ! -r "$1" || ! -r "$2" ]]
then
echo "Both files to be compared must exist and be readable."
exit $E_UNREADABLE
fi
cmp $1 $2 &> /dev/null # /dev/null buries the output of the "cmp" command.
# cmp -s $1 $2 has same result ("-s" silent flag to "cmp")
# Thank you Anders Gustavsson for pointing this out.
#
# Also works with 'diff', i.e., diff $1 $2 &> /dev/null
if [ $? -eq 0 ] # Test exit status of "cmp" command.
then
echo "File \"$1\" is identical to file \"$2\"."
else
echo "File \"$1\" differs from file \"$2\"."
fi
exit 0 |
| Use zcmp on
gzipped files. |
- comm
Versatile file comparison utility. The files must be
sorted for this to be useful. comm
-options
first-file
second-file comm file-1 file-2 outputs three columns:
column 1 = lines unique to file-1 column 2 = lines unique to file-2 column 3 = lines common to both.
The options allow suppressing output of one or more columns.
Utilities - basename
Strips the path information from a file name, printing
only the file name. The construction basename
$0 lets the script know its name, that is, the name it
was invoked by. This can be used for "usage" messages if,
for example a script is called with missing arguments:
echo "Usage: `basename $0` arg1 arg2 ... argn" |
- dirname
Strips the basename from
a filename, printing only the path information. | basename and dirname
can operate on any arbitrary string. The argument
does not need to refer to an existing file, or
even be a filename for that matter (see Example A-7). |
Example 12-33. basename and dirname #!/bin/bash
a=/home/bozo/daily-journal.txt
echo "Basename of /home/bozo/daily-journal.txt = `basename $a`"
echo "Dirname of /home/bozo/daily-journal.txt = `dirname $a`"
echo
echo "My own home is `basename ~/`." # `basename ~` also works.
echo "The home of my home is `dirname ~/`." # `dirname ~` also works.
exit 0 |
- split, csplit
These are utilities for splitting a file into smaller
chunks. They are usually used for splitting up large files
in order to back them up on floppies or preparatory to
e-mailing or uploading them. The csplit command splits a file
according to context, the split occuring
where patterns are matched. - sum, cksum, md5sum
These are utilities for generating checksums. A
checksum is a number mathematically
calculated from the contents of a file, for the purpose
of checking its integrity. A script might refer to a list
of checksums for security purposes, such as ensuring that
the contents of key system files have not been altered
or corrupted. For security applications, use the 128-bit
md5sum (message
digest 5
checksum) command. bash$ cksum /boot/vmlinuz
1670054224 804083 /boot/vmlinuz
bash$ echo -n "Top Secret" | cksum
3391003827 10
bash$ md5sum /boot/vmlinuz
0f43eccea8f09e0a0b2b5cf1dcf333ba /boot/vmlinuz
bash$ echo -n "Top Secret" | md5sum
8babc97a6f62a4649716f4df8d61728f -
|
| The cksum command shows the size,
in bytes, of its target, whether file or
stdout. The md5sum command displays a
dash when it receives its input from
stdout. |
Example 12-34. Checking file integrity #!/bin/bash
# file-integrity.sh: Checking whether files in a given directory
# have been tampered with.
E_DIR_NOMATCH=70
E_BAD_DBFILE=71
dbfile=File_record.md5
# Filename for storing records (database file).
set_up_database ()
{
echo ""$directory"" > "$dbfile"
# Write directory name to first line of file.
md5sum "$directory"/* >> "$dbfile"
# Append md5 checksums and filenames.
}
check_database ()
{
local n=0
local filename
local checksum
# ------------------------------------------- #
# This file check should be unnecessary,
#+ but better safe than sorry.
if [ ! -r "$dbfile" ]
then
echo "Unable to read checksum database file!"
exit $E_BAD_DBFILE
fi
# ------------------------------------------- #
while read record[n]
do
directory_checked="${record[0]}"
if [ "$directory_checked" != "$directory" ]
then
echo "Directories do not match up!"
# Tried to use file for a different directory.
exit $E_DIR_NOMATCH
fi
if [ "$n" -gt 0 ] # Not directory name.
then
filename[n]=$( echo ${record[$n]} | awk '{ print $2 }' )
# md5sum writes records backwards,
#+ checksum first, then filename.
checksum[n]=$( md5sum "${filename[n]}" )
if [ "${record[n]}" = "${checksum[n]}" ]
then
echo "${filename[n]} unchanged."
elif [ "`basename ${filename[n]}`" != "$dbfile" ]
# Skip over checksum database file,
#+ as it will change with each invocation of script.
# ---
# This unfortunately means that when running
#+ this script on $PWD, tampering with the
#+ checksum database file will not be detected.
# Exercise: Fix this.
then
echo "${filename[n]} : CHECKSUM ERROR!"
# File has been changed since last checked.
fi
fi
let "n+=1"
done <"$dbfile" # Read from checksum database file.
}
# =================================================== #
# main ()
if [ -z "$1" ]
then
directory="$PWD" # If not specified,
else #+ use current working directory.
directory="$1"
fi
clear # Clear screen.
echo " Running file integrity check on $directory"
echo
# ------------------------------------------------------------------ #
if [ ! -r "$dbfile" ] # Need to create database file?
then
echo "Setting up database file, \""$directory"/"$dbfile"\"."; echo
set_up_database
fi
# ------------------------------------------------------------------ #
check_database # Do the actual work.
echo
# You may wish to redirect the stdout of this script to a file,
#+ especially if the directory checked has many files in it.
exit 0
# For a much more thorough file integrity check,
#+ consider the "Tripwire" package,
#+ http://sourceforge.net/projects/tripwire/.
|
See also Example A-19 and Example 33-14 for creative uses of
the md5sum command. - shred
Securely erase a file by overwriting it multiple times with
random bit patterns before deleting it. This command has
the same effect as Example 12-54, but does it
in a more thorough and elegant manner. This is one of the GNU fileutils. | Advanced forensic technology may still be able to
recover the contents of a file, even after application of
shred. |
Encoding and Encryption - uuencode
This utility encodes binary files into ASCII characters, making them
suitable for transmission in the body of an e-mail message or in a
newsgroup posting. - uudecode
This reverses the encoding, decoding uuencoded files back into the
original binaries. Example 12-35. Uudecoding encoded files #!/bin/bash
# Uudecodes all uuencoded files in current working directory.
lines=35 # Allow 35 lines for the header (very generous).
for File in * # Test all the files in $PWD.
do
search1=`head -$lines $File | grep begin | wc -w`
search2=`tail -$lines $File | grep end | wc -w`
# Uuencoded files have a "begin" near the beginning,
#+ and an "end" near the end.
if [ "$search1" -gt 0 ]
then
if [ "$search2" -gt 0 ]
then
echo "uudecoding - $File -"
uudecode $File
fi
fi
done
# Note that running this script upon itself fools it
#+ into thinking it is a uuencoded file,
#+ because it contains both "begin" and "end".
# Exercise:
# --------
# Modify this script to check each file for a newsgroup header,
#+ and skip to next if not found.
exit 0 |
| The fold -s command
may be useful (possibly in a pipe) to process long uudecoded
text messages downloaded from Usenet newsgroups. |
- mimencode, mmencode
The mimencode and
mmencode commands process
multimedia-encoded e-mail attachments. Although
mail user agents (such as
pine or kmail)
normally handle this automatically, these particular
utilities permit manipulating such attachments manually
from the command line or in a batch by means of a shell
script. - crypt
At one time, this was the standard UNIX file encryption
utility.
Politically motivated government regulations
prohibiting the export of encryption software resulted
in the disappearance of crypt
from much of the UNIX world, and it is still
missing from most Linux distributions. Fortunately,
programmers have come up with a number of decent
alternatives to it, among them the author's very own cruft
(see Example A-4).
Miscellaneous - mktemp
Create a temporary file
with a "unique" filename. When invoked
from the command line without additional arguments,
it creates a zero-length file in the /tmp directory. bash$ mktemp
/tmp/tmp.zzsvql3154
|
PREFIX=filename
tempfile=`mktemp $PREFIX.XXXXXX`
# ^^^^^^ Need at least 6 placeholders
#+ in the filename template.
# If no filename template supplied,
#+ "tmp.XXXXXXXXXX" is the default.
echo "tempfile name = $tempfile"
# tempfile name = filename.QA2ZpY
# or something similar...
# Creates a file of that name in the current working directory
#+ with 600 file permissions.
# A "umask 177" is therefore unnecessary,
# but it's good programming practice anyhow. |
- make
Utility for building and compiling binary packages.
This can also be used for any set of operations that is
triggered by incremental changes in source files.
The make command checks a
Makefile, a list of file dependencies and
operations to be carried out. - install
Special purpose file copying command, similar to
cp, but capable of setting permissions
and attributes of the copied files. This command seems
tailormade for installing software packages, and as such it
shows up frequently in Makefiles
(in the make install :
section). It could likewise find use in installation
scripts. - dos2unix
This utility, written by Benjamin Lin and collaborators,
converts DOS-formatted text files (lines terminated by
CR-LF) to UNIX format (lines terminated by LF only),
and vice-versa. - ptx
The ptx [targetfile] command
outputs a permuted index (cross-reference list) of the
targetfile. This may be further filtered and formatted in a
pipe, if necessary. - more, less
Pagers that display a text file or stream to
stdout, one screenful at a time.
These may be used to filter the output of
stdout . . . or of a script. An interesting application of more
is to "test drive" a command sequence,
to forestall potentially unpleasant consequences.
ls /home/bozo | awk '{print "rm -rf " $1}' | more
# ^^^^
# Testing the effect of the following (disastrous) command line:
# ls /home/bozo | awk '{print "rm -rf " $1}' | sh
# Hand off to the shell to execute . . . ^^ |
Miscellaneous CommandsCommand that fit in no
special category - jot, seq
These utilities emit a sequence of integers, with a
user-selected increment. The normal separator character between each integer is a
newline, but this can be changed with the -s
option. bash$ seq 5
1
2
3
4
5
bash$ seq -s : 5
1:2:3:4:5
|
Both jot and seq
come in handy in a for
loop. Example 12-48. Using seq to generate loop arguments #!/bin/bash
# Using "seq"
echo
for a in `seq 80` # or for a in $( seq 80 )
# Same as for a in 1 2 3 4 5 ... 80 (saves much typing!).
# May also use 'jot' (if present on system).
do
echo -n "$a "
done # 1 2 3 4 5 ... 80
# Example of using the output of a command to generate
# the [list] in a "for" loop.
echo; echo
COUNT=80 # Yes, 'seq' may also take a replaceable parameter.
for a in `seq $COUNT` # or for a in $( seq $COUNT )
do
echo -n "$a "
done # 1 2 3 4 5 ... 80
echo; echo
BEGIN=75
END=80
for a in `seq $BEGIN $END`
# Giving "seq" two arguments starts the count at the first one,
#+ and continues until it reaches the second.
do
echo -n "$a "
done # 75 76 77 78 79 80
echo; echo
BEGIN=45
INTERVAL=5
END=80
for a in `seq $BEGIN $INTERVAL $END`
# Giving "seq" three arguments starts the count at the first one,
#+ uses the second for a step interval,
#+ and continues until it reaches the third.
do
echo -n "$a "
done # 45 50 55 60 65 70 75 80
echo; echo
exit 0 |
Example 12-49. Letter Count" #!/bin/bash
# letter-count.sh: Counting letter occurrences in a text file.
# Written by Stefano Palmeri.
# Used in ABS Guide with permission.
# Slightly modified by document author.
MINARGS=2 # Script requires at least two arguments.
E_BADARGS=65
FILE=$1
let LETTERS=$#-1 # How many letters specified (as command-line args).
# (Subtract 1 from number of command line args.)
show_help(){
echo
echo Usage: `basename $0` file letters
echo Note: `basename $0` arguments are case sensitive.
echo Example: `basename $0` foobar.txt G n U L i N U x.
echo
}
# Checks number of arguments.
if [ $# -lt $MINARGS ]; then
echo
echo "Not enough arguments."
echo
show_help
exit $E_BADARGS
fi
# Checks if file exists.
if [ ! -f $FILE ]; then
echo "File \"$FILE\" does not exist."
exit $E_BADARGS
fi
# Counts letter occurrences .
for n in `seq $LETTERS`; do
shift
if [[ `echo -n "$1" | wc -c` -eq 1 ]]; then # Checks arg.
echo "$1" -\> `cat $FILE | tr -cd "$1" | wc -c` # Counting.
else
echo "$1 is not a single char."
fi
done
exit $?
# This script has exactly the same functionality as letter-count2.sh,
#+ but executes faster.
# Why? |
- getopt
The getopt command
parses command-line options preceded by a dash. This external command
corresponds to the getopts
Bash builtin. Using getopt permits
handling long options by means of the -l
flag, and this also allows parameter reshuffling. Example 12-50. Using getopt to parse command-line
options #!/bin/bash
# Using getopt.
# Try the following when invoking this script:
# sh ex33a.sh -a
# sh ex33a.sh -abc
# sh ex33a.sh -a -b -c
# sh ex33a.sh -d
# sh ex33a.sh -dXYZ
# sh ex33a.sh -d XYZ
# sh ex33a.sh -abcd
# sh ex33a.sh -abcdZ
# sh ex33a.sh -z
# sh ex33a.sh a
# Explain the results of each of the above.
E_OPTERR=65
if [ "$#" -eq 0 ]
then # Script needs at least one command-line argument.
echo "Usage $0 -[options a,b,c]"
exit $E_OPTERR
fi
set -- `getopt "abcd:" "$@"`
# Sets positional parameters to command-line arguments.
# What happens if you use "$*" instead of "$@"?
while [ ! -z "$1" ]
do
case "$1" in
-a) echo "Option \"a\"";;
-b) echo "Option \"b\"";;
-c) echo "Option \"c\"";;
-d) echo "Option \"d\" $2";;
*) break;;
esac
shift
done
# It is usually better to use the 'getopts' builtin in a script,
#+ rather than 'getopt'.
# See "ex33.sh".
exit 0 |
See Example 9-12 for a simplified emulation
of getopt. - run-parts
The run-parts command
executes all the scripts in a target directory, sequentially
in ASCII-sorted filename order. Of course, the scripts
need to have execute permission. The cron daemon invokes
run-parts to run the scripts in
the /etc/cron.*
directories. - yes
In its default behavior the yes
command feeds a continuous string of the character
y followed
by a line feed to stdout. A
control-c
terminates the run. A different output string
may be specified, as in yes different
string, which would continually output
different string to
stdout. One might well ask the purpose
of this. From the command line or in a script, the output
of yes can be redirected or piped into a
program expecting user input. In effect, this becomes a sort
of poor man's version of expect. yes | fsck /dev/hda1 runs
fsck non-interactively (careful!). yes | rm -r dirname has same effect as
rm -rf dirname (careful!). | Caution advised when piping yes
to a potentially dangerous system command, such
as fsck or fdisk. It may have unintended
side-effects. |
- banner
Prints arguments as a large vertical banner to
stdout, using an ASCII character
(default '#'). This may be redirected to a printer for
hardcopy. - printenv
Show all the environmental
variables set for a particular user. bash$ printenv | grep HOME
HOME=/home/bozo
|
- lp
The lp and lpr
commands send file(s) to the print queue, to be printed as
hard copy.
These commands trace the origin of their names to the
line printers of another era. bash$ lp file1.txt
or bash lp
<file1.txt It is often useful to pipe the formatted output from
pr to lp. bash$ pr -options file1.txt | lp
Formatting packages, such as groff and
Ghostscript may send their output
directly to lp. bash$ groff -Tascii file.tr | lp
bash$ gs -options | lp file.ps
Related commands are lpq, for viewing
the print queue, and lprm, for removing
jobs from the print queue. - tee
[UNIX borrows an idea here from the plumbing trade.] This is a redirection operator, but with a difference. Like the
plumber's "tee," it permits "siponing
off" to a file the output of a command
or commands within a pipe, but without affecting the result. This is
useful for printing an ongoing process to a file or paper, perhaps to
keep track of it for debugging purposes. tee
|------> to file
|
===============|===============
command--->----|-operator-->---> result of command(s)
===============================
|
cat listfile* | sort | tee check.file | uniq > result.file |
(The file check.file contains
the concatenated sorted "listfiles",
before the duplicate lines are removed by uniq.)- mkfifo
This obscure command
creates a named pipe, a temporary
first-in-first-out buffer for
transferring data between processes.
Typically, one process writes to the FIFO, and the other
reads from it. See Example A-15. - pathchk
This command checks the validity of a filename. If the
filename exceeds the maximum allowable length (255
characters) or one or more of the directories in
its path is not searchable, then an error message
results. Unfortunately, pathchk does
not return a recognizable error code, and it is therefore
pretty much useless in a script. Consider instead the
file test operators. - dd
This is the somewhat obscure and much feared "data
duplicator" command. Originally a utility
for exchanging data on magnetic tapes between UNIX
minicomputers and IBM mainframes, this command still
has its uses. The dd command simply
copies a file (or stdin/stdout), but
with conversions. Possible conversions are ASCII/EBCDIC,
upper/lower case, swapping of byte pairs between input
and output, and skipping and/or truncating the head or
tail of the input file. A dd --help
lists the conversion and other options that this powerful
utility takes. # Converting a file to all uppercase:
dd if=$filename conv=ucase > $filename.uppercase
# lcase # For lower case conversion |
Example 12-51. A script that copies itself #!/bin/bash
# self-copy.sh
# This script copies itself.
file_subscript=copy
dd if=$0 of=$0.$file_subscript 2>/dev/null
# Suppress messages from dd: ^^^^^^^^^^^
exit $? |
Example 12-52. Exercising dd #!/bin/bash
# exercising-dd.sh
# Script by Stephane Chazelas.
# Somewhat modified by document author.
input_file=$0 # This script.
output_file=log.txt
n=3
p=5
dd if=$input_file of=$output_file bs=1 skip=$((n-1)) count=$((p-n+1)) 2> /dev/null
# Extracts characters n to p from this script.
# -------------------------------------------------------
echo -n "hello world" | dd cbs=1 conv=unblock 2> /dev/null
# Echoes "hello world" vertically.
exit 0 |
To demonstrate just how versatile dd is,
let's use it to capture keystrokes. Example 12-53. Capturing Keystrokes #!/bin/bash
# dd-keypress.sh: Capture keystrokes without needing to press ENTER.
keypresses=4 # Number of keypresses to capture.
old_tty_setting=$(stty -g) # Save old terminal settings.
echo "Press $keypresses keys."
stty -icanon -echo # Disable canonical mode.
# Disable local echo.
keys=$(dd bs=1 count=$keypresses 2> /dev/null)
# 'dd' uses stdin, if "if" (input file) not specified.
stty "$old_tty_setting" # Restore old terminal settings.
echo "You pressed the \"$keys\" keys."
# Thanks, Stephane Chazelas, for showing the way.
exit 0 |
The dd command can do random access on a
data stream.
echo -n . | dd bs=1 seek=4 of=file conv=notrunc
# The "conv=notrunc" option means that the output file will not be truncated.
# Thanks, S.C. |
The dd command can copy raw data
and disk images to and from devices, such as floppies and
tape drives (Example A-5). A common use is
creating boot floppies. dd if=kernel-image of=/dev/fd0H1440
Similarly, dd can copy the entire
contents of a floppy, even one formatted with a
"foreign" OS, to the hard drive as an
image file. dd if=/dev/fd0 of=/home/bozo/projects/floppy.img
Other applications of dd include
initializing temporary swap files (Example 28-2)
and ramdisks (Example 28-3). It can even do a
low-level copy of an entire hard drive partition, although
this is not necessarily recommended. People (with presumably nothing better to do with
their time) are constantly thinking of interesting
applications of dd. Example 12-54. Securely deleting a file #!/bin/bash
# blot-out.sh: Erase "all" traces of a file.
# This script overwrites a target file alternately
#+ with random bytes, then zeros before finally deleting it.
# After that, even examining the raw disk sectors by conventional methods
#+ will not reveal the original file data.
PASSES=7 # Number of file-shredding passes.
# Increasing this slows script execution,
#+ especially on large target files.
BLOCKSIZE=1 # I/O with /dev/urandom requires unit block size,
#+ otherwise you get weird results.
E_BADARGS=70 # Various error exit codes.
E_NOT_FOUND=71
E_CHANGED_MIND=72
if [ -z "$1" ] # No filename specified.
then
echo "Usage: `basename $0` filename"
exit $E_BADARGS
fi
file=$1
if [ ! -e "$file" ]
then
echo "File \"$file\" not found."
exit $E_NOT_FOUND
fi
echo; echo -n "Are you absolutely sure you want to blot out \"$file\" (y/n)? "
read answer
case "$answer" in
[nN]) echo "Changed your mind, huh?"
exit $E_CHANGED_MIND
;;
*) echo "Blotting out file \"$file\".";;
esac
flength=$(ls -l "$file" | awk '{print $5}') # Field 5 is file length.
pass_count=1
chmod u+w "$file" # Allow overwriting/deleting the file.
echo
while [ "$pass_count" -le "$PASSES" ]
do
echo "Pass #$pass_count"
sync # Flush buffers.
dd if=/dev/urandom of=$file bs=$BLOCKSIZE count=$flength
# Fill with random bytes.
sync # Flush buffers again.
dd if=/dev/zero of=$file bs=$BLOCKSIZE count=$flength
# Fill with zeros.
sync # Flush buffers yet again.
let "pass_count += 1"
echo
done
rm -f $file # Finally, delete scrambled and shredded file.
sync # Flush buffers a final time.
echo "File \"$file\" blotted out and deleted."; echo
exit 0
# This is a fairly secure, if inefficient and slow method
#+ of thoroughly "shredding" a file.
# The "shred" command, part of the GNU "fileutils" package,
#+ does the same thing, although more efficiently.
# The file cannot not be "undeleted" or retrieved by normal methods.
# However . . .
#+ this simple method would *not* likely withstand
#+ sophisticated forensic analysis.
# This script may not play well with a journaled file system.
# Exercise (difficult): Fix it so it does.
# Tom Vier's "wipe" file-deletion package does a much more thorough job
#+ of file shredding than this simple script.
# http://www.ibiblio.org/pub/Linux/utils/file/wipe-2.0.0.tar.bz2
# For an in-depth analysis on the topic of file deletion and security,
#+ see Peter Gutmann's paper,
#+ "Secure Deletion of Data From Magnetic and Solid-State Memory".
# http://www.cs.auckland.ac.nz/~pgut001/pubs/secure_del.html |
- od
The od, or octal
dump filter converts input (or files) to octal
(base-8) or other bases. This is useful for viewing or
processing binary data files or otherwise unreadable system
device files, such as /dev/urandom,
and as a filter for binary data. See Example 9-28 and Example 12-13. - hexdump
Performs a hexadecimal, octal, decimal, or ASCII
dump of a binary file. This command is the rough equivalent
of od, above, but not nearly as
useful. - objdump
Displays information about an object file or binary
executable in either hexadecimal form or as a disassembled
listing (with the -d option). bash$ objdump -d /bin/ls
/bin/ls: file format elf32-i386
Disassembly of section .init:
080490bc <.init>:
80490bc: 55 push %ebp
80490bd: 89 e5 mov %esp,%ebp
. . .
|
- mcookie
This command generates a "magic cookie", a
128-bit (32-character) pseudorandom hexadecimal number,
normally used as an authorization "signature"
by the X server. This also available for use in a script as
a "quick 'n dirty" random number.
Of course, a script could use md5 for the same purpose.
# Generate md5 checksum on the script itself.
random001=`md5sum $0 | awk '{print $1}'`
# Uses 'awk' to strip off the filename. |
The mcookie command gives yet another way
to generate a "unique" filename. Example 12-55. Filename generator #!/bin/bash
# tempfile-name.sh: temp filename generator
BASE_STR=`mcookie` # 32-character magic cookie.
POS=11 # Arbitrary position in magic cookie string.
LEN=5 # Get $LEN consecutive characters.
prefix=temp # This is, after all, a "temp" file.
# For more "uniqueness," generate the filename prefix
#+ using the same method as the suffix, below.
suffix=${BASE_STR:POS:LEN}
# Extract a 5-character string, starting at position 11.
temp_filename=$prefix.$suffix
# Construct the filename.
echo "Temp filename = "$temp_filename""
# sh tempfile-name.sh
# Temp filename = temp.e19ea
# Compare this method of generating "unique" filenames
#+ with the 'date' method in ex51.sh.
exit 0 |
- units
This utility converts between different units of measure.
While normally invoked in interactive mode,
units may find use in a script. Example 12-56. Converting meters to miles #!/bin/bash
# unit-conversion.sh
convert_units () # Takes as arguments the units to convert.
{
cf=$(units "$1" "$2" | sed --silent -e '1p' | awk '{print $2}')
# Strip off everything except the actual conversion factor.
echo "$cf"
}
Unit1=miles
Unit2=meters
cfactor=`convert_units $Unit1 $Unit2`
quantity=3.73
result=$(echo $quantity*$cfactor | bc)
echo "There are $result $Unit2 in $quantity $Unit1."
# What happens if you pass incompatible units,
#+ such as "acres" and "miles" to the function?
exit 0 |
- m4
A hidden treasure, m4 is a
powerful macro processing filter,
virtually a complete language. Although
originally written as a pre-processor for
RatFor, m4
turned out to be useful as a stand-alone utility. In
fact, m4 combines some of the
functionality of eval,
tr, and awk, in addition to its extensive
macro expansion facilities. The April, 2002 issue of Linux Journal
has a very nice article on m4 and
its uses. Example 12-57. Using m4 #!/bin/bash
# m4.sh: Using the m4 macro processor
# Strings
string=abcdA01
echo "len($string)" | m4 # 7
echo "substr($string,4)" | m4 # A01
echo "regexp($string,[0-1][0-1],\&Z)" | m4 # 01Z
# Arithmetic
echo "incr(22)" | m4 # 23
echo "eval(99 / 3)" | m4 # 33
exit 0 |
- doexec
The doexec command enables passing
an arbitrary list of arguments to a binary
executable. In particular, passing
argv[0] (which corresponds to $0 in a script) lets the
executable be invoked by various names, and it can then
carry out different sets of actions, according to the name
by which it was called. What this amounts to is roundabout
way of passing options to an executable. For example, the /usr/local/bin directory might
contain a binary called "aaa". Invoking
doexec /usr/local/bin/aaa list
would list all those files
in the current working directory beginning with an
"a", while invoking (the same executable
with) doexec /usr/local/bin/aaa delete
would delete those files. | The various behaviors of the executable
must be defined within the code of the executable itself,
analogous to something like the following in a shell script:
case `basename $0` in
"name1" ) do_something;;
"name2" ) do_something_else;;
"name3" ) do_yet_another_thing;;
* ) bail_out;;
esac |
|
- dialog
The dialog family of tools
provide a method of calling interactive
"dialog" boxes from a script. The more
elaborate variations of dialog --
gdialog, Xdialog,
and kdialog -- actually invoke X-Windows
widgets. See Example 33-19. - sox
The sox, or
"sound
exchange" command plays and
performs transformations on sound files. In fact,
the /usr/bin/play executable
(now deprecated) is nothing but a shell wrapper for
sox. For example, sox soundfile.wav
soundfile.au changes a WAV sound file into a
(Sun audio format) AU sound file. Shell scripts are ideally suited for batch processing
sox operations on
sound files. For examples, see the Linux Radio
Timeshift HOWTO and the MP3do
Project.
System and Administrative CommandsThe startup and shutdown scripts in
/etc/rc.d illustrate the uses
(and usefulness) of many of these comands. These are usually
invoked by root and used for system maintenance or emergency
filesystem repairs. Use with caution, as some of these commands
may damage your system if misused. Users and Groups - users
Show all logged on users. This is the approximate
equivalent of who -q. - groups
Lists the current user and the groups she belongs to.
This corresponds to the $GROUPS internal variable,
but gives the group names, rather than the numbers. bash$ groups
bozita cdrom cdwriter audio xgrp
bash$ echo $GROUPS
501 |
- chown, chgrp
The chown command changes the
ownership of a file or files. This command is a useful
method that root can use to
shift file ownership from one user to another. An ordinary
user may not change the ownership of files, not even her
own files.
The chgrp command changes the
group ownership of a file or
files. You must be owner of the file(s) as well as a member
of the destination group (or root)
to use this operation.
chgrp --recursive dunderheads *.data
# The "dunderheads" group will now own all the "*.data" files
#+ all the way down the $PWD directory tree (that's what "recursive" means). |
- useradd, userdel
The useradd administrative command
adds a user account to the system and creates a home
directory for that particular user, if so specified. The
corresponding userdel command removes
a user account from the system
and deletes associated files. | The adduser command is a synonym
for useradd and is usually a symbolic link to
it. |
- usermod
Modify a user account. Changes may be made to the password,
group membership, expiration date, and other attributes of
a given user's account. With this command, a user's password
may be locked, which has the effect of disabling the
account. - groupmod
Modify a given group. The group name and/or ID number may be
changed using this command. - id
The id command lists the real and
effective user IDs and the group IDs of the user
associated with the current process. This is the
counterpart to the $UID,
$EUID, and $GROUPS internal Bash
variables. bash$ id
uid=501(bozo) gid=501(bozo) groups=501(bozo),22(cdrom),80(cdwriter),81(audio)
bash$ echo $UID
501 |
| The id command shows the
effective IDs only when they differ
from the real ones. |
Also see Example 9-5. - who
Show all users logged on to the system. bash$ who
bozo tty1 Apr 27 17:45
bozo pts/0 Apr 27 17:46
bozo pts/1 Apr 27 17:47
bozo pts/2 Apr 27 17:49
|
The -m gives detailed information about
only the current user. Passing any two arguments to
who is the equivalent of who
-m, as in who am i or who
The Man. bash$ who -m
localhost.localdomain!bozo pts/2 Apr 27 17:49
|
whoami is similar to who
-m, but only lists the user name.
- w
Show all logged on users and the processes belonging to them. This is
an extended version of who. The output of w
may be piped to grep to find a specific user and/or process. bash$ w | grep startx
bozo tty1 - 4:22pm 6:41 4.47s 0.45s startx |
- logname
Show current user's login name (as found in
/var/run/utmp). This is a
near-equivalent to whoami,
above. bash$ logname
bozo
bash$ whoami
bozo |
However... bash$ su
Password: ......
bash# whoami
root
bash# logname
bozo |
| While logname prints the name
of the logged in user, whoami gives the
name of the user attached to the current process. As we have
just seen, sometimes these are not the same. |
- su
Runs a program or script as a
substitute user.
su rjones starts a shell as user
rjones. A naked su
defaults to root. See Example A-15. - sudo
Runs a command as root (or another user). This may
be used in a script, thus permitting a regular user to
run the script. #!/bin/bash
# Some commands.
sudo cp /root/secretfile /home/bozo/secret
# Some more commands. |
The file /etc/sudoers holds
the names of users permitted to invoke
sudo. - passwd
Sets, changes, or manages a user's password. The passwd command can be used in
a script, but should not be. Example 13-1. Setting a new password #!/bin/bash
# setnew-password.sh: For demonstration purposes only.
# Not a good idea to actually run this script.
# This script must be run as root.
ROOT_UID=0 # Root has $UID 0.
E_WRONG_USER=65 # Not root?
E_NOSUCHUSER=70
SUCCESS=0
if [ "$UID" -ne "$ROOT_UID" ]
then
echo; echo "Only root can run this script."; echo
exit $E_WRONG_USER
else
echo
echo "You should know better than to run this script, root."
echo "Even root users get the blues... "
echo
fi
username=bozo
NEWPASSWORD=security_violation
# Check if bozo lives here.
grep -q "$username" /etc/passwd
if [ $? -ne $SUCCESS ]
then
echo "User $username does not exist."
echo "No password changed."
exit $E_NOSUCHUSER
fi
echo "$NEWPASSWORD" | passwd --stdin "$username"
# The '--stdin' option to 'passwd' permits
#+ getting a new password from stdin (or a pipe).
echo; echo "User $username's password changed!"
# Using the 'passwd' command in a script is dangerous.
exit 0 |
The passwd command's -l,
-u, and -d options permit
locking, unlocking, and deleting a user's password. Only
root may use these options. - ac
Show users' logged in time, as read from
/var/log/wtmp. This is one of the GNU
accounting utilities. - last
List last logged in users, as read from
/var/log/wtmp. This command can also
show remote logins. For example, to show the last few times the system
rebooted: bash$ last reboot
reboot system boot 2.6.9-1.667 Fri Feb 4 18:18 (00:02)
reboot system boot 2.6.9-1.667 Fri Feb 4 15:20 (01:27)
reboot system boot 2.6.9-1.667 Fri Feb 4 12:56 (00:49)
reboot system boot 2.6.9-1.667 Thu Feb 3 21:08 (02:17)
. . .
wtmp begins Tue Feb 1 12:50:09 2005 |
- newgrp
Change user's group ID without logging out. This permits
access to the new group's files. Since users may be
members of multiple groups simultaneously, this command
finds little use.
Terminals - tty
Echoes the name of the current user's terminal.
Note that each separate xterm
window counts as a different terminal. - stty
Shows and/or changes terminal settings. This complex
command, used in a script, can control terminal behavior
and the way output displays. See the info page, and study
it carefully. Example 13-2. Setting an erase character #!/bin/bash
# erase.sh: Using "stty" to set an erase character when reading input.
echo -n "What is your name? "
read name # Try to backspace
#+ to erase characters of input.
# Problems?
echo "Your name is $name."
stty erase '#' # Set "hashmark" (#) as erase character.
echo -n "What is your name? "
read name # Use # to erase last character typed.
echo "Your name is $name."
# Warning: Even after the script exits, the new key value remains set.
exit 0 |
Example 13-3. secret password:
Turning off terminal echoing #!/bin/bash
# secret-pw.sh: secret password
echo
echo -n "Enter password "
read passwd
echo "password is $passwd"
echo -n "If someone had been looking over your shoulder, "
echo "your password would have been compromised."
echo && echo # Two line-feeds in an "and list."
stty -echo # Turns off screen echo.
echo -n "Enter password again "
read passwd
echo
echo "password is $passwd"
echo
stty echo # Restores screen echo.
exit 0
# Do an 'info stty' for more on this useful-but-tricky command. |
A creative use of stty is detecting a
user keypress (without hitting
ENTER). Example 13-4. Keypress detection #!/bin/bash
# keypress.sh: Detect a user keypress ("hot keys").
echo
old_tty_settings=$(stty -g) # Save old settings (why?).
stty -icanon
Keypress=$(head -c1) # or $(dd bs=1 count=1 2> /dev/null)
# on non-GNU systems
echo
echo "Key pressed was \""$Keypress"\"."
echo
stty "$old_tty_settings" # Restore old settings.
# Thanks, Stephane Chazelas.
exit 0 |
Also see Example 9-3. - setterm
Set certain terminal attributes. This command writes
to its terminal's stdout a string that
changes the behavior of that terminal. bash$ setterm -cursor off
bash$
|
The setterm command can be used within a
script to change the appearance of text written to
stdout, although there are certainly
better tools available
for this purpose. setterm -bold on
echo bold hello
setterm -bold off
echo normal hello |
- tset
Show or initialize terminal settings.
This is a less capable version of
stty. bash$ tset -r
Terminal type is xterm-xfree86.
Kill is control-U (^U).
Interrupt is control-C (^C).
|
- setserial
Set or display serial port parameters. This command must be
run by root user and is usually found in a system setup
script. # From /etc/pcmcia/serial script:
IRQ=`setserial /dev/$DEVICE | sed -e 's/.*IRQ: //'`
setserial /dev/$DEVICE irq 0 ; setserial /dev/$DEVICE irq $IRQ |
- getty, agetty
The initialization process for a terminal uses
getty or agetty
to set it up for login by a user. These commands are not
used within user shell scripts. Their scripting counterpart
is stty. - mesg
Enables or disables write access to the current user's
terminal. Disabling access would prevent another user
on the network to write
to the terminal. | It can be very annoying to have a message
about ordering pizza suddenly appear in the middle of
the text file you are editing. On a multi-user network,
you might therefore wish to disable write access to your
terminal when you need to avoid interruptions. |
- wall
This is an acronym for "write all", i.e., sending
a message to all users at every terminal logged into the
network. It is primarily a system administrator's tool,
useful, for example, when warning everyone that the
system will shortly go down due to a problem (see Example 17-1). bash$ wall System going down for maintenance in 5 minutes!
Broadcast message from bozo (pts/1) Sun Jul 8 13:53:27 2001...
System going down for maintenance in 5 minutes!
|
| If write access to a particular terminal has been
disabled with mesg, then
wall cannot send a message to
it. |
- dmesg
Lists all system bootup messages to
stdout. Handy for debugging and
ascertaining which device drivers were installed
and which system interrupts in use. The output
of dmesg may, of course, be
parsed with grep,
sed, or awk from within a script. bash$ dmesg | grep hda
Kernel command line: ro root=/dev/hda2
hda: IBM-DLGA-23080, ATA DISK drive
hda: 6015744 sectors (3080 MB) w/96KiB Cache, CHS=746/128/63
hda: hda1 hda2 hda3 < hda5 hda6 hda7 > hda4
|
Information and Statistics - uname
Output system specifications (OS, kernel version,
etc.) to stdout. Invoked with the
-a option, gives verbose system info
(see Example 12-5). The -s
option shows only the OS type. bash$ uname -a
Linux localhost.localdomain 2.2.15-2.5.0 #1 Sat Feb 5 00:13:43 EST 2000 i686 unknown
bash$ uname -s
Linux |
- arch
Show system architecture.
Equivalent to uname -m. See Example 10-26. bash$ arch
i686
bash$ uname -m
i686 |
- lastcomm
Gives information about previous commands, as stored
in the /var/account/pacct file. Command
name and user name can be specified by options. This is
one of the GNU accounting utilities. - lastlog
List the last login time of all system users. This
references the /var/log/lastlog
file. bash$ lastlog
root tty1 Fri Dec 7 18:43:21 -0700 2001
bin **Never logged in**
daemon **Never logged in**
...
bozo tty1 Sat Dec 8 21:14:29 -0700 2001
bash$ lastlog | grep root
root tty1 Fri Dec 7 18:43:21 -0700 2001
|
| This command will fail if the user invoking
it does not have read permission for the
/var/log/lastlog file. |
- lsof
List open files. This command outputs a detailed
table of all currently open files and gives information
about their owner, size, the processes associated with
them, and more. Of course, lsof may
be piped to grep and/or
awk to parse and analyze
its results. bash$ lsof
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
init 1 root mem REG 3,5 30748 30303 /sbin/init
init 1 root mem REG 3,5 73120 8069 /lib/ld-2.1.3.so
init 1 root mem REG 3,5 931668 8075 /lib/libc-2.1.3.so
cardmgr 213 root mem REG 3,5 36956 30357 /sbin/cardmgr
...
|
- strace
Diagnostic and debugging tool for tracing system
calls and signals. The simplest way of invoking it is
strace COMMAND. bash$ strace df
execve("/bin/df", ["df"], [/* 45 vars */]) = 0
uname({sys="Linux", node="bozo.localdomain", ...}) = 0
brk(0) = 0x804f5e4
...
|
This is the Linux equivalent of
truss. - nmap
Network port scanner. This command scans a server to
locate open ports and the services associated with those
ports. It is an important security tool for locking down
a network against hacking attempts. #!/bin/bash
SERVER=$HOST # localhost.localdomain (127.0.0.1).
PORT_NUMBER=25 # SMTP port.
nmap $SERVER | grep -w "$PORT_NUMBER" # Is that particular port open?
# grep -w matches whole words only,
#+ so this wouldn't match port 1025, for example.
exit 0
# 25/tcp open smtp |
- nc
The nc (netcat)
utility is a complete toolkit for connecting to and
listening to TCP and UDP ports. It is useful as a diagnostic
and testing tool and as a component in simple script-based HTTP
clients and servers. bash$ nc localhost.localdomain 25
220 localhost.localdomain ESMTP Sendmail 8.13.1/8.13.1; Thu, 31 Mar 2005 15:41:35 -0700 |
Example 13-5. Checking a remote server for identd #! /bin/sh
## Duplicate DaveG's ident-scan thingie using netcat. Oooh, he'll be p*ssed.
## Args: target port [port port port ...]
## Hose stdout _and_ stderr together.
##
## Advantages: runs slower than ident-scan, giving remote inetd less cause
##+ for alarm, and only hits the few known daemon ports you specify.
## Disadvantages: requires numeric-only port args, the output sleazitude,
##+ and won't work for r-services when coming from high source ports.
# Script author: Hobbit <hobbit@avian.org>
# Used in ABS Guide with permission.
# ---------------------------------------------------
E_BADARGS=65 # Need at least two args.
TWO_WINKS=2 # How long to sleep.
THREE_WINKS=3
IDPORT=113 # Authentication "tap ident" port.
RAND1=999
RAND2=31337
TIMEOUT0=9
TIMEOUT1=8
TIMEOUT2=4
# ---------------------------------------------------
case "${2}" in
"" ) echo "Need HOST and at least one PORT." ; exit $E_BADARGS ;;
esac
# Ping 'em once and see if they *are* running identd.
nc -z -w $TIMEOUT0 "$1" $IDPORT || { echo "Oops, $1 isn't running identd." ; exit 0 ; }
# -z scans for listening daemons.
# -w $TIMEOUT = How long to try to connect.
# Generate a randomish base port.
RP=`expr $$ % $RAND1 + $RAND2`
TRG="$1"
shift
while test "$1" ; do
nc -v -w $TIMEOUT1 -p ${RP} "$TRG" ${1} < /dev/null > /dev/null &
PROC=$!
sleep $THREE_WINKS
echo "${1},${RP}" | nc -w $TIMEOUT2 -r "$TRG" $IDPORT 2>&1
sleep $TWO_WINKS
# Does this look like a lamer script or what . . . ?
# ABS Guide author comments: "It ain't really all that bad,
#+ rather clever, actually."
kill -HUP $PROC
RP=`expr ${RP} + 1`
shift
done
exit $?
# Notes:
# -----
# Try commenting out line 30 and running this script
#+ with "localhost.localdomain 25" as arguments.
# For more of Hobbit's 'nc' example scripts,
#+ look in the documentation:
#+ the /usr/share/doc/nc-X.XX/scripts directory. |
And, of course, there's Dr. Andrew Tridgell's notorious
one-line script in the BitKeeper Affair:
echo clone | nc thunk.org 5000 > e2fsprogs.dat |
- free
Shows memory and cache usage in tabular form. The
output of this command lends itself to parsing, using
grep, awk or Perl. The
procinfo command shows all the
information that free does, and much
more. bash$ free
total used free shared buffers cached
Mem: 30504 28624 1880 15820 1608 16376
-/+ buffers/cache: 10640 19864
Swap: 68540 3128 65412 |
To show unused RAM memory: bash$ free | grep Mem | awk '{ print $4 }'
1880 |
- procinfo
Extract and list information and statistics from the
/proc
pseudo-filesystem. This gives a very extensive and
detailed listing. bash$ procinfo | grep Bootup
Bootup: Wed Mar 21 15:15:50 2001 Load average: 0.04 0.21 0.34 3/47 6829 |
- lsdev
List devices, that is, show installed hardware. bash$ lsdev
Device DMA IRQ I/O Ports
------------------------------------------------
cascade 4 2
dma 0080-008f
dma1 0000-001f
dma2 00c0-00df
fpu 00f0-00ff
ide0 14 01f0-01f7 03f6-03f6
...
|
- du
Show (disk) file usage, recursively. Defaults to current
working directory, unless otherwise specified. bash$ du -ach
1.0k ./wi.sh
1.0k ./tst.sh
1.0k ./random.file
6.0k .
6.0k total |
- df
Shows filesystem usage in tabular form. bash$ df
Filesystem 1k-blocks Used Available Use% Mounted on
/dev/hda5 273262 92607 166547 36% /
/dev/hda8 222525 123951 87085 59% /home
/dev/hda7 1408796 1075744 261488 80% /usr |
- stat
Gives detailed and verbose statistics
on a given file (even a directory or device file) or set
of files. bash$ stat test.cru
File: "test.cru"
Size: 49970 Allocated Blocks: 100 Filetype: Regular File
Mode: (0664/-rw-rw-r--) Uid: ( 501/ bozo) Gid: ( 501/ bozo)
Device: 3,8 Inode: 18185 Links: 1
Access: Sat Jun 2 16:40:24 2001
Modify: Sat Jun 2 16:40:24 2001
Change: Sat Jun 2 16:40:24 2001
|
If the target file does not exist, stat
returns an error message. bash$ stat nonexistent-file
nonexistent-file: No such file or directory
|
- vmstat
Display virtual memory statistics. bash$ vmstat
procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
0 0 0 0 11040 2636 38952 0 0 33 7 271 88 8 3 89
|
- netstat
Show current network statistics and information,
such as routing tables and active connections. This utility
accesses information in /proc/net
(Chapter 27). See Example 27-3. netstat -r is equivalent to route. bash$ netstat
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
Active UNIX domain sockets (w/o servers)
Proto RefCnt Flags Type State I-Node Path
unix 11 [ ] DGRAM 906 /dev/log
unix 3 [ ] STREAM CONNECTED 4514 /tmp/.X11-unix/X0
unix 3 [ ] STREAM CONNECTED 4513
. . . |
- uptime
Shows how long the system has been running, along with
associated statistics. bash$ uptime
10:28pm up 1:57, 3 users, load average: 0.17, 0.34, 0.27 |
- hostname
Lists the system's host name. This command sets the host
name in an /etc/rc.d
setup script (/etc/rc.d/rc.sysinit
or similar). It is equivalent to uname
-n, and a counterpart to the $HOSTNAME internal
variable. bash$ hostname
localhost.localdomain
bash$ echo $HOSTNAME
localhost.localdomain |
Similar to the hostname command are the
domainname,
dnsdomainname,
nisdomainname, and
ypdomainname commands. Use these to
display or set the system DNS or NIS/YP domain name. Various
options to hostname also perform these
functions. - hostid
Echo a 32-bit hexadecimal numerical identifier for the
host machine.
| This command allegedly fetches a "unique"
serial number for a particular system. Certain
product registration procedures use this number
to brand a particular user license. Unfortunately,
hostid only returns the machine
network address in hexadecimal, with pairs of bytes
transposed. The network address of a typical non-networked Linux
machine, is found in /etc/hosts. bash$ cat /etc/hosts
127.0.0.1 localhost.localdomain localhost |
As it happens, transposing the bytes of
127.0.0.1, we get
0.127.1.0, which translates in
hex to 007f0100, the exact equivalent
of what hostid returns, above. There
exist only a few million other Linux machines with this
identical hostid. |
- sar
Invoking sar (System Activity Reporter)
gives a very detailed rundown on system statistics. The
Santa Cruz Operation ("Old" SCO) released
sar as Open Source in June, 1999. This command is not part of the base Linux distribution,
but may be obtained as part of the sysstat utilities package, written by Sebastien
Godard. bash$ sar
Linux 2.4.9 (brooks.seringas.fr) 09/26/03
10:30:00 CPU %user %nice %system %iowait %idle
10:40:00 all 2.21 10.90 65.48 0.00 21.41
10:50:00 all 3.36 0.00 72.36 0.00 24.28
11:00:00 all 1.12 0.00 80.77 0.00 18.11
Average: all 2.23 3.63 72.87 0.00 21.27
14:32:30 LINUX RESTART
15:00:00 CPU %user %nice %system %iowait %idle
15:10:00 all 8.59 2.40 17.47 0.00 71.54
15:20:00 all 4.07 1.00 11.95 0.00 82.98
15:30:00 all 0.79 2.94 7.56 0.00 88.71
Average: all 6.33 1.70 14.71 0.00 77.26
|
- readelf
Show information and statistics about a designated
elf binary. This is part of the
binutils package. bash$ readelf -h /bin/bash
ELF Header:
Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
. . . |
- size
The size [/path/to/binary] command
gives the segment sizes of a binary executable or archive file.
This is mainly of use to programmers. bash$ size /bin/bash
text data bss dec hex filename
495971 22496 17392 535859 82d33 /bin/bash
|
System Logs - logger
Appends a user-generated message to the system log
(/var/log/messages). You do not have
to be root to invoke logger.
logger Experiencing instability in network connection at 23:10, 05/21.
# Now, do a 'tail /var/log/messages'. |
By embedding a logger command in a script,
it is possible to write debugging information to
/var/log/messages.
logger -t $0 -i Logging at line "$LINENO".
# The "-t" option specifies the tag for the logger entry.
# The "-i" option records the process ID.
# tail /var/log/message
# ...
# Jul 7 20:48:58 localhost ./test.sh[1712]: Logging at line 3. |
- logrotate
This utility manages the system log files, rotating,
compressing, deleting, and/or e-mailing them, as appropriate.
This keeps the /var/log
from getting cluttered with old log files.
Usually cron runs
logrotate on a daily basis. Adding an appropriate entry to
/etc/logrotate.conf makes it possible
to manage personal log files, as well as system-wide
ones. | Stefano Falsetto has created rottlog,
which he considers to be an improved version of
logrotate. |
Job Control - ps
Process
Statistics: lists currently
executing processes by owner and PID (process ID). This
is usually invoked with ax options,
and may be piped to grep
or sed to search for a
specific process (see Example 11-12 and Example 27-2). bash$ ps ax | grep sendmail
295 ? S 0:00 sendmail: accepting connections on port 25 |
To display system processes in graphical "tree"
format: ps afjx or
ps ax --forest. - pstree
Lists currently executing processes in
"tree" format. The -p option
shows the PIDs, as well as the process names. - top
Continuously updated display of most cpu-intensive
processes. The -b option displays in text
mode, so that the output may be parsed or accessed from
a script. bash$ top -b
8:30pm up 3 min, 3 users, load average: 0.49, 0.32, 0.13
45 processes: 44 sleeping, 1 running, 0 zombie, 0 stopped
CPU states: 13.6% user, 7.3% system, 0.0% nice, 78.9% idle
Mem: 78396K av, 65468K used, 12928K free, 0K shrd, 2352K buff
Swap: 157208K av, 0K used, 157208K free 37244K cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
848 bozo 17 0 996 996 800 R 5.6 1.2 0:00 top
1 root 8 0 512 512 444 S 0.0 0.6 0:04 init
2 root 9 0 0 0 0 SW 0.0 0.0 0:00 keventd
...
|
- nice
Run a background job with an altered
priority. Priorities run from 19 (lowest) to -20
(highest). Only root may set the
negative (higher) priorities. Related commands are
renice, snice,
and skill. - nohup
Keeps a command running even after user logs off.
The command will run as a foreground process unless followed
by &. If you use nohup
within a script, consider coupling it with a wait to avoid creating an orphan
or zombie process. - pidof
Identifies process ID (PID) of a
running job. Since job control commands, such as kill and renice
act on the PID of a process (not
its name), it is sometimes necessary to identify that
PID. The pidof
command is the approximate counterpart to the $PPID internal variable.
Example 13-6. pidof helps kill a process #!/bin/bash
# kill-process.sh
NOPROCESS=2
process=xxxyyyzzz # Use nonexistent process.
# For demo purposes only...
# ... don't want to actually kill any actual process with this script.
#
# If, for example, you wanted to use this script to logoff the Internet,
# process=pppd
t=`pidof $process` # Find pid (process id) of $process.
# The pid is needed by 'kill' (can't 'kill' by program name).
if [ -z "$t" ] # If process not present, 'pidof' returns null.
then
echo "Process $process was not running."
echo "Nothing killed."
exit $NOPROCESS
fi
kill $t # May need 'kill -9' for stubborn process.
# Need a check here to see if process allowed itself to be killed.
# Perhaps another " t=`pidof $process` " or ...
# This entire script could be replaced by
# kill $(pidof -x process_name)
# but it would not be as instructive.
exit 0 |
- fuser
Identifies the processes (by PID) that are accessing
a given file, set of files, or directory. May also be
invoked with the -k option, which kills
those processes. This has interesting implications for
system security, especially in scripts preventing
unauthorized users from accessing system services. bash$ fuser -u /usr/bin/vim
/usr/bin/vim: 3207e(bozo)
bash$ fuser -u /dev/null
/dev/null: 3009(bozo) 3010(bozo) 3197(bozo) 3199(bozo)
|
One important application for fuser is
when physically inserting or removing storage media, such
as CD ROM disks or USB flash drives. Sometimes trying
a umount fails with a
device is busy error message. This
means that some user(s) and/or process(es) are accessing
the device. An fuser -um /dev/device_name
will clear up the mystery, so you can kill any relevant
processes. bash$ umount /mnt/usbdrive
umount: /mnt/usbdrive: device is busy
bash$ fuser -um /dev/usbdrive
/mnt/usbdrive: 1772c(bozo)
bash$ kill -9 1772
bash$ umount /mnt/usbdrive
|
The fuser command, invoked with the
-n option identifies the processes
accessing a port. This
is especially useful in combination with nmap. root# nmap localhost.localdomain
PORT STATE SERVICE
25/tcp open smtp
root# fuser -un tcp 25
25/tcp: 2095(root)
root# ps ax | grep 2095 | grep -v grep
2095 ? Ss 0:00 sendmail: accepting connections
|
- cron
Administrative program scheduler, performing such
duties as cleaning up and deleting system log
files and updating the slocate
database. This is the superuser version of at (although each user may have their
own crontab file which can be changed
with the crontab command). It runs
as a daemon and executes
scheduled entries from /etc/crontab. | Some flavors of Linux run
crond, Matthew Dillon's version of
cron. |
Process Control and Booting - init
The init command is the parent of all processes. Called
in the final step of a bootup, init
determines the runlevel of the system from
/etc/inittab. Invoked by its alias
telinit, and by root only. - telinit
Symlinked to init, this is a means of changing the system runlevel,
usually done for system maintenance or emergency filesystem
repairs. Invoked only by root. This command can be dangerous - be
certain you understand it well before using! - runlevel
Shows the current and last runlevel, that is, whether the system
is halted (runlevel 0), in single-user mode
(1), in multi-user mode (2
or 3), in X Windows (5), or
rebooting (6). This command accesses the
/var/run/utmp file. - halt, shutdown, reboot
Command set to shut the system down, usually just prior to a power down.
Network - ifconfig
Network interface configuration
and tuning utility. bash$ ifconfig -a
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:10 errors:0 dropped:0 overruns:0 frame:0
TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:700 (700.0 b) TX bytes:700 (700.0 b) |
The ifconfig command is most often used
at bootup to set up the interfaces, or to shut them down
when rebooting. # Code snippets from /etc/rc.d/init.d/network
# ...
# Check that networking is up.
[ ${NETWORKING} = "no" ] && exit 0
[ -x /sbin/ifconfig ] || exit 0
# ...
for i in $interfaces ; do
if ifconfig $i 2>/dev/null | grep -q "UP" >/dev/null 2>&1 ; then
action "Shutting down interface $i: " ./ifdown $i boot
fi
# The GNU-specific "-q" option to "grep" means "quiet", i.e., producing no output.
# Redirecting output to /dev/null is therefore not strictly necessary.
# ...
echo "Currently active devices:"
echo `/sbin/ifconfig | grep ^[a-z] | awk '{print $1}'`
# ^^^^^ should be quoted to prevent globbing.
# The following also work.
# echo $(/sbin/ifconfig | awk '/^[a-z]/ { print $1 })'
# echo $(/sbin/ifconfig | sed -e 's/ .*//')
# Thanks, S.C., for additional comments. |
See also Example 29-6. - iwconfig
This is the command set for configuring a wireless network.
It is the wireless equivalent of ifconfig,
above. - route
Show info about or make changes to the kernel routing table. bash$ route
Destination Gateway Genmask Flags MSS Window irtt Iface
pm3-67.bozosisp * 255.255.255.255 UH 40 0 0 ppp0
127.0.0.0 * 255.0.0.0 U 40 0 0 lo
default pm3-67.bozosisp 0.0.0.0 UG 40 0 0 ppp0
|
- chkconfig
Check network configuration. This command lists and
manages the network services started at bootup in the
/etc/rc?.d
directory. Originally a port from IRIX to Red Hat Linux,
chkconfig may not be part of the core
installation of some Linux flavors. bash$ chkconfig --list
atd 0:off 1:off 2:off 3:on 4:on 5:on 6:off
rwhod 0:off 1:off 2:off 3:off 4:off 5:off 6:off
...
|
- tcpdump
Network packet "sniffer". This is a tool for
analyzing and troubleshooting traffic on a network by dumping
packet headers that match specified criteria. Dump ip packet traffic between hosts
bozoville and
caduceus:
bash$ tcpdump ip host bozoville and caduceus
|
Of course, the output of tcpdump can be
parsed, using certain of the previously discussed text processing
utilities.
Filesystem - mount
Mount a filesystem, usually on an external device,
such as a floppy or CDROM. The file
/etc/fstab provides a handy listing
of available filesystems, partitions, and devices,
including options, that may be automatically or manually
mounted. The file /etc/mtab shows
the currently mounted filesystems and partitions
(including the virtual ones, such as /proc). mount -a mounts all filesystems and
partitions listed in /etc/fstab,
except those with a noauto
option. At bootup, a startup script in
/etc/rc.d
(rc.sysinit or something similar)
invokes this to get everything mounted. mount -t iso9660 /dev/cdrom /mnt/cdrom
# Mounts CDROM
mount /mnt/cdrom
# Shortcut, if /mnt/cdrom listed in /etc/fstab |
This versatile command can even mount an ordinary file
on a block device, and the file will act as if it were a
filesystem. Mount accomplishes that by
associating the file with a loopback
device. One application of this is to mount and examine
an ISO9660 image before burning it onto a CDR.
Example 13-7. Checking a CD image # As root...
mkdir /mnt/cdtest # Prepare a mount point, if not already there.
mount -r -t iso9660 -o loop cd-image.iso /mnt/cdtest # Mount the image.
# "-o loop" option equivalent to "losetup /dev/loop0"
cd /mnt/cdtest # Now, check the image.
ls -alR # List the files in the directory tree there.
# And so forth. |
- umount
Unmount a currently mounted filesystem. Before physically removing a
previously mounted floppy or CDROM disk, the device must be
umounted, else filesystem corruption may result.
umount /mnt/cdrom
# You may now press the eject button and safely remove the disk. |
| The automount utility, if
properly installed, can mount and unmount floppies or
CDROM disks as they are accessed or removed. On laptops
with swappable floppy and CDROM drives, this can cause
problems, though. |
- sync
Forces an immediate write of all updated data from
buffers to hard drive (synchronize drive
with buffers). While not strictly necessary, a
sync assures the sys admin or
user that the data just changed will survive a sudden
power failure. In the olden days, a sync;
sync (twice, just to make absolutely sure) was a
useful precautionary measure before a system reboot. At times, you may wish to force an immediate buffer
flush, as when securely deleting a file (see Example 12-54) or when the lights begin to
flicker. - losetup
Sets up and configures loopback devices. Example 13-8. Creating a filesystem in a file SIZE=1000000 # 1 meg
head -c $SIZE < /dev/zero > file # Set up file of designated size.
losetup /dev/loop0 file # Set it up as loopback device.
mke2fs /dev/loop0 # Create filesystem.
mount -o loop /dev/loop0 /mnt # Mount it.
# Thanks, S.C. |
- mkswap
Creates a swap partition or file. The swap area must
subsequently be enabled with
swapon. - swapon, swapoff
Enable / disable swap partitition or file.
These commands usually take effect at bootup and
shutdown. - mke2fs
Create a Linux ext2 filesystem. This command must
be invoked as root. Example 13-9. Adding a new hard drive #!/bin/bash
# Adding a second hard drive to system.
# Software configuration. Assumes hardware already mounted.
# From an article by the author of this document.
# In issue #38 of "Linux Gazette", http://www.linuxgazette.com.
ROOT_UID=0 # This script must be run as root.
E_NOTROOT=67 # Non-root exit error.
if [ "$UID" -ne "$ROOT_UID" ]
then
echo "Must be root to run this script."
exit $E_NOTROOT
fi
# Use with extreme caution!
# If something goes wrong, you may wipe out your current filesystem.
NEWDISK=/dev/hdb # Assumes /dev/hdb vacant. Check!
MOUNTPOINT=/mnt/newdisk # Or choose another mount point.
fdisk $NEWDISK
mke2fs -cv $NEWDISK1 # Check for bad blocks & verbose output.
# Note: /dev/hdb1, *not* /dev/hdb!
mkdir $MOUNTPOINT
chmod 777 $MOUNTPOINT # Makes new drive accessible to all users.
# Now, test...
# mount -t ext2 /dev/hdb1 /mnt/newdisk
# Try creating a directory.
# If it works, umount it, and proceed.
# Final step:
# Add the following line to /etc/fstab.
# /dev/hdb1 /mnt/newdisk ext2 defaults 1 1
exit 0 |
See also Example 13-8 and Example 28-3. - tune2fs
Tune ext2 filesystem. May be used to change filesystem
parameters, such as maximum mount count. This must be
invoked as root. | This is an extremely dangerous command. Use it at
your own risk, as you may inadvertently destroy your filesystem.
|
- dumpe2fs
Dump (list to stdout) very verbose
filesystem info. This must be invoked as root. root# dumpe2fs /dev/hda7 | grep 'ount count'
dumpe2fs 1.19, 13-Jul-2000 for EXT2 FS 0.5b, 95/08/09
Mount count: 6
Maximum mount count: 20 |
- hdparm
List or change hard disk parameters. This command must be
invoked as root, and it may be dangerous if misused. - fdisk
Create or change a partition table on a storage device,
usually a hard drive. This command must be invoked as
root. | Use this command with extreme caution. If something
goes wrong, you may destroy an existing
filesystem. |
- fsck, e2fsck, debugfs
Filesystem check, repair, and debug command set. fsck: a front end for checking a UNIX
filesystem (may invoke other utilities). The actual
filesystem type generally defaults to ext2. e2fsck: ext2 filesystem checker. debugfs: ext2 filesystem debugger.
One of the uses of this versatile, but dangerous command
is to (attempt to) recover deleted files. For advanced users
only! | All of these should be invoked as root, and they
can damage or destroy a filesystem if misused. |
- badblocks
Checks for bad blocks (physical media flaws) on a
storage device. This command finds use when formatting
a newly installed hard drive or testing the integrity
of backup media.
As an example, badblocks /dev/fd0
tests a floppy disk. The badblocks command
may be invoked destructively (overwrite all data) or in
non-destructive read-only mode. If root user owns the
device to be tested, as is generally the case, then root
must invoke this command. - lsusb, usbmodules
The lsusb command lists all USB
(Universal Serial Bus) buses and the devices hooked up to
them. The usbmodules command outputs
information about the driver modules for connected USB
devices. root# lsusb
Bus 001 Device 001: ID 0000:0000
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 1.00
bDeviceClass 9 Hub
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 8
idVendor 0x0000
idProduct 0x0000
. . .
|
- mkbootdisk
Creates a boot floppy which can be used to bring up the
system if, for example, the MBR (master boot record) becomes
corrupted. The mkbootdisk command is actually
a Bash script, written by Erik Troan, in the /sbin directory. - chroot
CHange ROOT directory. Normally commands are fetched
from $PATH, relative to
/, the default root
directory. This changes the root directory to a different
one (and also changes the working directory to there).
This is useful for security purposes, for instance when
the system administrator wishes to restrict certain users,
such as those telnetting
in, to a secured portion of the filesystem (this
is sometimes referred to as confining a guest user
to a "chroot jail"). Note that after a
chroot, the execution path for system
binaries is no longer valid. A chroot /opt would cause
references to /usr/bin
to be translated to /opt/usr/bin. Likewise,
chroot /aaa/bbb /bin/ls would
redirect future instances of ls
to /aaa/bbb as the base directory,
rather than / as is
normally the case. An alias XX 'chroot /aaa/bbb
ls' in a user's ~/.bashrc
effectively restricts which portion of the filesystem
she may run command "XX" on. The chroot command is also handy
when running from an emergency boot floppy
(chroot to /dev/fd0),
or as an option to lilo when recovering
from a system crash. Other uses include installation from a
different filesystem (an rpm
option) or running a readonly filesystem from a CD ROM.
Invoke only as root, and use with care. | It might be necessary to copy certain system
files to a chrooted directory,
since the normal $PATH can no longer
be relied upon. |
- lockfile
This utility is part of the procmail
package (www.procmail.org).
It creates a lock file, a semaphore file that
controls access to a file, device, or resource. The lock file
serves as a flag that this particular file, device, or resource is
in use by a particular process ("busy"), and
this permits only restricted access (or no access) to other
processes. Lock files are used in such applications as protecting
system mail folders from simultaneously being changed
by multiple users, indicating that a modem port
is being accessed, and showing that an instance of
Netscape is using its cache.
Scripts may check for the existence of a lock file created
by a certain process to check if that process is running.
Note that if a script attempts to create a lock file that
already exists, the script will likely hang. Normally, applications create and check for lock files
in the /var/lock
directory. A script can test for the presence of a lock file by
something like the following.
appname=xyzip
# Application "xyzip" created lock file "/var/lock/xyzip.lock".
if [ -e "/var/lock/$appname.lock" ]
then
... |
- mknod
Creates block or character device files (may be
necessary when installing new hardware on the system). The
MAKEDEV utility has virtually
all of the functionality of mknod,
and is easier to use. - MAKEDEV
Utility for creating device files. It must be run as root,
and in the /dev
directory.
This is a sort of advanced version of
mknod.- tmpwatch
Automatically deletes files which have not been accessed
within a specified period of time. Usually invoked by
cron to remove stale log
files.
Backup - dump, restore
The dump command is an elaborate
filesystem backup utility, generally used on larger
installations and networks.
It reads raw disk partitions and writes a backup file
in a binary format. Files to be backed up may be saved
to a variety of storage media, including disks and tape
drives. The restore command restores
backups made with dump. - fdformat
Perform a low-level format on a floppy disk.
System Resources - ulimit
Sets an upper limit on use
of system resources. Usually invoked with the
-f option, which sets a limit on file size
(ulimit -f 1000 limits files to 1 meg
maximum). The -t option limits the coredump
size (ulimit -c 0 eliminates coredumps).
Normally, the value of ulimit
would be set in /etc/profile
and/or ~/.bash_profile (see Appendix G). | Judicious use of ulimit can
protect a system against the dreaded fork
bomb. #!/bin/bash
# This script is for illustrative purposes only.
# Run it at your own peril -- it *will* freeze your system.
while true # Endless loop.
do
$0 & # This script invokes itself . . .
#+ forks an infinite number of times . . .
#+ until the system freezes up because all resources exhausted.
done # This is the notorious "sorcerer's appentice" scenario.
exit 0 # Will not exit here, because this script will never terminate. |
A ulimit -Hu XX (where
XX is the user process limit) in
/etc/profile would abort
this script when it exceeds the preset limit.
|
- quota
Display user or group disk quotas. - setquota
Set user or group disk quotas from the command line. - umask
User file creation permissions
mask. Limit the default file
attributes for a particular user. All files created
by that user take on the attributes specified by
umask. The (octal) value passed to
umask defines the file permissions
disabled. For example, umask
022 ensures that new files will have at most
755 permissions (777 NAND 022).
Of course, the user may later change the
attributes of particular files with chmod. The usual practice
is to set the value of umask
in /etc/profile and/or
~/.bash_profile (see Appendix G). Example 13-10. Using umask to hide an output file
from prying eyes #!/bin/bash
# rot13a.sh: Same as "rot13.sh" script, but writes output to "secure" file.
# Usage: ./rot13a.sh filename
# or ./rot13a.sh <filename
# or ./rot13a.sh and supply keyboard input (stdin)
umask 177 # File creation mask.
# Files created by this script
#+ will have 600 permissions.
OUTFILE=decrypted.txt # Results output to file "decrypted.txt"
#+ which can only be read/written
# by invoker of script (or root).
cat "$@" | tr 'a-zA-Z' 'n-za-mN-ZA-M' > $OUTFILE
# ^^ Input from stdin or a file. ^^^^^^^^^^ Output redirected to file.
exit 0 |
- rdev
Get info about or make changes to root device, swap space, or video
mode. The functionality of rdev has generally been taken over by
lilo, but rdev remains
useful for setting up a ram disk. This is a dangerous command, if misused.
Modules - lsmod
List installed kernel modules. bash$ lsmod
Module Size Used by
autofs 9456 2 (autoclean)
opl3 11376 0
serial_cs 5456 0 (unused)
sb 34752 0
uart401 6384 0 [sb]
sound 58368 0 [opl3 sb uart401]
soundlow 464 0 [sound]
soundcore 2800 6 [sb sound]
ds 6448 2 [serial_cs]
i82365 22928 2
pcmcia_core 45984 0 [serial_cs ds i82365]
|
| Doing a cat /proc/modules gives the
same information. |
- insmod
Force installation of a kernel module (use
modprobe instead, when possible). Must
be invoked as root. - rmmod
Force unloading of a kernel module. Must be invoked
as root. - modprobe
Module loader that is normally invoked automatically
in a startup script. Must be invoked as root. - depmod
Creates module dependency file, usually invoked from
startup script. - modinfo
Output information about a loadable module. bash$ modinfo hid
filename: /lib/modules/2.4.20-6/kernel/drivers/usb/hid.o
description: "USB HID support drivers"
author: "Andreas Gal, Vojtech Pavlik <vojtech@suse.cz>"
license: "GPL"
|
Miscellaneous - env
Runs a program or script with certain environmental variables
set or changed (without changing the overall system
environment). The [varname=xxx]
permits changing the environmental variable
varname for the duration of the
script. With no options specified, this command lists all
the environmental variable settings. | In Bash and other Bourne shell derivatives, it is
possible to set variables in a single command's environment.
var1=value1 var2=value2 commandXXX
# $var1 and $var2 set in the environment of 'commandXXX' only. |
|
| The first line of a script (the
"sha-bang" line) may use env
when the path to the shell or interpreter is unknown.
#! /usr/bin/env perl
print "This Perl script will run,\n";
print "even when I don't know where to find Perl.\n";
# Good for portable cross-platform scripts,
# where the Perl binaries may not be in the expected place.
# Thanks, S.C. |
|
- ldd
Show shared lib dependencies for an executable file. bash$ ldd /bin/ls
libc.so.6 => /lib/libc.so.6 (0x4000c000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000) |
- watch
Run a command repeatedly, at specified time intervals. The default is two-second intervals, but this may be changed
with the -n option. watch -n 5 tail /var/log/messages
# Shows tail end of system log, /var/log/messages, every five seconds. |
- strip
Remove the debugging symbolic references from an executable
binary. This decreases its size, but makes debugging it
impossible. This command often occurs in a Makefile,
but rarely in a shell script. - nm
List symbols in an unstripped compiled binary. - rdist
Remote distribution client: synchronizes, clones,
or backs up a file system on a remote server.
|
|