bash scripting

Unix shell scripting

This document attempts to be a bare-bones outline for how to write scripts under Unix, and how to use crontab to run scripts at regular intervals.

A much more complete reference can be found at http://tldp.org/LDP/abs/html.

I will assume that you are using bash, or some other similar shell like zsh or ksh. Plain sh won't do some of the things here, and csh is just too different.

Executables and your PATH

A bash script is just a text file that contains unix commands. If the file is named foo, then you can run it with bash -c foo, but it is much more common to make the file executable, with chmod +x foo, and then put it somewhere in your PATH. Once it is executable you can run it with ./foo without worrying about your PATH, but it is common to create a directory $HOME/bin and then append this to your PATH with PATH=$PATH:$HOME/bin. This means that just typing foo will now run your script, regardless of whether you've changed directory to somewhere else.

Most people put the PATH assignment above into their bash startup file, .bashrc. Note that if you want your script to run in an environment where .bashrc isn't seen, such as crontab, you should either set the PATH or use a fully-qualified name ($HOME/bin/foo).

Redirection

You can direct the output of a command into a file with the > operator. If you type ls > ls.out, then the output of the ls command goes to the file ls.out; this works for built-in commands like ls and also your own commands like foo.

If you want to append to a log file, use >> instead. The following would append three lines of output to the file log.out:
    date >> log.out
    uptime >> log.out
    snmpget -v 1 -c public snmphost system.1.0   >> log.out      # assume this generates one line of output
You can also create a single script file foo containing
    date
    uptime
    snmpget ....
and then redirect the whole thing with foo >> log.out. For complex logging, this is often simpler.

The > and >> operators redirect the standard output. If you also want to redirect the standard error, to get your error messages, you can do it with 2>, but the most common idiom is 2>&1, which means to redirect the standard error (file descriptor 2) to wherever the standard output is currently heading. Note that you must redirect stdout before invoking 2>&1.

Shell Variables

The shell can use string variables for both programming and for simple bookkeeping. The variables HOME and PATH were used above; to view these, the env command will print them all or you can use echo $HOME, etc. Note that to access the value of a variable, you need to prepend a $.

Getting command output into a shell variable

The echo command takes what is on the command line (including variables) and prints it out (that is, sends it to stdout). If you want to go the other way, capturing the output of a command into a shell variable, use:
LS_OUT=$(ls -l myfile.stuff)
It is simplest if you do this with commands that you know will produce only a single line of output.

Parsing command output lines

Often command output is intended to be human-readable and thus has lots of stuff in it that you don't want if you're feeding it back into another script. For example, the ps command will list all sorts of information about a process, but perhaps all you want is the process Id, which is field 1 (after a leading space). If the delimitering is based on single separation characters (eg TAB or : or ;), or is based on fixed column counts, consider the cut command. For free-form output that forms distinct tokens, awk works well. To get the first word of output, use awk '{ print $1}' (note the quotes!!). $2 will give you the second word.

Looping and Conditionals

Shell conditionals are done with the if command; looping with the while and for commands (there is also case, and a few other forms). For looping, if you want a loop variable to increment, consider using the expr command that evaluates the rest of the command line as an arithmetical expression. Thus, if you are using the shell variable INDEX in your loop, currently 15, then
    expr $INDEX + 1       # note the spaces
shell-evaluates to
    expr 15 + 1
which then would print 16 to its stdout. Typically you want this back into the variable INDEX, which would be achieved with
    INDEX=$(expr $INDEX + 1)

Parameters

Shell commands have parameters. The first parameter is $1, then $2, etc; $0 is the name of the command. $* is the entire command line and $# is the number of parameters. For the hard-core, there is also shift.

crontab

The cron daemon runs commands at designated intervals; the file of your personal cron entries is crontab. Nowadays there is usually also a command called crontab that edits this file: crontab -e.

A crontab line consists of five date/time fields and a command, separated by spaces. We'll only consider the first two of the date/time fields here, for minutes and hours; a * in the other fields means to run on any date when the specified minute and hour is reached. If you want to run a command every 4 hours, the following crontab entry will do that:
0 0,4,8,12,16,20 * * * $HOME/bin/foo
If you want to run a command every fifteen minutes, the following will do:
0,15,30,45 * * * * $HOME/bin/theCommand