|
This video is about a few systems programming facilities in sh.
Understanding this video requires understanding the first four shell
programming videos, plus some of the basics about how processes work in unix.
|
|
First, let's examine the "exec" keyword in sh.
In most cases when the shell invokes other programs, it does so in accordance
with the standard fork/exec/wait idiom: The shell forks, the child process
execs your command, and the parent process waits for the child process to
exit before the parent proceeds.
The "exec" keyword means: don't fork(). So for commands which invoke a
program, the shell program is replaced by that other program.
The 'exec' caused the echo command to replace the shell program, so the rest
of that shell script was not executed — the "echo goodbye" was executed, and
the echo program terminated, and that was it.
|
|
What happens if we do a redirection to the file 'output' without any command
to execute?
The file is created, but that's it. The shell forks as usual; the child
process opens the file for write... and that's it! Then we continue with the
next command.
|
|
Now let's combine this with exec.
What happened here is that as I said in the beginning, the 'exec' keyword
makes the shell refrain from the fork().
So rather than forking and having the child redirect the standard output, the
main shell itself redirects the standard output.
This means that all subsequent commands executed by this shell script will
have that file as their standard output, until some other redirection occurs.
This can be quite useful in a shell script; for example, a shell script which
generates some file.
You can tell people to run your shell script directed out to the appropriate
file, and sometimes that works best; but you can alternatively do the
redirection yourself using exec.
|
|
Here's another modification to the standard fork/exec/wait idiom.
First, for the basic example:
In a command-line like
"prog1; prog2"
the prog2 command will not be started until after prog1 has completed.
However, if we replace the semicolon with an ampersand:
"prog1 & prog2"
then this means to run both programs simultaneously.
The shell will fork; the child process will exec prog1; but then the shell
will not wait for this child process, but will proceed with forking
again and execing prog2.
The syntax is a little bit out of sync with the concept behind the syntax.
Interactively, we can write simply
"prog1 &",
and then we will get our shell prompt back immediately, while prog1 starts
executing. More user-friendly modern versions of sh will asynchronously tell
us when prog1 exits, too, but we're not going to get into that right now.
We tend to call this a "background process" — it runs in the background while
you continue to execute further commands.
In a shell script, we can also start programs to run concurently in this way,
either by replacing the separating semicolon with an ampersand, or simply by
putting an ampersand at the end of a command-line.
|
|
We might then want to know the process ID of a so-called "background" process
— we'll have a use for it in two commands I'll mention shortly.
The special variable
"$!"
is the process ID number of the most-recently-started background process.
So we could write something like this...
and then pid1 would contain the process ID of the running prog1 and pid2 the
process ID of prog2.
|
|
We have the expected difference between
this
as opposed to
this.
Given the semicolon, in both cases prog1 and prog2 will run sequentially,
not simultaneously; but in the case on the right prog3 will be executed
immediately after prog1 is started, whereas in the case on the left,
everything waits for prog1 to terminate, then we start prog2 and then
immediately start prog3 without waiting for prog2.
|
|
Since an ampersand suppresses the normal 'wait', there is a way to cause the
wait to happen later.
The sh command for this is "wait".
A wait command with no arguments waits for all outstanding background
processes.
Alternatively, you can say wait and a process ID number.
For example, we can use that process ID number we saved earlier for prog2:
"wait $pid2"
Something else you can do other than to wait for one of these processes to
exit on their own is to send them a signal.
Since the commonest use of this is to terminate the process, the command is
named "kill".
If we write
"kill 41"
this sends signal 15 to process 41.
Of course that process ID number is often from a variable.
You might even be able to use $! here directly, although using $! any time
more than a line or two after the ampersanded command can be error-prone since
$! means the last background process ID number so it's easily overwritten by
new events. So usually we assign $! to a normal variable immediately, if we
are going to want to use it later.
If you want to send a signal number other than 15, you make it an option by
prefacing it with a minus sign;
For example,
"kill -3 41"
sends signal 3 to process 41, and
"kill -3 41 48 59"
sends signal 3 to each of these three processes.
This looks like a standard unix command-line option, but it's not quite.
For example,
"kill -12 41"
might look like there are separate options -1 and -2, but that's not what it
means — it just sends signal 12 to process ID 41.
It's not a standard option format; it's a minus sign and then a number, of
however many digits.
|
|
The last topic in this video is about making your shell scripts executable,
so that you can execute them by typing their path name, without having to
type "sh" as part of the command.
If we have a shell script:
...
so far we've been executing it by typing
"sh s7".
This makes sense, but it makes it seem like not a normal program.
We want to be able to type something like
"./s7",
or maybe even simply "s7" depending on the value of your PATH variable.
But of course,
you get an error from typing this, because the file is not executable:
Now, the naïve approach might be to say, hey, I know the chmod command!
...
Surely that can't work?
How can that possibly work?
What happens is, just to make things like this work, when the shell tries to
exec a command, if it gets an error, as it will from trying to exec this file
which is not a machine language program nor any other format of executable
recognized by the operating system kernel, if the file is nevertheless marked
as executable, the shell figures it must be a shell script so it goes ahead
and executes it as one! It is the shell, after all; the shell is right here,
it can go ahead and execute this shell script even without explicit operating
system assistance.
Modern versions of sh are cleverer about this in that they won't
attempt to execute certain files which are certainly not shell scripts but
are, for example, machine language programs but for a different CPU
architecture. But that's a detail. The basic idea is that a failing exec on
an executable file means it's a shell script, so the shell goes ahead and
executes it.
This doesn't give us everything we want, in two ways, which is why it's been
improved upon. The above still works, but consider two things.
First of all, the other program trying to exec our shell script might not be
the shell. Suppose we write a C program which tries to exec another program
and that other program is our shell script? Or for an example which comes
up a lot these days, suppose we write our program to run under a web server
based on certain web requests from the net? The web server isn't the shell;
it will call exec on our program, but it won't have a fallback of executing
it as a shell script when the exec fails.
Another problem is that these days there are at least two shells, sh and csh.
Their programming languages are completely different.
You might write a shell script for one of them, and then when it's executed
you want it to be executed with the correct shell automatically.
In fact, the mechanism I'm about to discuss works for executing any interpreted
programming language, so it can also be used to mark a python program so that
you can chmod +x it and make the python interpreter automatically be executed
when someone tries to execute your program. Or any other interpreter.
|
|
The way it works is by putting this logic back into the operating system
kernel.
When you exec a file, the kernel is already examining the first few bytes of
the file, which for most binary file formats contains what we call a "magic
number".
This tells all sorts of software what kind of file it is.
For machine language files it says what platform it's for, so the operating
system can gracefully refuse to exec a machine language program written for
an incompatible CPU or operating system.
There are standard magic numbers for various image file formats too, and so
on, all sorts of binary files.
And, there's a particular magic number, which corresponds to two bytes in the
ASCII character range, which tells the kernel to read the rest of that line,
up to a newline character, to tell it what interpreter to run!
The two characters are the number sign and an exclamation point.
If we
preface our file with
"#!/bin/sh",
then the operating system knows to use /bin/sh as the interpreter.
This looks awfully similar. But in this case it's happening in the operating
system kernel, it's not a feature of the shell. And it doesn't have to
specify /bin/sh — I'll give examples of this in a moment.
These two characters are cleverly chosen... or at least the first one is.
Because it does execute this file as a shell script.
So why doesn't it give a syntax error for the #!/bin/sh line?
Because the number sign introduces a comment in sh.
|
|
The number sign also introduces a comment in Python, so you can do this with
python just the same.
|
|
And we can use other programs similarly. Suppose we have a file containing
some text. We can make it self-printing by using 'cat':
...
This is a bit weak, though, because we get to see the #! line.
So instead, let's use sed to delete line 1 of the file.
Of course you would probably be using this for more text than just one line.
This can be applied to any format of file and any program to process that
file, so long as either that file-processing program treats '#' as introducing
a comment to end of line, or there is some other way to make it ignore the
first line as we just did by using sed.
|