Chapter 1

Introduction

This is the reference manual for scsh, a Unix shell that is embedded within Scheme. Scsh is a Scheme system designed for writing useful standalone Unix programs and shell scripts -- it spans a wide range of application, from ``script'' applications usually handled with perl or sh, to more standard systems applications usually written in C.

Scsh comes built on top of Scheme 48, and has two components: a process notation for running programs and setting up pipelines and redirections, and a complete syscall library for low-level access to the operating system. This manual gives a complete description of scsh. A general discussion of the design principles behind scsh can be found in a companion paper, ``A Scheme Shell.''

1.1  Copyright & source-code license

Scsh is open source. The complete sources come with the standard distribution, which can be downloaded off the net. Scsh has an ideologically hip, BSD-style license.

We note that the code is a rich source for other Scheme implementations to mine. Not only the code, but the APIs are available for implementors working on Scheme environments for systems programming. These APIs represent years of work, and should provide a big head-start on any related effort. (Just don't call it ``scsh,'' unless it's exactly compliant with the scsh interfaces.)

Take all the code you like; we'll just write more.

1.2  Obtaining scsh

Scsh is distributed via net publication. We place new releases at well-known network sites, and allow them to propagate from there. We currently release scsh to the following Internet sites:

ftp://ftp.scsh.net/pub/scsh http://prdownloads.sourceforge.net/scsh/

Each should have a compressed tar file of the entire scsh release, which includes all the source code and the manual, and a separate file containing just this manual in Postscript form, for those who simply wish to read about the system.

However, nothing is certain for long on the Net. Probably the best way to get a copy of scsh is to use a network resource-discovery tool, such as archie, to find ftp servers storing scsh tar files. Take the set of sites storing the most recent release of scsh, choose one close to your site, and download the tar file.

1.3  Building scsh

Scsh currently runs on a fairly large set of Unix systems, including Linux, FreeBSD, OpenBSD, NetBSD, MacOS X, SunOS, Solaris, AIX, NeXTSTEP, Irix, and HP-UX. We use the Gnu project's autoconfig tool to generate self-configuring shell scripts that customise the scsh Makefile for different OS variants. This means that if you use one of the common Unix implementations, building scsh should require exactly the following steps:

gunzip scsh.tar.gz	Uncompress the release tar file.
untar xfv scsh.tar	Unpack the source code.
cd scsh-0.6.x	Move to the source directory.
./configure	Examine host; build Makefile.
make	Build system.

When you are done, you should have a virtual machine compiled in file scshvm, and a heap image in file scsh/scsh.image. Typing

make install

will install these programs in your installation directory (by default, /usr/local), along with a small stub startup binary, scsh.

If you don't have the patience to do this, you can start up a Scheme shell immediately after the initial make by simply saying

./scshvm -o ./scshvm -i scsh/scsh.image

See chapter 11 for full details on installation locations and startup options.

It is not too difficult to port scsh to another Unix platform if your OS is not supported by the current release. See the release notes for more details on how to do this.

1.4  Caveats

It is important to note what scsh is not, as well as what it is. Scsh, in the current release, is primarily designed for the writing of shell scripts -- programming. It is not a very comfortable system for interactive command use: the current release lacks job control, command-line editing, a terse, convenient command syntax, and it does not read in an initialisation file analogous to .login or .profile. We hope to address all of these issues in future releases; we even have designs for several of these features; but the system as-released does not currently provide these features.

1.5  Naming conventions

Scsh follows a general naming scheme that consistently employs a set of abbreviations. This is intended to make it easier to remember the names of things. Some of the common ones are:

fdes

Means ``file descriptor,'' a small integer used in Unix to represent I/O channels.

...*

A given bit of functionality sometimes comes in two related forms, the first being a special form that contains a body of Scheme code to be executed in some context, and the other being a procedure that takes a procedural argument (a ``thunk'') to be called in the same context. The procedure variant is named by taking the name of the special form, and appending an asterisk. For example:

;;; Special form:
(with-cwd "/etc"
  (for-each print-file (directory-files))
  (display "All done"))

;;; Procedure:
(with-cwd* "/etc"
  (lambda ()
    (for-each print-file (directory-files))
    (display "All done")))

action/modifier

The infix ``/'' is pronounced ``with,'' as in exec/env -- ``exec with environment.''

call/...

Procedures that call their argument on some computed value are usually named ``call/...,'' e.g., (call/fdes port proc), which calls proc on port's file descriptor, returning whatever proc returns. The abbreviated name means ``call with file descriptor.''

with-...

Procedures that call their argument, and special forms that execute their bodies in some special dynamic context frequently have names of the form with-.... For example, (with-env env body₁ ...) and (with-env* env thunk). These forms set the process environment body, execute their body or thunk, and then return after resetting the environment to its original state.

create-

Procedures that create objects in the file system (files, directories, temp files, fifos, etc.), begin with create-....

delete-

Procedures that delete objects from the file system (files, directories, temp files, fifos, etc.), begin with delete-....

record:field

Procedures that access fields of a record are usually written with a colon between the name of the record and the name of the field, as in user-info:home-dir.

%...

A percent sign is used to prefix lower-level scsh primitives that are not commonly used.

-info

Data structures packaging up information about various OS entities frequently end in ...-info. Examples: user-info, file-info, group-info, and host-info.

Enumerated constants from some set s are usually named s/const₁, s/const₂, .... For example, the various Unix signal integers have the names signal/cont, signal/kill, signal/int, signal/hup, and so forth.

1.6  Lexical issues

Scsh's lexical syntax is just R5RS Scheme, with the following exceptions.

1.6.1  Extended symbol syntax

Scsh's symbol syntax differs from R5RS Scheme in the following ways:

• In scsh, symbol case is preserved by read and is significant on symbol comparison. This means

  (run (less Readme))

  displays the right file.

• ``-'' and ``+'' are allowed to begin symbols. So the following are legitimate symbols:

  -O2 -geometry +Wn

• ``|'' and ``.'' are symbol constituents. This allows | for the pipe symbol, and .. for the parent-directory symbol. (Of course, ``.'' alone is not a symbol, but a dotted-pair marker.)

• A symbol may begin with a digit. So the following are legitimate symbols:

  9x15 80x36-3+440

1.6.2  Extended string syntax

Scsh strings are allowed to contain the ANSI C escape sequences such as \n and \161.

1.6.3  Block comments and executable interpreter-triggers

Scsh allows source files to begin with a header of the form

#!/usr/local/bin/scsh -s

The Unix operating system treats source files beginning with the headers of this form specially; they can be directly executed by the operating system (see chapter 11 for information on how to use this feature). The scsh interpreter ignores this special header by treating #! as a comment marker similar to ;. When the scsh reader encounters #!, it skips characters until it finds the closing sequence newline/exclamation-point/sharp-sign/newline.

Although the form of the #! read-macro was chosen to support interpreter-triggers for executable Unix scripts, it is a general block-comment sequence and can be used as such anywhere in a scsh program.

1.6.4  Here-strings

The read macro #< is used to introduce ``here-strings'' in programs, similar to the << ``here document'' redirections provided by sh and csh. There are two kinds of here-string, character-delimited and line-delimited; they are both introduced by the #< sequence.

1.6.4.1  Character-delimited here-strings

A character-delimited here-string has the form

#<x...stuff...x

where x is any single character (except <, see below), which is used to delimit the string bounds. Some examples:

Here-string syntax	Ordinary string syntax
#<|Hello, world.|	"Hello, world."
#<!"Ouch," he said.!	"\"Ouch,\" he said."

There is no interpretation of characters within the here-string; the characters are all copied verbatim.

1.6.4.2  Line-delimited here-strings

If the sequence begins "#<<" then it introduces a line-delimited here-string. These are similar to the ``here documents'' of sh and csh. Line-delimited here-strings are delimited by the rest of the text line that follows the "#<<" sequence. For example:

#<<FOO
Hello, there.
This is read by Scheme as a string,
terminated by the first occurrence
of newline-F-O-O-newline or newline-F-O-O-eof.
FOO

Thus,

#<<foo
Hello, world.
foo

is the same thing as

"Hello, world."

Line-delimited here-strings are useful for writing down long, constant strings -- such as long, multi-line format strings, or arguments to Unix programs, e.g.,

;; Free up some disk space for my netnews files.
(run (csh -c #<<EOF
cd /urops
rm -rf *
echo All done.

EOF
))

The advantage they have over the double-quote syntax (e.g., "Hello, world.") is that there is no need to backslash-quote special characters internal to the string, such as the double-quote or backslash characters.

The detailed syntax of line-delimited here-strings is as follows. The characters "#<<" begin the here-string. The characters between the "#<<" and the next newline are the delimiter line. All characters between the "#<<" and the next newline comprise the delimiter line -- including any white space. The body of the string begins on the following line, and is terminated by a line of text which exactly matches the delimiter line. This terminating line can be ended by either a newline or end-of-file. Absolutely no interpretation is done on the input string. Control characters, white space, quotes, backslash -- everything is copied as-is. The newline immediately preceding the terminating delimiter line is not included in the result string (leave an extra blank line if you need to put a final newline in the here-string -- see the example above). If EOF is encountered before reading the end of the here-string, an error is signalled.

1.6.5  Dot

It is unfortunate that the single-dot token, ``.'', is both a fundamental Unix file name and a deep, primitive syntactic token in Scheme -- it means the following will not parse correctly in scsh:

(run/strings (find . -name *.c -print))

You must instead quote the dot:

(run/strings (find "." -name *.c -print))

When you write shell scripts that manipulate the file system, keep in mind the special status of the dot token.

1.7  Record types and the define-record form

Scsh's interfaces occasionally provide data in structured record types; an example is the file-info record whose various fields describe the size, protection, last date of modification, and other pertinent data for a particular file. These record types are described in this manual using the define-record notation, which looks like the following:

(define-record ship
  x
  y
  (size 100))

This form defines a ship record, with three fields: its x and y coordinates, and its size. The values of the x and y fields are specified as parameters to the ship-building procedure, (make-ship x y), and the size field is initialised to 100. All told, the define-record form above defines the following procedures:

An implementation of define-record is available as a macro for Scheme programmers to define their own record types; the syntax is accessed by opening the package defrec-package, which exports the single syntax form define-record. See the source code for the defrec-package module for further details of the macro.

You must open this package to access the form. Scsh does not export a record-definition package by default as there are several from which to choose. Besides the define-record macro, which Shivers prefers¹, you might instead wish to employ the notationally-distinct define-record-type macro that Jonathan Rees prefers². It can be found in the define-record-types structure.

Alternatively, you may define your own, of course.

1.8  A word about Unix standards

``The wonderful thing about Unix standards is that there are so many to choose from.'' You may be totally bewildered about the multitude of various standards that exist. Rest assured that nowhere in this manual will you encounter an attempt to spell it all out for you; you could not read and internalise such a twisted account without bleeding from the nose and ears.

However, you might keep in mind the following simple fact: of all the standards, POSIX is the least common denominator. So when this manual repeatedly refers to POSIX, the point is ``the thing we are describing should be portable just about anywhere.'' Scsh sticks to POSIX when at all possible; its major departure is symbolic links, which aren't in POSIX (see -- it really is a least common denominator).