Differences between BSD 4.3 csh and the Athena csh Draft Version 1.0A

Theodore Y. Ts’o

July 15, 1988

Contents

1 Introduction 3 1.1 History, or the geneology of tcsh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 Structure of this document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Internal code changes 5 2.1 BSD 4.2 compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2 New version of printf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.3 Code cleanup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3 Additions to the user interface 6 3.1 A new configuration file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.2 The tcsh version builtin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.3 The where builtin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.4 Line editing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.5 Warning messages for non-zero exit status . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.6 Automatic execution of a command before each prompt . . . . . . . . . . . . . . . . . . . 7 3.7 Expanded prompt formatting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.8 A termination script for non-login shells . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.9 Limiting the number of completions to be listed . . . . . . . . . . . . . . . . . . . . . . . . 8 3.10 Time-stamped history list . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.11 Alternative semantics for “pushd +n” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.12 Accessing entries in the directory stack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.13 Extension to file globbing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.14 Byte-swapped binaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.15 Automatic TERMCAP handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.16 Printing the directory of jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.17 New shell variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.18 File suffix of symbolic links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

4 Bug fixes 10 4.1 Incorrect error message with exit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 4.2 Error recovery of getwd() during shell startup . . . . . . . . . . . . . . . . . . . . . . . . . 10 4.3 Logging out in non-login shell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 4.4 Freeing strings and then using them . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 4.5 Erroneously reported stopped jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

5 Optional features not currently enabled 11 5.1 Automatic logout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 5.2 Login monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

6 Outstanding issues and bugs 12 6.1 Relative pathnames and command completion . . . . . . . . . . . . . . . . . . . . . . . . . 12 6.2 Default binding of control-T . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 6.3 Control-characters in setting keyboard bindings . . . . . . . . . . . . . . . . . . . . . . . . 12 6.4 Portability to different operating systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 6.5 Errors in logout scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 6.6 Killing process groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 6.7 RT compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

7 Suggested enhancements 14 7.1 Signal driven activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 7.2 Sourcing a file at startup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 7.3 Improving history expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 7.4 Security hacks and dot files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

8 Differences in source files 15

1 Introduction

1.1 History, or the geneology of tcsh

The version of csh that is currently running at MIT Project Athena had its beginnings in an alternative shell called tcsh, which was supported by the Student Information Processing Board (SIPB). Originally, tcsh was a set of changes to the standard Berkeley 4.2 csh written by Ken Greer at HP in 1983 adding Tenex-style file name completion to csh — hence the name “tcsh”. These modifications were the basis for at least three different tcshs which have been diverging and cross-breeding since then.

The first of these versions had its beginnings in modifications made at MIT by the Student Information Processing Board (and perhaps at S1?). A line editor and the “where” builtin were added by Earl Cohen, Bill Saphir, and Dan Morgan. This version of tcsh was supported by SIPB and was very popular; many users choose tcsh as their login shell. In the summer of 1987,^{1} Project Athena adopted it as the default shell, replacing /bin/csh.

Another version of tcsh became incorporated into the BSD 4.3 sources, giving the BSD 4.3 csh file completion which could be accessed by hitting the escape and control-D keys. When the BSD 4.3 was released, the tcsh that was supported by SIPB was merged with the BSD 4.3 csh. This can still be seen today by the partial duplication of some of the file completion code. (done in the name of providing as much compatibility with the BSD 4.3 csh as possible when line editing was turned off)

Yet another version of tcsh was wrtting by Paul Placeway at Ohio State, who added a different line editor to tcsh, as well as some additional features such as automatically logging out a user if was idle for more than a specified time. This was published in net.sources in late 1984. (It is not clear if any features from the 1984 posting were imported into the tcsh at Athena, but it appears to be very likely.) Another, more featureful version was published in comp.sources.unix in the summer of 1987. Both versions were based on the BSD 4.2 csh.

Many new features from the Ohio State tcsh were incorporated into the Project Athena tcsh in the fall of 1987. These included an improved prompt format (section 3.7), execution of a command prior to each prompt (section 3.6), new ways of accessing entries in the directory stack (sections 3.11 and 3.12), expanded file globbing (section 3.13), and several new shell variables (section 3.17).

In addition, at that time I added several features of my own. Those included dynamic keymaps for the line editor (section 3.4), optional warning messages for processes that exit with a non-zero exit status (section 3.5), optional sourcing of a shutdown script when a non-login shell terminates (section 3.8), optional automatic TERMCAP updating (section 3.15), printing the directory of a process when it terminates (section 3.16), and detecting the attempted execution of byte-swapped binaries (section 3.14). Several bugs were also fixed as documented in the Bugs section of this document.

1.2 Structure of this document

Differences between the C shell (hereafter known as csh) as shipped in the BSD 4.3 release tapes and the csh as currently being run by Athena can be roughly be divided into several areas:

1. Internal code changes having no effect on the user interface

2. Changes to the user interface 3. Bug fixes

1Date and release when this happened is not certain

4. Optional features not currently enabled

2 Internal code changes

These differences were usually made for compatibility or performance reasons; in case of a new release from Berkeley, it would not be critical (although it might be nice) for these changes to be integrated in to a new version of csh.

2.1 BSD 4.2 compatibility

The code is conditionalized so that it may be run on BSD 4.2 systems. It is not known if csh can still be compiled without any changes on BSD 4.2 systems.

2.2 New version of printf

The minimal printf used by csh is written in C, instead of the VAX assembly language version provided by Berkeley. (This change we probably want to keep for those who prefer to use RT’s.)

2.3 Code cleanup

The code was cleaned up with help from suggestions made by lint and saber. The changes were mostly casts and function and variable declarations.

3 Additions to the user interface

Changes to the user interface were designed to be as transparent as possibly, to retain backwards compatibility with the BSD csh. In many cases, additional features must be enabled by setting certain shell variables or certain aliases. In other cases, the names of the new builtin commands were chosen so that would not likely conflict with an executable program which a user might want to execute.

3.1 A new configuration file

When the Athena csh is started, it first sources the file ˜ /.tcshrc (if it exists), before it sources the .cshrc file. This allows user to place commands which are specific to the Athena csh in the .tcshrc file, so that the .cshrc may remain compatible with the Berkeley version of the shell.

3.2 The tcsh version builtin

The tcsh version is a new builtin command. It prints a version string which allows the user to determine which version csh he or she is running. This is most useful when trying to debug a problem which is due to version skew or due to the user running an unauthorized version of the shell.

3.3 The where builtin

The “where” builtin was added to the Athena csh. It allows the user to see where or what the csh would execute if the user executed a particular command. It searches through the list of aliases, shell builtins, and executable files in the user’s path. If possible, it will take advantage of csh’s hashing scheme.

3.4 Line editing

If the shell variable lineedit is turned on, an emacs-like line editor takes the place of the character-, word-, and line-erase features of the tty driver. The line editor has most of the simple editing commands available in Emacs to move around and change the current line. In addition, the line editor may be used to step forwards and backwards through the shell’s history list.

The line editor works on any CRT terminal which meets the following requirements:

• Sending a backspace character (or some other character defined in the “le” entry in the terminal’s termcap) to the terminal must move the cursor one column to the left without erasing the character in that column.

• sending a space character to the terminal must replace the character in the current column with a blank space.

The line editor is based on the concept of keymaps which is a table of line editor functions and their arguments (if the function takes an argument), with an entry for every ascii character.

Initially, the key strokes are interpreted in the context of the starting keymap. When a key is pressed, the line editor looks up the function associated with that key in the current keymap, resets the current keymap to the starting keymap, and executes the line editor function in the keymap. The special line editor function prefix changes the current keymap to the keymap specified in its argument. In this way, the next key pressed will be interpreted in another keymap.

For example, by default, the starting keymap is the pre-defined keymap base, which has all of the printing characters bound to the line editor function self-insert and the control characters to line editor functions which emulate their equivalent emacs functions. In addition the escape key is bound to prefix with an argument of meta. The meta keymap contains the mappings of the emacs commands whose first character is an escape — the meta-commands.

3.5 Warning messages for non-zero exit status

If the shell variable print exit status is set, then csh will print a warning message if an executed command has a non-zero exit status. This is useful for programs which signal errors only by non-zero exit status returns^{2}. It is also useful for detecting programs which do not clear their exit status when they terminate.

3.6 Automatic execution of a command before each prompt

If the alias precmd is defined, then the shell will execute that alias before each prompt is printed. This allows users who wish to customize their prompts every time they execute a commands the ability to do so.

3.7 Expanded prompt formatting

The format for the prompt shell variable has been changed to include many new things, such as the current time of day, current working directory, etc.. The new format uses “%<char>” to signal an expansion, much like printf. The available sequences are:

Format command Description
%d  Current working directory.
%c  Trailing component of cwd.
%h or (%!)  Current history event number
%M  The full machine hostname
%m  The hostname up to the first “.”
%S  Start (stop) standout mode
%s  Stop standout mode
%t  Current time-of-day, in 12 hour, am/pm format.
%%  A single %.

If some other character follows the percent sign, both the percent sign and the unknown character will be displayed. Also, for backwards compatibility, then a backslashed exclamation point will be replaced with the current history event number.

The shell variable prompt2 controls the prompt printed while the shell is prompting for the body of for and while loops, and when the shell is ignoring commands due to an if statement.

2Such as the 5.5 version of attach

3.8 A termination script for non-login shells

If the shell variable exitscript is set, when a non-login shell exits, it will try to source the file named by exitscript, analogous to the .logout file. This file will typically include commands to clean up the environment, such as destroying root kerberos tickets. Note that since a shell variable must be set for this processing to occur, its use will not interfere with shell scripts.

3.9 Limiting the number of completions to be listed

If the shell variable maxcompletions is set, then if there are more possible completions than the number specified by the shell variable, instead of listing the possible completions, csh will just print the number of possible completions. Thus the user will not be inundated by hundreds of completions.

3.10 Time-stamped history list

The “history” builtin accepts a new option, “-t”. If this option is specified, a time-stamp indicating when each command was executed is printed along with each command. This allows the user to keep track of when the various events occurred. The time stamps are not maintained on the saved history list; thus, on logging back in, all the saved history events will be recorded with the login time as their time stamp. This was done since the format of the .history needed to be preserved for compatibility concerns.

3.11 Alternative semantics for “pushd +n”

If the shell variable dextract is set, then the command “pushd +n” extracts the nth directory from the directory stack and places it on the top of the stack (thereby changing to current working directory to it), instead of rotating the stack so that the nth entry is on the top. Some users find this behavior more useful.

3.12 Accessing entries in the directory stack

If the shell variable stack hack is set, the “=” character becomes a magic character like “*” or “?” during file globbing. Any instance of “=n” becomes replaced by the nth entry on the directory stack. This is highly useful; however, it conflicts with the common usage of using the equals sign in X geometry specifications — if stack hack is enabled, then the equals sign must be backslashed to avoid this expansion.

3.13 Extension to file globbing

The syntax for any character in a range (for example “.[a-z]*”) has been extended so as to conform with standard Unix regular expression syntax. Specifically, after an open bracket (“[”), if the first character is a caret (“ˆ ”) then the character matched will be any character not in the range specified.

3.14 Byte-swapped binaries

Before csh attempts to run a program, it checks to see if the magic number for the executable is byteswapped. If so, the binary was probably a Vax binary that the user attempted to run on a RT, or vice versa. Instead of attempting to feed the erroneous binary to /bin/sh causing many confusing error messages, csh prints an error message.

3.15 Automatic TERMCAP handling

If the environment variable TERMCAP is set and it does not contain a filename to a termcap file, csh will try to automatically update TERMCAP in the following instances:

• If the user sets the shell variable term or the environment variable TERM, TERMCAP will be updated to reflect the new terminal type.

• In addition, if the shell receives a SIGWINCH signal, which means that the window which the shell is running in has changed size, TERMCAP will be updated to reflect the new size of the window.

3.16 Printing the directory of jobs

When a child process of csh changes state (by being stopped, killed, or terminated), the shell will print a message describing the changed state of the process to the user. In the Athena csh, if the current directory of the shell does not match the current working directory of the process when it was started, it will also print the current working directory of the process when it was started. This is useful when a user has three or four jobs with the same name, but which were started in different directories (like “make”). This features allows the user to distinguish between these jobs.

3.17 New shell variables

The following shell variables are automatically set by the Athena csh when it start up:

Variable Description
uid  Contains the user id number of the current user
host  Contains the machine name of the current host
hosttype Contains the type of the computer (“vax” or “rt”)

3.18 File suffix of symbolic links

During file completion, when a list of possible file completions is being printed, symbolic links which do not point to a valid file or directory are printed with a suffix of “?” instead of “@”. (Like the BSD csh, if the symbolic link points to a directory, the symbolic link is printed with a suffix of “>”.)

4 Bug fixes

These are bug fixes which have been made locally. Most have not be reported to Berkeley.

4.1 Incorrect error message with exit

The BSD csh complains that there are stopped jobs when the “exit” command is used to terminate the processing of a source’d file, although the exit takes place anyway, as it should. This has been fixed. (Not reported to Berkeley)

4.2 Error recovery of getwd() during shell startup

In the BSD 4.3 csh, if the shell cannot access the current working directory, it prints its error message to a closed file descriptor (so no error message would be output) and exits. This behavior was extremely mystifying to the user. The Athena csh prints the error message to standard output and in addition it prints a message reminding the user of the most common reason why this error message would happen: “Did you su in a fascist directory?” (Not reported to Berkeley)

4.3 Logging out in non-login shell

If the user uses the logout builtin command in a non-login shell, and there are stopped processes, the user will receive the error message “There are stopped jobs.” This may be factually accurate, but it is not the correct error message, which should be “Not login shell.” (Not reported to Berkeley.)

4.4 Freeing strings and then using them

In many sections of the code, BSD csh would free strings and then use them in calls to the subroutine Perror, since that subroutine would not return control to the calling routine. While it works, this is dependent on the implementation of malloc and is generally not recommended. Athena csh uses a new subroutine, PerrorF, which uses the passed argument to print the error message and then frees it.

4.5 Erroneously reported stopped jobs

When a child processing belonging to csh exits, csh marks its associated process information structure to be freed by setting its pid to zero. In certain situations, it is possible for csh to look at this structure before it gets freed when csh is determining if there are any stopped jobs or not. This code was changed so that it checks if the structure contains valid information (by checking if the pid is nonzero). (Not reported to Berkeley)

5 Optional features not currently enabled

5.1 Automatic logout

If AUTOLOGOUT is defined in sh.local.h, then code will be included in csh such that if the shell variable autologout is set, then after the number seconds of inactivity specified by the value of autologout, csh will automatically logout the user. This feature has been disabled, since many users still have autologout set from the days of time-sharing, and this features is extremely dangerous in a window environment.

5.2 Login monitoring

If WATCHLOGINS is defined in sh.local.h, then code will be included in csh so that the user can be notified as other users log in. The code is controlled by the special shell variable watch login. It should contain a list of user and terminal pairs to specify which login or logout events the shell should notice and print to the terminal. The user may specify “any” for a user or terminal as a wild card. If the first entry in watch login is a number, it is interpreted as the minimum number of minutes which the shell should wait before checking if anybody has logged in or out; by default this value is ten minutes.

6 Outstanding issues and bugs

The following enhancements are put forward as possible extensions to csh, pending discussion and staff comment.

6.1 Relative pathnames and command completion

Currently, csh does not handle relative pathnames in its search paths gracefully when doing command completion. Since in this case which directories should be searched is dependent on the current directory, which changes often, the programs in these directories cannot be easily cached. However, since relative pathnames (with the exception of the current directory) are rarely used in csh’s search paths, as a “quick fix”, csh always searches the current directory when doing command completion. This was originally considered a temporary workaround, but release deadlines dictated otherwise. However, as first approximation of what users want the shell do to, it has worked very well, as no one has complained.

6.2 Default binding of control-T

Currently, control-T in the base keymap is bound to the line editor function twiddle chars. Unfortunately, this is not compatible with the binding of control-T within Gnuemacs. Gnuemacs’s transpose-chars function, which is bound to control-T, transposes the characters before and after the cursor, whilst the line editor’s twiddle chars transposes the two previous characters before the cursor.

Many of csh’s users are used to this misfeature, and changing it would break compatibility with previous version of csh. In addition, the line editor function emacs-twiddle, which is completely compatible with Gnuemacs’s transpose-chars, is provided, and it is a simple matter to place in one’s .tcshrc file the line:

set_keymap base ^T emacs-twiddle

However, there has been precedence in breaking compatibility with previous version of csh for the purpose of gaining compatibility with Gnuemacs. For example, control-K used to be bound to reverse search; when the new line editor function delete eol was introduced, people immediately demanded that the default binding of control-K be changed.

6.3 Control-characters in setting keyboard bindings

Currently, to set the binding of a control character, it is necessary to insert the actual control character into the command. This is bad. This means that there are generally invisible control characters in users’ .tcshrc files, a concept which is confusing at best to the novice user. The shell should recognize the two character sequence of a caret (“ˆ ”) and a letter to be its corresponding control character. This was not implemented last summer due to time and release pressures.

6.4 Portability to different operating systems

Work is underway to port the Athena csh to Ultrix and SunOS. The latter should be extremely useful in exterminating null pointer dereferencing bugs. When these ports are done, the changes (with appropriate conditionals, as necessary) should be incorporated into Athena’s csh.

The Athena csh has been ported to the Apollo operating system^{3} by Bill Sommerfeld. Due to hack in the Apollo kernel that specifically allows null pointer dereferencing, the port was fairly straight forward. This port should also be incorporated into csh, with the appropriate conditionals.

6.5 Errors in logout scripts

In the both the Berkeley and Athena version of csh, if there is an error in a builtin command (such as an attempt to kill an nonexistent process) in the .logout script and the logout was initiated by the “logout” command (as opposed to control-D), the logout is aborted. This can be dangerous, if the user assumes that a logout cannot be aborted, and walks away from the terminal or workstation.

6.6 Killing process groups

Although this is not explicitly defined in the manual page of csh, both the Athena and Berkeley versions of the csh attempt to kill the process group of the job, instead of just the process, when the argument to the kill command is specified by a percent sign (“%”). If the process changes its process group, then when the user tries to kill the job (via “kill %1”), csh reports the error message “No such process,” which is confusing, since the process does exist. A better procedure would be if the attempt to kill the process group fails, the shell should attempt to kill all of the process associated with the job.

6.7 RT compatibility

At present, the IBM RT version of csh is being compiled with the 4.2 compatibilty code turned. Since BSD 4.3 is finally running on the RT’s, the compatibility code should be turned off.

3Should put official name of Apollo OS here

7 Suggested enhancements

The following enhancements have been suggested by various staff members and users of csh. Some of these suggestions may not be appropriate modifications to csh, but all have been listed here for discussion and staff comment.

7.1 Signal driven activity

Upon receipt on of a SIGUSR1 or SIGUSR2 signal, have csh execute a command (most likely a special alias: cf. precmd). Since the shell is not generally very interruptible, the command would probably executed only at certain well defined times — such as before printing the prompt. This could be useful as a way to signal the shell to execute a rehash command, for example. (Suggested by Jay Berkenbilt)

7.2 Sourcing a file at startup

Define a new option to csh, the “-S” option. It would cause the shell to source the file specified after the option after executing the standard .tcshrc and .cshrc scripts. This would allow an interactive shell to appear with shell variables and aliases automatically set without requiring that the users’ dot files be modified. (Suggested by Ademola Aderibigbe and Ted Ts’o)

7.3 Improving history expansion

Currently, if the line editor is enabled, history references (those starting with an exclamation) are expanded when a space is typed. Unfortunately, the line editor does not understand the full syntax of history expressions. At some point, this probably should be expanded. (Lack of feature reported as bug by Dan Geer and others.)

7.4 Security hacks and dot files

Currently (unless the csh is compiled with the INGRES option, which it normally is not), if the user’s dot files aren’t owned by the user and the group ownership of the files doesn’t match the users’ primary group id, csh will refuse to source the files, as a security hack. Since in the Athena environment group access does not mean much, we might want to disable this feature.

8 Differences in source files

New source files present in Athena csh:

• pwprintf.c (replaces printf.c and doprnt.c)

• nmalloc.c (replaces alloc.c)

• sh.machmhk.c (check to see if running a byte-swapped binary)

• sh.resize.c (handle window resizing)

• sh.sig.c (4.2 compatibility library)

• tcsh.c

• tcsh.h

• tcsh.keymap.c

• tcsh.keytab.c

• tcsh.ledit.c

• tcsh.lsupp.c

• tcsh.nfunc.c

• tw.h

• tw.help.c

• tw.init.c

• tw.parse.c

• tw.spell.c

Sources files no longer used by Athena csh:

• alloc.c

• doprnt.c

• printf.c