\documentstyle{article}

\oddsidemargin .25in
\evensidemargin 0in
\topmargin 0in
\headheight 0in			% no header
\headsep \headheight
\textwidth 6.25in
\textheight 8.5in
\footskip .4in
\parskip .1in

\author{Theodore Y. Ts'o}
\date{July 15, 1988}
\title{Differences between BSD 4.3 csh and the Athena csh \\
Draft Version 1.0A}

\begin{document}
\maketitle
\tableofcontents
\newpage

\section{Introduction}

\subsection{History, or the geneology of tcsh}

The version of csh that is currently running at MIT Project Athena had
its beginnings in an alternative shell called tcsh, which was supported
by the Student Information Processing Board (SIPB).  Originally, tcsh
was a set of changes to the standard Berkeley 4.2 csh written by Ken
Greer at HP in 1983 adding Tenex-style file name completion to csh ---
hence the name ``tcsh''.  These modifications were the basis for at
least three different tcshs which have been diverging and cross-breeding
since then.

The first of these versions had its beginnings in modifications made at
MIT by the Student Information Processing Board (and perhaps at S1?).  A
line editor and the ``where'' builtin were added by Earl Cohen, Bill
Saphir, and Dan Morgan.  This version of tcsh was supported by SIPB and
was very popular; many users choose tcsh as their login shell.  In the
summer of 1987,\footnote{Date and release when this happened is not
certain} Project Athena adopted it as the default shell, replacing
/bin/csh. 

Another version of tcsh became incorporated into the BSD 4.3 sources,
giving the BSD 4.3 csh file 
completion which could be accessed by hitting the escape and
control-D keys.  When the BSD 4.3 was released, the tcsh that was
supported by SIPB was merged with the BSD 4.3 csh.  This can still be
seen today by the partial duplication of some of the file completion
code.  (done in the name of providing as much compatibility
with the BSD 4.3 csh as possible when line editing was turned off)

Yet another version of tcsh was wrtting by Paul Placeway
at Ohio State, who added a different line editor to tcsh, as well as
some additional features such as automatically logging out a user if was
idle for more than a specified time.  This was published in
net.sources in late 1984.  (It is not clear if any features from the 1984
posting were imported into the tcsh at Athena, but it appears to be very
likely.)  Another, more featureful version was published in
comp.sources.unix in the summer of 1987.   Both versions were based on
the BSD 4.2 csh.   

Many new features from the Ohio State tcsh were incorporated into the
Project Athena tcsh in the fall of 1987.  These included
an improved prompt format (section \ref{prompt-format}), execution of a
command prior to each prompt (section \ref{precmd}), new ways of
accessing entries in the directory stack (sections \ref{dextract} and
\ref{stack-hack}), expanded file globbing (section \ref{globbing}), and
several new shell variables (section \ref{shell-variables}). 


In addition, at that time I added several features of my own.  Those
included dynamic keymaps for the line editor (section \ref{lineedit}),
optional warning messages for processes that exit with a non-zero exit
status (section \ref{nonzero}), optional sourcing of a shutdown script
when a non-login shell terminates (section \ref{exitscript}), 
optional automatic TERMCAP updating (section \ref{termcap}), printing
the directory of a process when it terminates (section \ref{jobdir}),
and detecting the attempted execution of byte-swapped binaries (section
\ref{machtype}).   Several bugs were also fixed as documented in the
Bugs section of this document.

\subsection{Structure of this document}

Differences between the C shell (hereafter known as csh) as shipped in
the BSD 4.3 release tapes and the csh as currently being run by Athena
can be roughly be divided into several areas:

\begin{enumerate}
\item Internal code changes having no effect on the user interface
\item Changes to the user interface
\item Bug fixes
\item Optional features not currently enabled
\end{enumerate}

\newpage
\section{Internal code changes}

These differences were usually made for compatibility or performance
reasons; in case of a new release from Berkeley, it would not be
critical (although it might be nice) for these changes to be integrated
in to a new version of csh.

\subsection{BSD 4.2 compatibility}
The code is conditionalized so that it may be run on BSD 4.2
systems.  It is not known if csh can still be compiled without any
changes on BSD 4.2 systems.

\subsection{New version of printf}
The minimal printf used by csh is written in C, instead of the VAX
assembly language version provided by Berkeley.  (This change we
probably want to keep for those who prefer to use RT's.)

\subsection{Code cleanup}
The code was cleaned up with help from suggestions made by lint and
saber.  The changes were mostly casts and function and variable
declarations.  

\newpage
\section{Additions to the user interface}

Changes to the user interface were designed to be as transparent as
possibly, to retain backwards compatibility with the BSD csh.  In many
cases, additional features must be enabled by setting certain shell
variables or certain aliases.  In other cases, the names of the new
builtin commands were chosen so that would not likely conflict with an
executable program which a user might want to execute. 

\subsection{A new configuration file}
\label{tcshrc}

When the Athena csh is started, it first sources the file \~{}/.tcshrc
(if it exists), before it sources the .cshrc file.  This allows user to
place commands which are specific to the Athena csh in the .tcshrc file,
so that the .cshrc may remain compatible with the Berkeley version of
the shell. 

\subsection{The tcsh\_version builtin}
\label{tcsh-version}

The {\em tcsh\_version}\/ is a new builtin command.  It prints a
version string which allows the user to determine which version csh he
or she is running.  This is most useful when trying to debug a problem
which is due to version skew or due to the user running an unauthorized
version of the shell.

\subsection{The where builtin}

\label{where}
The ``where'' builtin was added to the Athena csh.  It allows the user
to see where or what the csh would execute if the user executed a
particular command.  It searches through the list of aliases, shell
builtins, and executable files in the user's path.  If possible, it will
take advantage of csh's hashing scheme.

\subsection{Line editing}
\label{lineedit}

If the shell variable {\em lineedit}\/ is turned on,  an emacs-like line
editor takes the place of the character-, word-, and line-erase
features of the tty driver.  The line editor has most of the simple
editing commands available in Emacs to move around and change the
current line.  In addition, the line editor may be used to step
forwards and backwards through the shell's history list.

The line editor works on any CRT terminal which meets the following 
requirements: 

\begin{itemize}
\item Sending a backspace character (or some other character defined in
the ``le'' entry in the terminal's termcap) to the terminal must move
the cursor one column to the left without erasing the character in that
column.

\item sending a space character to the  terminal must replace the
character in the current column with a blank space.
\end{itemize}

The line editor is based on the concept of {\em keymaps}\/ which is a
table of line editor functions and their arguments (if the function
takes an argument), with an entry for every ascii character.

Initially, the key strokes are interpreted in the context of the {\em
starting keymap}.  When a key is pressed, the line editor looks up the
function associated with that key in the {\em current keymap}, resets the
{\em current keymap} to the {\em starting keymap}, and executes the line
editor function in the keymap.  The special line editor function {\em
prefix} changes the {\em current keymap}\/ to the keymap specified in its
argument.  In this way, the next key pressed will be interpreted in
another keymap.   

For example, by default, the {\em starting keymap}\/ is the pre-defined
keymap {\em base}, which has all of the printing characters bound to the
line editor function {\em self-insert}\/ and the control characters to
line editor functions which emulate their equivalent emacs functions.
In addition the escape key is bound to {\em prefix}\/ with an argument of
{\em meta}.  The {\em meta}\/ keymap contains the mappings of the emacs
commands whose first character is an escape --- the meta-commands.

\subsection{Warning messages for non-zero exit status}
\label{nonzero}

If the shell variable {\em print\_exit\_status} is set, then csh will
print a warning message if an executed command has a non-zero exit
status.  This is useful for programs which signal errors only by
non-zero exit status returns\footnote{Such as the 5.5 version of attach}.
It is also useful for detecting programs which do not clear their exit
status when they terminate.

\subsection{Automatic execution of a command before each prompt}
\label{precmd}

If the alias {\em precmd}\/ is defined, then the shell will 
execute that alias before each prompt is printed.  This allows users
who wish to customize their prompts every time they execute a
commands the ability to do so.

\subsection{Expanded prompt formatting}
\label{prompt-format}

The format for the {\em prompt}\/ shell variable has been changed to
include many new things, such as the current time of day, current
working directory, etc..  The new format uses ``\%$<$char$>$'' to signal
an expansion, much like printf.  The available sequences are:

\begin{center}
\begin{tabular}{|l|l|}
\hline
Format command  & Description \\
\hline
\%d 	&	Current working directory. \\
\%c 	&	Trailing component of cwd. \\
\%h	or (\%!)	 & Current history event number \\
\%M  & The full machine hostname \\
\%m	 &	The hostname up to the first ``.'' \\
\%S	 & Start (stop) standout mode \\
\%s	 &	Stop standout mode \\
\%t  &		Current time-of-day, in 12 hour, am/pm format. \\
\%\%	 &	A single \%. \\
\hline
\end{tabular}
\end{center}

If some other character follows the percent sign, both the percent
sign and the unknown character will be displayed.  Also, for backwards
compatibility, then a backslashed exclamation point will be replaced
with the current history event number.

The shell variable {\em prompt2}\/ controls the prompt printed while the
shell is prompting for the body of for and while loops, and when the
shell is ignoring commands due to an if statement.

\subsection{A termination script for non-login shells}
\label{exitscript}

If the shell variable {\em exitscript}\/ is set, when a non-login shell
exits, it will try to source the file named by {\em exitscript},
analogous to the .logout file.  This file will typically include
commands to clean up the environment, such as destroying root kerberos
tickets.  Note that since a shell variable must be set for this
processing to occur, its use will not interfere with shell scripts.

\subsection{Limiting the number of completions to be listed}
\label{maxcompletions}

If the shell variable {\em maxcompletions}\/ is set, then if there are
more possible completions than the number specified by the shell
variable, instead of listing the possible completions, csh will just
print the number of possible completions.  Thus the user will not be
inundated by hundreds of completions.

\subsection{Time-stamped history list}
\label{history}

The ``history'' builtin accepts a new option, ``-t''.  If this option
is specified, a time-stamp indicating when each command was executed is
printed along with each command.  This allows the user to keep track of
when the various events occurred.  The time stamps are not maintained on
the saved history list; thus, on logging back in, all the saved history
events will be recorded with the login time as their time stamp.  This
was done since the format of the .history needed to be preserved for
compatibility concerns.  

\subsection{Alternative semantics for ``pushd +n''}
\label{dextract}

If the shell variable {\em dextract}\/ is set, then the command ``pushd
+{\em n}'' extracts the {\em n}th directory from the directory stack and
places it on the top of the stack (thereby changing to current working
directory to it), instead of rotating the stack so that the {\em n}th
entry is on the top.  Some users find this behavior more useful.

\subsection{Accessing entries in the directory stack}
\label{stack-hack}

If the shell variable {\em stack\_hack}\/ is set, the ``='' character 
becomes a magic character like ``*'' or ``?'' during file globbing.  Any
instance of ``={\em n}'' becomes replaced by the {\em n}th entry on the
directory stack.  This is highly useful; however, it conflicts with the
common usage of using the equals sign in X geometry specifications ---
if stack\_hack is enabled, then the equals sign must be backslashed to
avoid this expansion.

\subsection{Extension to file globbing}
\label{globbing}

The syntax for any character in a range (for example ``.[a-z]*'') has
been extended so as to conform with standard Unix regular expression
syntax.  Specifically, after an open bracket (``[''), if the first
character is a caret (``\^{}'') then the character matched will be any
character {\em not}\/ in the range specified.

\subsection{Byte-swapped binaries}
\label{machtype}

Before csh attempts to run a program, it checks to see if the magic
number for the executable is byte-swapped.  If so, the binary was
probably a Vax binary that the user attempted to run on a RT, or vice
versa.  Instead of attempting to feed the erroneous binary to /bin/sh
causing many confusing error messages, csh prints an error message.

\subsection{Automatic TERMCAP handling}
\label{termcap}

If the environment variable TERMCAP is set and it does not contain a
filename to a termcap file, csh will try to automatically update TERMCAP
in the following instances:

\begin{itemize}
\item  If the user sets the shell variable {\em term}\/ or the
environment variable TERM, TERMCAP will be updated to reflect the new
terminal type. 

\item  In addition, if the shell receives a SIGWINCH signal, which means
that the window which the shell is running in has changed size, TERMCAP
will be updated to reflect the new size of the window.
\end{itemize}

\subsection{Printing the directory of jobs}
\label{jobdir}

When a child process of csh changes state (by being stopped, killed, or
terminated), the shell will print a message describing the changed state
of the process to the user.  In the Athena csh, if 
the current directory of the shell does not match the current working
directory of the process when it was started, it will also print the
current working directory of the process when it was started.  This is
useful when a user has three or four jobs with the same name, but which
were started in different directories (like ``make'').  This features
allows the user to distinguish between these jobs.

\subsection{New shell variables}
\label{shell-variables}

The following shell variables are automatically set by the Athena csh
when it start up:

\begin{center}
\begin{tabular}{|c|l|}
\hline
Variable & Description \\
\hline
uid & Contains the user id number of the current user \\
host & Contains the machine name of the current host \\
hosttype & Contains the type of the computer (``vax'' or ``rt'')  \\
\hline
\end{tabular}
\end{center}

\subsection{File suffix of symbolic links}
\label{file-suffix}

During file completion, when a list of possible file completions is
being printed, symbolic links which do not point to a valid file or
directory are printed with a suffix of ``?'' instead of ``@''.  (Like
the BSD csh, if the symbolic link points to a directory, the symbolic
link is printed with a suffix of ``$>$''.)

\newpage
\section{Bug fixes}

These are bug fixes which have been made locally.  Most have not
be reported to Berkeley.

\subsection{Incorrect error message with exit}

The BSD csh complains that there are stopped jobs when the
``exit'' command is used to terminate the processing of a source'd
file, although the exit takes place anyway, as it should.
This has been fixed.  (Not reported to Berkeley)

\subsection{Error recovery of getwd() during shell startup}

In the BSD 4.3 csh, if the shell cannot access the current working 
directory, it prints its error message to a closed file descriptor (so
no error message would be output) and exits.  This behavior was
extremely mystifying to the user.  The Athena csh prints the
error message to standard output and in addition it prints a message
reminding the user of the most common reason why this error message
would happen: ``Did you su in a fascist directory?''  (Not reported to
Berkeley)  

\subsection{Logging out in non-login shell}

If the user uses the logout builtin command in a non-login shell, and
there are stopped processes, the user will receive the error message
``There are stopped jobs.''  This may be factually accurate, but it is
not the correct error message, which should be ``Not login shell.''
(Not reported to Berkeley.) 

\subsection{Freeing strings and then using them}

In many sections of the code, BSD csh would free strings and then use
them in calls to the subroutine {\em Perror}, since that subroutine
would not return control to the calling routine.  While it works, this
is dependent on the implementation of malloc and is generally not
recommended.  Athena csh uses a new subroutine, {\em PerrorF}, which
uses the passed argument to print the error message and then frees it.

\subsection{Erroneously reported stopped jobs}

When a child processing belonging to csh exits, csh marks its associated
process information structure to be freed by setting its pid to zero.
In certain situations, it is possible for csh to look at this structure
before it gets freed when csh is determining if there are any stopped
jobs or not.  This code was changed so that it checks if the structure
contains valid information (by checking if the pid is nonzero).  (Not
reported to Berkeley)

\newpage
\section{Optional features not currently enabled}

\subsection{Automatic logout}
If {\sc AUTOLOGOUT} is defined in sh.local.h, then code will be included
in csh such that if the shell variable {\em autologout}\/ is set, then
after the number seconds of inactivity specified by the value of {\em
autologout}, csh will automatically logout the user.  This feature has
been disabled, since many users still have {\em autologout} set from the
days of time-sharing, and this features is extremely dangerous in a
window environment.

\subsection{Login monitoring}
If {\sc WATCHLOGINS} is defined in sh.local.h, then code will be
included in csh so that the user can be notified as other users log in.
The code is controlled by the special shell variable {\em watch\_login}.
It should contain a list of user and terminal pairs to specify which
login or logout events the shell should notice and print to the
terminal.  The user may specify ``any'' for a user or terminal as a wild
card.  If the first entry in {\em watch\_login} is a number, it is
interpreted as the minimum number of minutes which the shell should wait
before checking if anybody has logged in or out; by default this value
is ten minutes.

\newpage
\section{Outstanding issues and bugs}

The following enhancements are put forward as possible extensions to
csh, pending discussion and staff comment.  

\subsection{Relative pathnames and command completion}

Currently, csh does not handle relative pathnames in its search paths
gracefully when doing command completion.  Since in this case which
directories should be searched is dependent on the current directory,
which changes often, the programs in these directories cannot be easily
cached.  However, since relative pathnames (with the exception of the
current directory) are rarely used in csh's search paths, as a ``quick
fix'', csh always searches the current directory when doing command
completion. This was originally considered a temporary workaround, but
release deadlines dictated otherwise.  However, as first approximation
of what users want the shell do to, it has worked very well, as no one
has complained. 

\subsection{Default binding of control-T}

Currently, control-T in the base keymap is bound to the line editor
function {\em twiddle\_chars}.  Unfortunately, this is not compatible
with the binding of control-T within Gnuemacs.  Gnuemacs's
{\em transpose-chars}\/ function, which is bound to control-T, transposes
the characters before and after the cursor, whilst the line editor's
{\em twiddle\_chars}\/ transposes the two previous characters before the
cursor.

Many of csh's users are used to this misfeature, and changing it would
break compatibility with previous version of csh.  In addition, the line
editor function {\em emacs-twiddle}, which is completely compatible with
Gnuemacs's {\em transpose-chars}, is provided, and it is a simple matter
to place in one's .tcshrc file the line:
\begin{verbatim}
set_keymap base ^T emacs-twiddle
\end{verbatim}

However, there has been precedence in breaking compatibility with
previous version of csh for the purpose of gaining compatibility with
Gnuemacs.  For example, control-K used to be bound to {\em
reverse\_search}; when the new line editor function {\em delete\_eol}\/
was introduced, people immediately demanded that the default binding of
control-K be changed.

\subsection{Control-characters in setting keyboard bindings}
Currently, to set the binding of a control character, it is necessary to
insert the actual control character into the command.  This is bad.
This means that there are generally invisible control characters in
users' .tcshrc files, a concept which is confusing at best to the novice
user.   The shell should recognize the two character sequence of a caret
(``\^{}'')  and a letter to be its corresponding control character.
This was not implemented last summer due to time and release pressures.

\subsection{Portability to different operating systems}

Work is underway to port the Athena csh to Ultrix and SunOS.  The
latter should be extremely useful in exterminating null pointer
dereferencing bugs.  When these ports are done, the changes (with
appropriate conditionals, as necessary) should be incorporated into
Athena's csh.

The Athena csh has been ported to the Apollo operating
system\footnote{Should put official name of Apollo OS here} by Bill
Sommerfeld.  Due to hack in the Apollo kernel that specifically allows
null pointer dereferencing, the port was fairly straight forward.  This
port should also be incorporated into csh, with the appropriate
conditionals. 

\subsection{Errors in logout scripts}

In the both the Berkeley and Athena version of csh, if there is an error
in a {\bf builtin} command (such as an attempt to kill an nonexistent
process) in the .logout script and the logout was initiated by the
``logout'' command (as opposed to control-D), the logout is aborted.
This can be dangerous, if the user assumes that a logout cannot be
aborted, and walks away from the terminal or workstation.  

\subsection{Killing process groups}
 
Although this is not explicitly defined in the manual page of csh, both
the Athena and Berkeley versions of the csh attempt to kill the {\em
process group} of the job, instead of just the {\em process}, when the
argument to the kill command is specified by a percent sign
(``\%'').  If the process changes its process group, then when the user
tries to kill the job (via ``kill \%1''), csh reports the error message
``No such process,'' which is confusing, since the process does exist.
A better procedure would be if the attempt to kill the process group
fails, the shell should attempt to kill all of the process associated
with the job.  

\subsection{RT compatibility}

At present, the IBM RT version of csh is being compiled with the 4.2
compatibilty code turned.  Since BSD 4.3 is finally running on the RT's,
the compatibility code should be turned off.

\newpage
\section{Suggested enhancements}

The following enhancements have been suggested by various staff members
and users of csh.  Some of these suggestions may not be appropriate
modifications to csh, but all have been listed here for discussion and
staff comment.

\subsection{Signal driven activity}

Upon receipt on of a SIGUSR1 or SIGUSR2 signal, have csh execute a
command (most likely a special alias: cf. precmd).  Since the shell is
not generally very interruptible, the command would probably executed
only at certain well defined times --- such as before printing the prompt.
This could be useful as a way to signal the shell to execute a
rehash command, for example.  (Suggested by Jay Berkenbilt)

\subsection{Sourcing a file at startup}

Define a new option to csh, the ``-S'' option.  It would cause the shell
to source the file specified after the option after executing the
standard .tcshrc and .cshrc scripts.  This would allow an interactive
shell to appear with shell variables and aliases automatically set
without requiring that the users' dot files be modified.  (Suggested by
Ademola Aderibigbe and Ted Ts'o)

\subsection{Improving history expansion}

Currently, if the line editor is enabled, history references (those
starting with an exclamation) are expanded when a space is typed.
Unfortunately, the line editor does not understand the full syntax of
history expressions.  At some point, this probably should be expanded.
(Lack of feature reported as bug by Dan Geer and others.)

\subsection{Security hacks and dot files}

Currently (unless the csh is compiled with the INGRES option, which it
normally is not), if the user's dot files aren't owned by the user and
the group ownership of the files doesn't match the users' primary group
id, csh will refuse to source the files, as a security hack.  Since in
the Athena environment group access does not mean much, we might want to
disable this feature.

\newpage
\section{Differences in source files}

New source files present in Athena csh:
\begin{itemize}
\item pwprintf.c   (replaces printf.c and doprnt.c)
\item nmalloc.c	   (replaces alloc.c)
\item sh.machmhk.c (check to see if running a byte-swapped binary)
\item sh.resize.c (handle window resizing)
\item sh.sig.c (4.2 compatibility library)
\item tcsh.c
\item tcsh.h
\item tcsh.keymap.c
\item tcsh.keytab.c
\item tcsh.ledit.c
\item tcsh.lsupp.c
\item tcsh.nfunc.c
\item tw.h
\item tw.help.c
\item tw.init.c
\item tw.parse.c
\item tw.spell.c
\end{itemize}

Sources files no longer used by Athena csh:
\begin{itemize}
\item alloc.c
\item doprnt.c
\item printf.c
\end{itemize}

\end{document}


