Layered Athena Design Mark Rosenstein May 21, 1993 This document describes the overall design of Layered Athena_how we expect to implement it in the current Athena environment. The document does not stand alone. For a summary of the scope and requirements for the project, read Athena Layering Requirements. This document also assumes the reader is somewhat familiar with the current (1993) Athena environment, particularly the services and delivery mechanisms. The design discussed in this document is intended only for Unix platforms. While the terminology and high-level concepts will also apply to personal computers, this level of design is very much Unix specific. We will address PCs and Macs at a later date. 1 Architecture Overall, Layered Athena will look a lot like mkserv in terms of how it works. Layered Athena will be distributed as a filesystem very similar to a regular Athena system pack, with an added directory of configuration files and programs to implement Layered Athena. There will be a simple command line based client which does all of the work, handling installations, removals, and updates. Both a graphical user interface and an friendly ASCII interface will be written as a front-ends for the simple client, but will not duplicate the actual guts of the code. There are two different kinds of things to be done: set options and manipulate subsets. The options affect how and when subsets are loaded, immediate configuration changes of the workstation, and the eventual functioning of the subsets once they are running. The options are: o Handling of updates_whether to auto-update when a newer version is available, or just to send an email notification that an update is needed o Who to notify about system maintenance, such as the availability of new versions o When the last message requesting an update was sent to the workstation owner mentioned above. This is necessary so that they do not get messages once an hour. o If the owner has acknowledged notification of an update request, and does not want to receive further notices about this particular version (they will start again anyway when the next release is available). o Whether to allow monitoring of the workstation configuration via SNMP o Where an additional copy of the configuration information is stored. A local file is always maintained, but a remote copy may also be kept up-to-date. o If this workstation is centrally registered_should I/S be informed in an automated manner about configuration changes of this workstation? o Local customization scripts: the workstation may have customizations to the standard Athena software For each subset, several options are recorded: o Subset status: ignored, use it remotely, copy it local o Version: what version is currently loaded These settings are stored in a configuration file on the local disk. This is a flat ascii file with a format similar to that described below for subset configuration. This file will be placed in a "Layered Athena directory", such as /var/athena although it will be different for different vendor operating systems. Any other state that must be remembered is also kept in files in this directory. The graphical client has dialogs to set each of these options. The standard client implements them by registering and storing the configuration if requested, copying any necessary files to the local disk with synctree, then running scripts to finish the installation and/or customize the subsets. Eventually mkserv options should be supported and run during an update as well, but this will not be attempted for the first phase. Mkserv must fail gracefully if attempted on a Layered Athena workstation. Some subsets and configurations require boot-time support. This is handled by an rc.athena script. This script handles the initial activation of the workstation. A reactivate script also needs to be run occasionally. This will be done hourly through cron, and the script will check to see if anyone is logged in before doing anything disruptive. Reactivation of a private workstation consists of checking for any needed updates, changing the attached system pack if the hesiod information changes, updating AFS cell configuration, detaching lockers attached by users no longer logged in, and resetting zephyr locations for logged out users. If the workstation is running the Athena login program, then password and group file cleanup must be performed as well. 1.1 Communication Layered Athena makes information available to the workstation owner, to Athena operations personnel, and to anyone polling the workstation. Much of the communication is email based. This is a potential problem because email may not work on the workstation, either because the vendor or system manager did not configure it properly for the MIT environment. When this system sends mail, it will invoke sendmail with an alternate configuration file that is likely to get the mail to the mailhub under most circumstances. Messages are generated to the workstation owner when new software versions are available. The "owner" is identified when Layered Athena is installed, and may be an individual or a mailing list. If the workstation is allowed to auto-update, it will do so, and just send a single piece of email confirming the update. Otherwise, it will send a piece of email requesting the manual update. This email will not be from root, but will instead use a special return address to collect info about bounced notices. The message can contain some release notes extracted from the release. If the workstation does not get updated, it will repeat this message once a week until it is updated. The owner may "acknowledge" the message and quit receiving reminders until the next version is available, even if the workstation does not get updated. Unless the workstation owner specifically requests anonymity, Layered Athena will also send a message to Athena operations when it is first installed, and when it updates or the configuration is changed. This will be in the form of an email message containing the all of the Layered Athena configuration information for the workstation. The message will be sent to an address which archives these into a database. This database will allow us to help the workstation owner with problems, and give us an idea of how people are using Layered Athena in the field. Finally, the workstation configuration is available via SNMP to anyone who queries for it. We will add several new variables to the Athena MIB to report this. Access to these variables may be turned off in the configuration for those workstation owners who do not want to allow others to find out what software is on their workstation. When the SNMP agent scans the Layered Athena configuration file, it will check that it is OK to report as well as getting the configuration. 2 Subsets There will only be a small number of subsets. However, quite a bit of information needs to be available about each subset: o What other subsets it depends upon o The disk space required to install it o What files are in it o Whether it is installable on the local disk, usable from a fileserver, usable without any installation or privileges, or any combination of these o What special actions are necessary to install it - So that it can be installed on the local disk - So that it can be used from a fileserver - So that it can be used once without any installation - If a reboot is necessary after installation or update o What special actions are necessary to remove it o What is the version of this subset Most of this information goes in a simple configuration file. This configuration file is flat ASCII, with the following format: % Lines beginning with percent are comments % (blank lines are ignored) % Entries that start with a keyword, followed by a colon. The % remainder of the line is the value for that keyword. Multi-line % values are handled by starting successive lines with whitespace. % The long name of the subset, for display purposes subset: Subset Name % The approximate size in Kbytes if copied to the local disk localsize: 1500 % The approximate size in Kbytes required on the local disk if the % subset is used remotely remotesize: 50 % Any subsets that must be present before this subset can be used. % They are in a comma separated list. dependencies: kerberos, basic % Where it can be installed locations: local, fileserver, transient % A version number for the subset. version: 4 Subset version numbers are integers which increase each time the subset changes. Somewhere else we will keep a mapping of which subset versions are in a particular syspack release. What's missing from this configuration file is the synctree configuration and the install and remove shell scripts. These go in separate files. 2.1 Packaging There are two paths we could follow for the packaging of the subsets. We could keep them in a filesystem such as the current system packs, or we could store them in an archive such as tar or dump format. Using an archive format is more efficient in disk space and time to read all of the contents back for an installation. On the other hand, we need a filesystem hierarchy anyway to allow people remote access without copying everything local. If this filesystem hierarchy could be an actual Athena system pack, then it requires no or little additional disk space to make a release also available as Layered Athena. One way then to identify the files making up a subset is to use a subscription list or other similar mechanism to describe portions of a larger hierarchy. We will use synctree for this because of the flexibly of its rules files. If we find that we need something more efficient and/or network friendly, synctree can be enhanced to use statfiles full of checksums similar to the way track does now. We recommend synctree over track even without this because it is more flexible and easier to port. There are two different actions that synctree implements here. Subsets can be copied local, or made available locally while remaining on the fileserver. Rather than require two different synctree rules files, we will only use one but it gets run through the m4 macro pre-processor before it is used. We use m4 rather than cpp because cpp behaves too differently on different platforms, particularly with newer C compilers where it does tokenizing. The name of each subset being used will be defined, as will an operation variable which is either "link" or "copy" specifying whether the subset is to be copied local or symlinked in. The rules files for each subset in use will be concatenated together and fed to synctree to actually do the copy. So a subset will consist of a system pack and several files: the configuration file, a synctree rules file, an add script, and a remove script. 2.2 Contents Here we examine the actual subsets in more detail. Note that several of these subsets include additions to the /etc/services file. In the event that there is already an incorrect entry for a service, we will overwrite the bad information. Many of the subsets depend upon other subsets. Figure 1 shows these module interdependencies. 2.2.1 Kerberos This subset includes the necessary support for Kerberos clients. It does not depend upon any other subsets. It provides the following end-user applications: kinit, klist, kpasswd, rkinit, ksu, and the utility programs ksrvtgt and ksrvutil. These files are copied or symlinked into /usr/athena/bin. The files /etc/athena/krb.conf and /etc/athena/krb.realms will always be copied local. The file /etc/services will be edited to contain entries for klogin, kshell, erlogin, kerberos, kerberos_master, and passwd_server if they are not already present. 2.2.2 AFS This subset adds AFS client filesystem support to the workstation. It requires the Kerberos subset. The largest user visible addition will be the /afs/... hierarchy will be visible. The applications aklog, unlog, fs, pts, and vos will also be added. Disk space will be allocated for the cache, the system startup files will be modified, and it may be necessary to install a new kernel as well. The implementation of this subset will vary from platform to platform. 2.2.3 Basic Athena Services This subset requires that Kerberos be installed as well. It includes support for hesiod, zephyr, moira, online consulting, global message of the day, kpop email, printing, and attach. This includes the following end-user applications: zwgc, zwrite, zctl, moira, blanche, chpobox, mailmaint, listmaint, Figure 1: Subset dependencies olc, get_message, from, inc, lpr, lpq, lpquota, attach, detach, and these utility programs hesinfo, zhm, zinit, zlocate, zstat, fsid, and movemail. These files are copied or symlinked into /usr/athena/bin. The file /etc/services will be edited to contain entries for zephyr-srv, zephyr-clt, zephyr-hm, sms_db, sms_update, sms_ureg, globalmessage, kpop, lpallow, lpquery, and lplog if they are not already present. It requires the following configuration files to be on the local disk: /etc/resolv.conf, /etc/named.boot, /etc/named.mit, and /etc/athena/attach.conf. Installing this subset will interact with nameservice on the workstation, since hesiod is implemented with the named. The rest of the subset will not be a problem. The printing utilities will fallback to using the local /etc/printcap if the specified printer is not in hesiod, so it will work with a local printer. To ease use of these services when the login subset is not loaded, we will include a login_athena script which will prompt the user for their kerberos password and get them tickets, start zephyr, attach their locker, and any other necessary setup. Once Basic Athena Services are installed, users on this workstation will have access to Athena lockers and third party software. This access does not mean that they have licensed or paid for the use of this software. When the policy issues are decided, it may become necessary to display and get positive feedback about licensing when this subset is installed. 2.2.4 X The Athena X subset will include the standard X11 clients as we compile them. This subset does not depend upon any other subsets. It may duplicate some of what the vendor provides. We will install our clients in /usr/athena/bin rather than /usr/bin/X11 to attempt to avoid conflicts. On some platforms, the X subset will include a server as well. On such platforms, the appropriate server for the hardware will be installed in /etc and symlinked from its usual name to /etc/X. 2.2.5 Login This subset runs the Athena login program on the workstation. This means that anyone with an active Athena account can login to the workstation unless access is specifically restricted. It requires the Athena Basic Services and X subsets. Besides the Athena login screen, the end user will see the applications chsh, chfn, show_console, hide_console, config_console, session_gate, logout, and /bin/athena/tcsh. This subset also includes rlogind and telnetd for network logins. Other necessary local files include timeout, dm, xlogin, console, and a number of files in /etc/athena/login. Installing this subset will also cause /etc/ttys or /etc/inittab to be edited to start up the login program. 2.2.6 Emacs This subset includes emacs and its libraries and utilities. It does not depend upon any other subsets. 2.2.7 Man This subset includes the man pages for all Athena programs and on some platforms, a modified man program as well. No configuration files are required. 2.2.8 Tex This subset provides the TEX and LaTEX formatters and associated utilities. This subset does not depend upon any other subsets. It includes the applications tex, latex, slitex, virtex, dvi2ps, xdvi, dvitype, and bibtex. It also includes a large number of font and macro files. 2.2.9 Andrew The subset provides ez and the other applications built on the Andrew Tool Kit. On some platforms it will require the X subset, on others the vendor's X implementation is sufficient to run Andrew. It includes the applications ahelp, bush, eq, ez, ez2ascii, ez2ps, ezviewer, raster, and table, along with several directories of fonts, libraries, and help files. 2.2.10 Applications This subset contains most of the Athena applications, other than those specifically mentioned above. It requires the Athena Basic Services, and the X subset is recommended. Some applications will not run without those subsets loaded. Some of the more notable applications include: olh, delete, mh, rcs, tcsh, eos, imake, notes, kerberized remote access, discuss, dash, perl, and much more. Note that while olh may be copied to the local disk, the documentation database may not be copied local. Not allowing this dynamic database to be copied local assures that the information will not grow stale. 2.2.11 Development This subset contains the libraries, include files, and other tools necessary to write programs that use the Athena basic services. It requires the Athena Basic Services subset since the developer cannot test what has been written without those services. Besides libraries and include files, this subset contains compile_et, mk_cmds, and imake. 2.2.12 Public This subset turns your workstation into an Athena public workstation. It requires all other subsets as well. We do not expect to have this subset ready for the initial release of Layered Athena. 3 Client Programs As described above, there will be three client programs for Layered Athena. All of the real work is done by a simple command line interface which is not intended to be used by end-users. A graphical user interface will be written as a front end to make it more user friendly. There will also be an ASCII friendly interface for those without the X Window System. 3.1 Basic Client The basic client, called layer_athena takes all of its arguments on the command line. The command format is: layer_athena command [options] where there are six different commands defined and a number of options depending upon the command. The set command will set options that apply to all of Layered Athena. Any of these settable options may also be used with the other commands listed below. Root privileges are necessary to set any options other than in a use command. The available options are: layer_athena set autoupdate_noautoupdate owner=email@address location=roomnumber monitor_nomonitor syspack=path save=path_nosave register_noregister custom=script If an option is not specified, it will retain its previous value. The autoupdate flag controls whether this workstation will automatically take new software as Athena releases it. Autoupdates default to being enabled. The owner specifies the email address used for notifications about this workstation. The location is purely informational. The monitor flag specifies that it is OK for the configuration of this workstation to be retrieved via SNMP. This defaults to being enabled. Use syspack to indicate the mount point for the Athena system pack if it is different from /srvd. The save option records where an extra copy of the workstation configuration should be kept. The register flag indicates that I/S should be notified about configuration changes to this workstation. This also defaults to being enabled. The custom option indicates a customization script that should be run whenever Layered Athena changes anything on the workstation. The install command indicates the configuration of individual subsets. Root privileges are necessary to use this command. layer_athena install subset=local_remote_ignore This command indicates, for each subset listed, whether they should be copied to the local disk, setup so that the subset can be used remotely, or not used at all. Many subsets may be specified on the same command line. Any subset not mentioned will keep its previous status. The update and autoupdate commands have no options. Root privileges are necessary to use these commands. They cause the workstation to update any loaded subsets to newer versions if they are available. An update will always do this for all loaded subsets. An autoupdate will check if autoupdates are allowed. If they are, it will update. If not, it will send an email message to the owners of the workstation informing them that an update is necessary. It will record when it sent this message so that duplicate messages are not sent, although after each week that the update is overdue it will generate another message. The use command allows a non-privileged user to use some subsets without installing them on the workstation. The options and syntax are the same as the install command. When a user issues this command, layer_athena will store its state for this in /usr/tmp instead of the regular path, and will verify that everything is in place to get some functionality from the requested subsets. Status messages will be printed to stderr, and a series of csh commands will be printed to stdout which when sourced will setup the user's environment. The status command reports on the current configuration of the workstation. It has two options: layer_athena status verbose_brief If verbose is specified, it will print out several lines of information that are self-explanatory. In brief mode, it will display a compressed version of this information on one line. This command defaults to verbose mode. When layer_athena runs, it will first update any configuration options. Then it will compare the requested configuration with what is on the machine, and the version numbers of the available subsets. From this it will determine if it needs to perform an update. If so, it will first run the macro processor to generate a synctree configuration, then run synctree to put the files in place, and finally run any necessary installation scripts. If it is asked to perform an auto-update, it will check if anyone is logged in first, and create /etc/nologin to make sure that no one logs in during the update. If any of the installed subsets indicate that a reboot is necessary after update, layer_athena will then reboot the workstation. 3.2 GUI Client The GUI client uses the X Toolkit and Athena Widgets, and is called xlayer. Its main screen is shown in Figure 2. Note that the buttons across the top activate menus or actions. The owner and location should be filled in by the user. The buttons below that, "Minimal Configuration", "Standard Configuration", and "Local Configuration" will configure the subsets according to what we believe will be some popular setups. The rest of the window contains a list of the subsets. Clicking on the "status" box following each subset name will pop up a menu allowing the user to set what should be done with that subset. Clicking on the "Description" box following each subset name will pop up a description of what is in that subset. If the user attempts to use a subset without first selecting the any dependent subsets, a dialog will indicate the problem and offer to select the missing subsets. Something similar will happen if a dependent subset is removed. In typical usage, it is only necessary to fill in the owner and location fields, click on the "Standard Configuration" button, then "Do it", which will ask for a confirmation. Other less used functions are found in the pull-down menus at the top. This includes having the workstation not announce itself to Athena Operations, and saving and restoring the configuration from a file. The wording used to describe the various privacy options must be considered very carefully so that most people will choose not to exercise these options. This client will know how to read the subset configuration files to initialize the subset descriptions. However, it will perform all of its real work by running layer_athena, the regular client, as a sub-process. Figure 2: GUI Client Mockup. Clicking on a description button pops up a box with the description of that subset. 3.3 Friendly ASCII Client Because one cannot count on X Windows being available, a friendly terminal-based client also must be available. This client is called layer. To be maximally useful without any documentation or training, this client will display ASCII menus of choices. These menus will be printed teletype-style, without the curses library, so that they may be used in any environment. As with the GUI client described above, it will invoke layer_athena to actually do the work. It will display a short main menu corresponding to the obvious functions at the top of the GUI client. Submenus will handle the less frequently used options and subsets themselves. 4 Transient Use Transient use of Layered Athena is when a non-privileged user of a workstation temporarily uses parts of Athena without permanently modifying the workstation. While not all parts (or even any) of Athena may be usable from any given workstation, where possible we would like to allow users to get at software. While this could be a useful option, it is not practical to implement it at this time. Root privileges are necessary to initially mount the system pack. Most workstations would not have a /mit directory for attach to use. It has been said, "attach is Athena." Since attach is so problematical, it is probably not worth doing this at all, at least for the time being. Some services require minor changes to be able to run without any changes to the local filesystem. These changes are still useful without "transient use," as they make the Athena software more forgiving of configuration errors. These changes are outlined in the following section. 5 Other Changes Several Athena services require changes to run without changes to the root filesystem and without stepping on vendor services. 5.1 Kerberos Kerberos expects to find several things on the local filesystem that must be handled: /etc/athena/krb.conf: Kerberos needs the contents of this file to function properly. The options are to hard-code fallbacks into the programs, or have them look in another place for the configuration file. Since hard-coding configuration into program binaries is generally a bad idea, we will have the library look in several places for this file. It will use a search path of: 1.$ATHENACONF, where this environment variable names a directory that may be on a remote filesystem. 2./etc/athena 3./etc Note that the environment variable is searched first so that it can be used to test alternate configurations on an otherwise properly configured workstation. Also note that the environment variable will be ignored if the real and effective UIDs do not match. This avoids introducing a security hole in setuid programs. /etc/athena/krb.realms: This file will be handled the same as /etc/athena/krb.conf described above. /etc/athena/srvtab: Machines without Kerberos permanently installed have no business providing service and have no need for a srvtab file. /etc/services: It is likely that the /etc/services file will not contain all of the necessary port numbers. The kerberos library will be changed to first look in the local /etc/services file, then try a hesiod lookup of the port number, and finally fall back on a compiled in value if all else fails. As an aid in discovering that the real /etc/services file entries are missing in permanent installations, the fallback code could syslog a warning if (and only if) the /etc/athena/krb.conf file does exist in the expected place. /tmp/tkt*: While the ticket files will be created as needed at runtime, there is the question of cleaning them up at logout. The Athena login takes care of this, but nothing is guaranteed to do this on other systems. The best way for this to be handled is for the user's logout file to contain an invocation of kdestroy. The start_athena script could check for this, and offer to add it for the user if it is not already there. Another consideration for Kerberos is that the vendor also may provide Kerberos. If the vendor's Kerberos is compatible with Athena, then it is just a matter of making sure that the configuration is correct. Without root privileges, if the configuration is not correct, there is nothing that the user can do. If the vendor's Kerberos is incompatible, we can have Athena's version first in all search paths, and hope that the proper one is always found. 5.2 Hesiod Providing hesiod support is somewhat problematical. One cannot start a named daemon without root privileges, as it uses a privileged network port. We are presented with three cases: the workstation does not support the domain name system at all, the workstation has a name server that works but does not support hesiod, or the workstation's named can support hesiod. Note that even in this third case, it will still need to be properly configured. Given the difficulty in both enabling and configuring hesiod, the best choice might be to allow class IN queries for hesiod as well as class HS. This way only a functioning named is required, with no additional configuration. The switch could be handled as an on-the-fly conversion in the hesiod servers, once the MIT servers are taught to forward such requests to the hesiod servers. 5.3 Zephyr The only problem with providing Zephyr in this fashion is making sure that a host manager (zhm) is running and that it knows which Zephyr servers to use. The host manager can run without privileges, so it can be started by the start_athena script (first checking that one is not already running). It will find which servers exist using hesiod. 5.4 Port numbers Many systems will have to have compiled to check hesiod for the port number to use if the value is not in /etc/services, with fallback to a compiled-in value if that fails as well. These systems include: o Moira o Global Message of the Day o Kerberized Pop (inc and movemail) o Kerberized rlogin and rsh o NFS mapping (attach) o OLC o Athena network write o Andrew ahelp 5.5 X The X clients will not require any changes. However, several environment changes will be necessary. XFILESEARCHPATH will need to be set to include where the Athena app defaults files can be found. The server font path will need to be extended to include our fonts. 5.6 TeX TEX will require similar changes to X. The TEXINPUTS environment variable will need to be set to point to the macros directory. 5.7 Emacs Emacs will require changes to find its lisp library files and sub programs such as movemail and loadst. Some changes to the lisp startup code could copy certain environment variables to the appropriate lisp variables when emacs starts. 5.8 SNMP SNMP will be difficult to port to some operating systems. The vendor may provide an snmp agent with some vendor-specific features which should be there. The problem is that we probably cannot add the Athena local MIB to this vendor agent, so are faced with the choice of the vendor's extensions or our agent, which will have our extensions but may not be able to find some of the basic host information much less any vendor extensions.