How to generate ε-TEX

(Version 1, October 1996)

by The NTS Team

Peter Breitenlohner, Max-Planck-Institut für Physik, München Philip Taylor, RHBNC, University of London

Given an implementaion of TEX82 for a particular system, this report describes how to generate a corresponding implementation of ε-TEX.

The preparation of this report was supported in part by DANTE, Deutschsprachige Anwendervereinigung TEX e.V. ‘TEX’ is a trademark of the American Mathematical Society.

Introduction. Let us first review the process of generating an implementation of TEX82 for a particular system from the source files as, e.g., described in the WEB manual [1]. The system independent source file tex.web must remain unmodified. All changes to tex.web necessary for a particular operating system and/or compiler are to be collected in a system dependent change file, typically named tex.ch. Both files tex.web and tex.ch are effectively merged when input by the WEB system programs WEAVE and TANGLE. When WEAVE processes this merged input, a file tex.tex is produced. Further processing by TEX yields a ‘pretty printed’ version of the input together with an index.

When TANGLE processes the merged input, a string pool file tex.pool and a Pascal file tex.pas (or similar) are produced. The Pascal file can then be further processed by a suitable compiler and/or language converter such as web2c, and eventually yields an executable program.

There are actually three versions of the program: First there is INITEX with its capability to initialize all of TEX’s tables and to write them in compact form to a format file. Next there is the production version VIRTEX requiring a format file as input. Finally there is TRIPTEX, a version of INITEX with special values for some of TEX’s parameters, for the TRIP test [2] that should be used to validate a TEX implementation. Depending on the capabilities of the compiler, these three versions of the program are generated from three slightly different change files or they are generated from one change file with different compiler options. They might even be one and the same executable program used with different run time options.

Generating ε-TEX. The process of generating ε-TEX is essentially the same as that of generating TEX as described above. Conceptually there is a system independent source file etex.web and a system dependent change file etex.sys. Processing these two files by TANGLE yields a string pool file etex.pool as well as a Pascal file etex.pas, whilst processing by WEAVE produces a TEX source file, etex.tex.

It may, however, be necessary to increase some of the constants defined in TANGLE and WEAVE. The following values should suffice in most cases:

max bytes × ww = 100 000  TANGLE and WEAVE
max texts = 2 500  TANGLE and WEAVE
max toks × zz = 180 000  TANGLE
max names = 5 000  TANGLE
max scraps = 3 000  WEAVE
stack size = 300  WEAVE

The source file etex.web for ε-TEX does not, however, exist as a physical file. It is the hypothetical file obtained by applying the changes in the actual source file etex.ch to tex.web. Thus etex.web inherits the bulk of code from tex.web, whilst the system independent source file etex.ch for ε-TEX defines the differences between etex.web and tex.web. In order to generate an implementation of ε-TEX two change files have to be applied to tex.web, one after the other (the actual file names may differ):

0.  tex.web  system independent WEB source for TEX
1.  etex.ch  system independent changes for ε-TEX
2.  etex.sys  system dependent changes for ε-TEX

The process of merging several change files into tex.web should certainly not be performed by hand. There are programs such as TIE and PATCHWEB that perform this process automatically. If no such program is available, a TEX program WEBMERGE can be used. WEBMERGE reads tex.web and up to nine change files and produces a merged change file that can then be processed, together with tex.web, by TANGLE and WEAVE. [On systems such as VMS, use of WEBMERGE can leave a large number of temporary files lying around; this can be avoided by setting a version limit (e.g. 1) on any existing versions of those files, or by setting a version limit on the directory in which they will be created. On other systems, it will probably leave one large temporary file.]

Every implementor of ε-TEX is responsible for creating and maintaining a suitable etex.sys in the same way as every implementor of TEX is responsible for creating and maintaining tex.ch. Since the bulk of code

2 GENERATING ε-TEX

in etex.web is identical to that in tex.web the bulk of the system dependent changes in etex.sys for a particular system will be identical to those in tex.ch for the same system. In the following we try to give some hints where etex.sys for a particular system might deviate from the corresponding tex.ch.

First, it might be necessary to increase the size of TEX’s string pool in order to accommodate ε-TEX’s additional strings (message texts as well as multi-letter control sequences). If this turns out to be necessary for ε-TEX it would certainly not be harmful to do it for TEX as well. TEX and ε-TEX use three constants related to the string pool: max strings the maximal number of strings, pool size the maximal number of string characters, and string vacancies the minimal number of available string characters in addition to those occupied by strings from the pool file. It is, therefore, sufficient to increase pool size (or reduce string vacancies ) by the number of ε-TEX’s additional string characters and to increase max strings by the number of ε-TEX’s additional strings. The later will, however, be unnecessary for most implementations as max strings is usually increased substantially beyond its standard value in order to accommodate large TEX macro packages.

For Version 1 of ε-TEX, there are less than 100 additional strings with less that 1000 additional string characters. The precise numbers can be obtained by running POOLTYPE on TEX’s and ε-TEX’s pool files (POOLTYPE reports the total number of strings and string characters in a pool file).

Next, etex.sys may contain a system dependent modification of the eTeX banner string. The modified eTeX banner string must contain ‘e-TeX’ as well as the ε-TEX version number. Note, however, that the banner string modified by tex.ch will not be referenced by ε-TEX unless the implementor intentionally changes that aspect of ε-TEX’s functionality: therefore etex.sys can modify the banner string in the same way as does tex.ch.

Then, etex.sys might deviate from tex.ch in order to use a different pool file name and/or format file extension (see below).

Finally, etex.sys will necessarily deviate whenever etex.ch and tex.ch try to change the same piece of WEB code or when the system independent ε-TEX changes from etex.ch and the system dependent TEX changes from tex.ch interfere in some other way. In case of any such interference implementors must first of all determine how to combine the respective changes from etex.ch and tex.ch in order to obtain ε-TEX’s functionality for a particular system. Obviously, this process cannot be automated since it requires human insight.

The NTS team has tried to formulate etex.ch such that interferences with system dependent change files tex.ch are unlikely. Suggestions by implementors how any remaining such interferences could be avoided by a reformulation of etex.ch will be taken under serious consideration. Such interferences can be further reduced by reformulating the system dependent change file tex.ch for TEX, e.g. by reducing the range of change entries from entire modules to the pieces of code that are actually changed.

Implementors might prefer to maintain the system dependent change file etex.sys not as a physical file but as a hypothetical file defined through its deviation from tex.ch. If there are no interferences of the kind mentioned above, then the effect of applying the changes from the hypothetical etex.sys to the hypothetical etex.web can be achieved by applying 3 change files, one after the other, to tex.web (using some tool such as TIE, PATCHWEB, or WEBMERGE):

0.  tex.web  system independent WEB source for TEX
1.  etex.ch  system independent changes for ε-TEX
2.  tex.ch  system dependent changes for TEX
3.  tex.ech  additional system dependent changes for ε-TEX

The third change file tex.ech will be rather short and contains just the differences between etex.sys and tex.ch. It is recommended that implementors try to remove all interferences between etex.ch and tex.ch and use this method to generate ε-TEX.

As with TEX there are three versions of ε-TEX: e-INITEX, e-VIRTEX, and e-TRIPTEX. Depending on the implementation they will again be generated from the three slightly different versions of tex.ch or with different compiler options or they may be one and the same program used with different run time options.

ε-TEX MODES 3

ε-TEX modes. In order to ensure maximal compatibility with TEX, ε-TEX can run in either compatibility mode or extended mode. The possibility of this choice is, of course, an extended feature of ε-TEX. Once ε-TEX has chosen compatibility mode it is, however, a legitimate implementation of TEX (assuming the TEX implementation itself is legitimate). The only differences between ε-TEX in compatibility mode and TEX are those allowed by D.E. Knuth [2] between different implementations of TEX.

An e-TRIP test suite [3] defines the criteria by which a program can qualify to use the name ‘ε-TEX’. Part of the e-TRIP test consists of the standard TRIP test for e-TRIPTEX in compatibility and extended mode.

ε-TEX can therefore be used instead of TEX without the necessity to maintain both programs. For the case that both programs should nevertheless co-exist on a system, it might be a good idea to name the pool file for ε-TEX etex.pool instead of tex.pool and to use an extension other than .fmt, e.g., .efmt for ε-TEX format files. (Format files for TEX and ε-TEX are incompatible). All this will require additional changes in

the file tex.ech.

When INITEX or VIRTEX start, they inspect the first non-blank character from the command line or in response to the ** prompt. This may be an & immediately followed by the name of a format file to be loaded. Otherwise VIRTEX loads a default format, whereas INITEX starts without loading a format.

When e-INITEX or e-VIRTEX start, they inspect the first non-blank character from the command line or in response to the ** prompt. This may again be an & immediately followed by the name of a format file to be loaded; otherwise e-VIRTEX loads a default format. For e-INITEX the first non-blank character may be an * immediately followed by what would normally be the input for INITEX (without intervening blanks). e-INITEX enters extended mode in response to the *, or compatibility mode otherwise. This mode is recorded in format files produced by e-INITEX and entered again when such a format is loaded (by either

e-INITEX or e-VIRTEX).

References.

[1] The WEB system of structured documentation, by Donald E. Knuth, Stanford Computer Science Report 980.

[2] A torture test for TEX, by Donald E. Knuth, Stanford Computer Science Report 1027.

[3] A torture test for ε-TEX, by The NTS Team (Peter Breitenlohner and Bernd Raichle).