\documentstyle[fullpage]{article}
\begin{document}

\begin{center}
{\Large Moira Server Internals}
\end{center}

\section{Basic Operation}

On startup, {\tt main()} initializes the com\_err library and
(assuming no {\it /etc/smsdown\/}) the connection to the DBMS. It
sets up signal handlers for {\tt SIGTERM}, {\tt SIGUSR1}, {\tt
SIGUSR2}, {\tt SIGCHLD}, and {\tt SIGHUP}, opens the journal file, and
finally opens a listening connection. It then loops forever,
processing requests and timing out idle connections.

A semaphore file, {\it /etc/smsdown}, and the signals {\tt SIGUSR1}
and {\tt SIGUSR2} control the availability of the database. If the
semaphore exists when the server is first started, or when the last
active client connection is closed, then the database will be closed.
This means that it disconnects from the DBMS backend, and no queries
will be answered. The contents of the semaphore file are returned as
the Message of the Day while it is present, even if the database is
still open. This will cause most clients to disconnect, although the
protocol doesn't require this. If the semaphore is no longer there
when a new client connects to the server, the database will be opened
again. Rather than wait until the last client disconnects or first
client connects to close and open the database, {\tt SIGUSR1} or {\tt
SIGUSR2} may be sent to the server to cause this to happen as soon as
the current query finishes executing, regardless of how many clients
are currently connected. Closing and opening the database also flushes
the name to ID cache.

For each connection, the server creates a client structure. The client
structure contains connection state, authentication information, and
the array of tuples currently waiting to be sent back to the client.

The operations supported are {\bf noop, auth, motd, query, access,
proxy} and {\bf version}. In {\tt server/mr\_scall.c(client\_read)},
these operations are decoded and dispatched. (If the database is
closed, only the {\bf noop} and {\bf motd} operations are allowed.)

The {\bf auth} and {\bf proxy} operations are taken care of in {\tt
server/mr\_sauth.c}. {\bf Auth} requires the client to send a Kerberos
ticket for {\bf moira}.{\it servername\/}@{\it localrealm}. It will
extract the client's principal name, and use that for later
authentication purposes. To speed up later access checking, it will
also resolve the principal name to the {\it users\_id} of that person
in the Moira database. If the Kerberos principal does not correspond
to a user in the database, but does correspond to a Kerberos principal
in the {\it strings\/} table, then that {\it string\_id\/} will be
stored in the client record. The {\bf auth} request also takes the
name of the client program used to connect, which is recorded in the
{\it modwith\/} field of data that the client modifies.

The {\bf proxy} operation checks that the user is on the ``proxy''
{\it capacl\/} entry, and if so, changes the user that the client is
authenticated as. Proxied users are not allowed to do certain
operations, as explained later.

The {\bf motd} operation checks for a Message of the Day (in {\it
/etc/smsdown\/}), and returns it if it exists. Clients should check
the motd before doing anything else, and exit if it exists.

When a {\bf query} is requested, {\tt server/mr\_scall.c(do\_retr)} is
called. It first checks to see if the query is {\bf \_list\_users},
and calls {\tt list\_users} if so. Otherwise, it calls {\tt
server/qrtn.pc(mr\_process\_query)} to do the actual work. For each
tuple to be returned, {\tt server/mr\_scall.c(retr\_callback)} will be
called, and will in turn call {\tt
server/mr\_scall.c(client\_return\_tuple)} to queue the data to be
returned in the client structure. Finally, {\tt
server/mr\_scall.c(client\_reply)} is called to set the return value
of the query.

The {\bf access} operation works similarly, calling {\tt
server/mr\_qrtn.pc(mr\_check\_access)} to run the query's
access-checking routines without executing the query.

The {\bf version} operation tells the server what the highest query
version the client knows about it. If no {\bf version} call is made,
the server assumes the client is using query version 2. Passing ``-1''
for the version parameter tells the server to use the highest-known
query version by default.

\subsection{Query processing}

A query has at most 5 phases:

\begin{description}
\item[Validation]---The number, size, and type of the query arguments
are checked. Most queries that update or look up objects make sure
that the named objects exist during this step, and most queries that
add new objects to the database or update existing objects check the
object's name(s) or ID for uniqueness in this step.

\item[Access Check]---First the query is looked up in the {\bf
capacls} table. If an entry is found, the server checks if the client
is on the indicated list. If so, the {\it priviliged\/} flag is set,
and the access check phase is done. Otherwise, the query-specific
access routine, if any, is run. If the query has no access routine and
the client is not on the capacl, the server returns {\tt MR\_PERM}.

\item[Setup]---If the query has a setup routine, it is run next.
Queries that delete objects use this step to make sure the object to
be deleted is no longer referenced in the database. Other queries use
it to perform argument validation that is more complicated than the
validation step allows.

\item[Main Processing]---For most queries, this will be a single SQL
statement, defined in the query definition, but for very complicated
queries, this will be a C routine (which also doubles as the followup
step).

\item[Followup]---Much of the followup phase is an artifact of Ingres.
Most of the queries only select data out of a single table, and the
resolve the cross-references in followup routines. Some newer ones
actually just do multiple-table selects in the main processing step.
However, followup routines are always needed to parse type/ID
combinations (like ACLs), and modby fields. This is also the step
where the modby/modwith/modtime fields are set by queries that change
the database.
\end{description}

{\tt mr\_process\_query} is the heart of the server. First it catches
the special queries {\bf \_list\_queries} and {\bf \_help}. If the
query is not one of those, it looks it up in the query dispatch table.
Next, the query is validated with {\tt mr\_verify\_query}. This takes
care of the validation and access checking steps. If the validation
passes, {\tt mr\_process\_query} will perform any specified setup
routine. Then it saves the `before' copy of the data for use in
incremental updates. Next, the actual query is performed, followed by
the followup routine if specified. Then, the `after' copy of the data
is saved for possible incremental updates. If the server modified the
database and everything was successful, it commits the transaction,
begins any necessary incremental updates and logs it all in the
journal. If the query was not successful, it rolls back the
transaction.

An access operation is like an actual query, except that it only does
the query validation.

Incremental updates are handled as a queue of pairs of `before' and
`after' argument vectors. Each query saves these vectors as it
executes, and if the transaction is committed, then
incremental\_update() is called begin any updates. A given database
change only starts an incremental update if the `incremental' table in
the database indicates that a service has interest in that table and
the program /moira/bin//{\it service}.{\bf incr} exists.

\subsection{Logging}

The server itself logs to standard output.  However, it is normally
started with the {\tt startmoira} program, which will direct the logs into
{\tt /moira/moira.log}.

\section{Source Organization}

Most of the server sources are in the server\footnote{All paths are
given as relative to the root of the Moira source tree.} subdirectory.
The {\bf mr\_*.c} files deal with the operations of the server that
don't directly interact with the DBMS:

\begin{description}
\item[mr\_main.c]	Initialization and top-level dispatch routines.

\item[mr\_scall.c]	Deals with most of the protocol processing

\item[mr\_sauth.c]	Performs the {\bf auth} and {\bf proxy} operations.

\item[mr\_shutdown.c]	Catches the no-longer-supported shutdown operation.

\item[mr\_util.c]	Various utility routines needed within the server.
\end{description}

The {\bf q*.pc} and {\bf q*.c} files implement the queries, and should be
the only part of the server that depends on the DBMS.

\begin{description}
\item[qrtn.qc]		This has the routines to interpret the query
			dispatch table and execute queries.  It also
			handles opening and closing the database, and
			catching DBMS errors.

\item[qsubs.c]		This has subroutines needed by the query
			software which are in pure C code without
			embedded SQL.

\item[queries2.c]	This is the dispatch table for version 2 of
			the protocol. The structures used here are
			defined in {\bf query.h}.

\item[qvalidate.pc]	Generic query validation routines, to check
			the arguments of queries.

\item[qaccess.pc]	This contains the routines used to check
			access to specific queries.

\item[qsetup.pc]	Setup routines for specific queries.

\item[qsupport.pc]	Code to implement complicated queries that
			can't be done in a single line of SQL.

\item[qfollow.pc]	Followup routines for queries, to set the
			modtime fields, reformat data, etc.

\item[increment.pc]	Routines to handle incremental updates.  This
			touches the database so that it can capture
			the before and after images of the data.

\item[cache.pc]		The name-to-id cache used to cut down on
			database accesses.
\end{description}

The server also links against the moira library for some common
routines, the zephyr library for error notices, the Kerberos and DES
libraries for authentication, the com\_err library for error reporting
and logging, and lots of other libraries needed by the DBMS.


\section{Query Dispatch Table}

The query table consists of an array of records, one for each query.

\begin{verbatim}
/* Query Definition Structure */

struct query
{
    char *name;			/* query name */
    char *shortname;		/* abbreviated query name */
    int  version;		/* query version */
    enum query_type type;	/* query type */
    char *rvar;			/* range variable */
    enum tables rtable;		/* range table */
    char *tlist;		/* target list */
    char **fields;		/* input and output field names */
    int  vcnt;			/* variable count */
    char *qual;			/* format string for `where' clause */
    int  argc;			/* number of args for qualifier */
    char *sort;			/* fields to sort on */
    struct validate *validate;	/* validation support */
};
\end{verbatim}

The {\it name\/} is the full name of the query, as in
`get\_user\_account\_by\_login'.

The {\it shortname\/} is a 4 character abbreviation of the name.  It is
important that this be exactly 4 characters long, and must be unique
among all of the queries in the table.

The {\it version\/} indicates the version number of the query.
Multiple versions of a query can be defined, but it is important that
the appear in the Queries table in order from oldest to newest.

Note that versions behave like timestamps, in that the ``highest-known
query version'' is a global value. When a query interface is changed,
it should get a new version number which is 1 larger than the highest
existing query version number, {\em not\/} necessarily a version
number which is 1 larger than its own previous version number.

The {\it type\/} is one of:

\begin{verbatim}
/* Query Types */
enum query_type {RETRIEVE, UPDATE, APPEND, DELETE, SPECIAL};
\end{verbatim}

The {\it rvar\/} specifies the range variable to be used in this
query, in a range statement in conjunction with rtable. It should be
the same variable (if any) used in the {\it tlist}, if any. If the
query is implemented with a special routine (in {\tt qsupport.pc}),
this field should be left blank.

The {\it rtable\/} specifies the table to be operated on in this
query. For special queries, this field may be left blank. If this is
filled in, the statistics for that table will be updated when the
query is processed.

{\it tlist\/} is the target list to be used in the query about to be
executed. This specifies the variables to be retrieved or stored
during the query. This is used to contruct the SQL query. Queries
implemented with C routines can have this left blank.

{\it fields\/} lists input and output field names displayed by the
{\it \_help\/} query. Help is the only thing they are used for.

The variable count ({\it vcnt\/}) is the total number of arguments for an
append, update or delete, or the number of return values for a
retrieve.

{\it qual\/} is the format string for the qualifier. This will be used
in the where clause of the SQL query, and should have \%s and/or \%d
in it to insert values from the query arguments. (Note that \%d can
only be used when a query setup or validation routine has replaced one
of the original string arguments with a number.)

{\it argc\/} is the number of args that will be used in the qualifier, as
described above.

\bigskip

\noindent Validation support: if {\it validate\/} is non-null, it
points to a validation structure:

\begin{verbatim}
/* Query Validation Structure */
struct validate
{
    /* object validation (e.g., user, filesys, type) */
    struct valobj *valobj;	/* object validation descriptors */
    int objcnt;			/* size of array */
    /* row validation: retrieve (exists=any(rvar.field where qual)) */
    char *field;		/* field to check for */
    char *qual;			/* format string for `where' clause */
    int  argc;			/* number of args used in qual */
    /* values field containing current max object id */
    char *object_id;
    /* routine to verify access permission on objects */
    int (*acs_rtn)(struct query *q, char *Argv[], client *cl);
    /* pre-processing routine (var setup only) */
    int (*pre_rtn)(struct query *q, char *Argv[], client *cl);
    /* post-processing routine */
    int (*post_rtn)();
};
\end{verbatim}

{\it valobj\/} is a pointer to an array of validation objects. (See
below.) There will be one entry in this array for each argument that
needs to be validated. This pointer may be NULL if {\it objcnt\/} is
zero.

{\it objcnt\/} is the count of entries in the valobj array. (Many {\it
valobj\/}s are used by multiple validation structures, but with
different {\it objcnt\/}s.)

{\it field}, {\it qual}, and {\it argc\/} are used together for row
validation (verifying that a specified row already exists in the
table). {\it field\/} names the field to look for, {\it qual\/} is a
format string for the `where' clause of the query that does the row
validation. It is like the {\it qual\/} field in the main dispatch
table. {\it argc\/} is the count of arguments used in the qualifier.
(You must use the arguments from the query in order.)

{\it object\_id\/} is the name of the ID field in the record being updated.
It has two purposes:  it is used to allocate new IDs automatically
for appends, and it is used to find the proper row to set the modtime
information after an append or update.

The {\it acs\_rtn\/} is the access routine. This is called if the
client is not on the query ACL to determine if they will be allowed to
perform the query. (Clients on the query ACL are permitted to perform
the query without the {\it acs\_rtn\/} being called.)

The {\it pre\_rtn\/} is the query setup routine. If non-NULL, this
will be called after the validation and before performing the actual
query.

The {\it post\_rtn\/} is the query followup routine or, for queries
with no {\it rvar\/}, the actual query routine. Currently, there are
two different kinds of {\it post\_rtn\/}s, called by {\it
mr\_process\_query} with different arguments in different
circumstances. This will probably be changing in a later release of
Moira.

\bigskip

\noindent Validation objects are defined as follows:

\begin{verbatim}
/* Validated Object Types */
enum vo_type {V_NAME, V_ID, V_TYPE, V_TYPEDATA, V_RENAME, V_CHAR,
	      V_LEN, V_NUM};

/* Validated Object Definition */
struct valobj
{
  enum vo_type type;
  int index;			/* index of object to validate */
  enum tables table;     	/* table containing object */
  char *namefield;		/* table's fieldname for object */
  char *idfield;		/* table's corresponding id field (if any) */
  int error;
};
\end{verbatim}

{\it type\/} gives the type of validation to be performed. {\it
index\/} tells which query argument should be checked. {\it table},
{\it namefield}, and {\it idfield\/} are used to identify a table and
a name field, and ID field within that table. {\it error\/} gives an
error to return if validation fails.

The validation types are:

\begin{description}
\item[V\_NAME]---Used by {\tt APPEND} queris to verify that the
indicated argument does not have the same name (as indicated by the
{\it namefield\/}) as any other existing row. Also checks that the
name fits in the given table field.

\item[V\_ID]---Verifies that the named argument exists in the table,
and replaces the char * pointing to the name in the query argv with
an int * pointing to its ID (meaning the query {\it tlist\/} will use
a {\tt \%d} to refer to it).

\item[V\_TYPE]---Verifies that the argument is one of the values in the
{\it alias\/} table with name indicated by {\it namefield} and type
``TYPE''. (E.g., if the {\it namefield} contains the string
``boolean'', the argument must be one of ``TRUE'', ``FALSE'', and
``DONTCARE''.)

\item[V\_TYPEDATA]---Must follow a {\bf V\_TYPE} validation. Checks
that the argument matches the kind of data indicated for the given
type in the {\it alias\/} table, and replaces it with an ID, as with
{\bf V\_ID} above. For example, if the {\bf V\_TYPE} was ``member'',
and the argument was ``LIST'', the {\bf V\_TYPEDATA} must be a list
(the value of the {\it alias} entry with name ``LIST'' and type
``TYPEDATA''), and the validation will translate the list name into a
{\it list\_id}.

\item[V\_RENAME]---Like {\bf V\_NAME}, but for {\tt UPDATE} queries.
Checks that the new name is either unique, or the same as the old
name. (It assumes the old name is the query argument immediately
before the new name.)

\item[V\_CHAR]---Verifies that the argument consists only of
``reasonable'' characters, and fits in the given table field.

\item[V\_LEN]---Verifies only that the argument fits into the given
table field.

\item[V\_INT]---Verifies that the argument is a number.

\end{description}

\end{document}
