To: thorpej@netbsd.org, sommerfeld@netbsd.org, marc@mit.edu
Subject: scheduler activations interface draft
From: nathanw@mit.edu (Nathan J. Williams)
Date: 26 Jun 2000 18:25:34 -0400
Message-ID: <mtu3dm03wrl.fsf@contents-vnder-pressvre.mit.edu>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Lines: 227
Xref: contents-vnder-pressvre.mit.edu mit.thesis:19


Each of you expressed interest in the scheduler activations work at
some point, so I'm giving you the first crack at critiquing my
user-level interface spec.

This more-or-less assumes that you're familiar with the original ideas
of the scheduler activations paper. I have a copy of it at
http://web.mit.edu/nathanw/work/sched/papers/p53-anderson.pdf (that
directory includes a number of other relevant papers, including the
async threads paper and the scheduler activations on Mach and BSD/386
papers). 

I'd appreciate any comments you have, and can answer questions about
design decisions (which are set in soft styrofoam at the moment).

        - Nathan


/* Draft interface for user programs to interact with scheduler
 * activations subsystem.
 * $Id: interface.h,v 1.2 2000/06/26 22:26:13 nathanw Exp $
 */


/* The sa_t structure is used to describe the state of an activation
 * to the user process. 
 *
 * The sa_context field contains the user-space context of an
 * activation that has blocked or been preempted. See ucontext.h,
 * setcontext(), swapcontext(), etc. for usage.
 * 
 * The sa_id field is a small integer which uniquely identifies the
 * activation in the context of the process. It may be used by
 * user-space code to associate activations with execution contexts.
 * 
 * The sa_cpu field is a small integer which identifies the CPU that
 * an activation is or was last running on. It is provided to permit
 * the user-space scheduler to manage processor affinity. 
 *
 * XXX This is only useful for the "new" activation; it tells the
 * XXX upcall what processor it is running on. The processors of the
 * XXX other activations are known from when they started, and can be
 * XXX stored by the user-level code and looked up by sa_id. Would it
 * XXX better to make the structure smaller by eliminating sa_cpu, and
 * XXX providing a whatcpuami() syscall, or by making the cpu value a
 * XXX upcall parameter, rather than part of struct sa_t?
 */

struct sa_t {
	ucontext_t sa_context;
	int sa_id;
	int sa_cpu; 
};

/* This is the signature of the upcall routine that the kernel
 * invokes. The parameters are as follows:
 * 
 * "new" points to the activation that the upcall is running on. The
 * sa_context field does not contain meaningful information.
 *
 * "event" points to a null-terminated array of pointers to sa_t
 * structures describing the activations involved in an event. 
 * 
 * "interrupted" points to a null-terminated array of pointers to sa_t 
 * structures describing the activations that were stopped in order to
 * deliver this upcall; that is, the "innocent bystanders". 

 * XXX Provide a length and base sa_t * instead of a null-terminated
 * XXX array? The structures have to be allocated on the upcall stack
 * XXX anyway, so they're going to be contiguous.... I have no idea
 * XXX which is better/more efficent/preferred style.
 * 
 * The "aux" pointer points to optional data.
 */
typedef void (*sa_upcall_t)(struct sa_t *new, struct sa_t **event, 
							struct sa_t **interrupted, void *aux);

/* SA_UPCALL_NEWPROC is used to notify the process of a new processor
 * allocation. Since it runs on the new processor, both "event" and
 * "interrupted" should be empty. The "aux" parameter is not used.
 */
#define SA_UPCALL_NEWPROC		0

/* SA_UPCALL_PREEMPTED is used to notify the process of a reduction in
 * its processor allocation. There may be multiple "event" activations
 * if the allocation was reduced by several processors. The "aux"
 * parameter is not used.  
 */
#define SA_UPCALL_PREEMPTED		1

/* SA_UPCALL_BLOCKED is used to notify the process that an activation
 * has blocked in the kernel. Since a processor is made avaliable by
 * the blocking of the activation, the "interrupted" set should be
 * empty (XXX might this be false?) The "aux" parameter is not used.
 * The context of the event sa should not be continued until a
 * SA_UPCALL_UNBLOCKED event has been delivered for the same
 * activation.  
 */
#define SA_UPCALL_BLOCKED		2

/* SA_UPCALL_UNBLOCKED is used to notify the process that an
 * activation which previously blocked (and for which a
 * SA_UPCALL_BLOCKED upcall was delivered) is now ready to be
 * continued. The "aux" parameter is not used.
 */
#define SA_UPCALL_UNBLOCKED		3

/* SA_UPCALL_SIGNAL is used to deliver a signal to the process. If the
 * signal is a synchronous trap (ILL, TRAP, FPE, BUS, SEGV), then
 * "event" points to the activation which triggered the trap. For
 * asynchronous signals, "event" is the empty set. 
 * The "aux" parameter points to a structure containing the signal
 * number and signal code (when appropriate). 
 * XXX specify!
 */
#define SA_UPCALL_SIGNAL		4

/* SA_UPCALL_NUPCALLS gives the number of upcalls defined */
#define SA_UPCALL_NUPCALLS		5


/* Register the upcalls that will be used with the kernel.  
 * 
 * The parameter new_upcalls points to an array of SA_UPCALL_NUPCALLS
 * function pointers, indexed by the SA_UPCALL_* macros.
 *
 * The paramter old_upcalls is either NULL, or points to an array of 
 * length SA_UPCALL_UPCALLS that will be filled in with the values of 
 * any previously existing upcalls (or NULL if none were registered).
 * 
 * Return values:
 *   The value 0 indicates that the call succeeded.  The value -1
 *   indicates an error occurred and errno is set to indicate the
 *   reason.
 *
 * Errors:
 *   sa_register_upcalls() will fail and no new upcalls will be installed if 
 *   one of the following occurs:
 *  
 *   [EFAULT]	Either new_upcalls or old_upcalls points to memory that is
 *              not a valid part of the process address space.  
 */
int sa_register_upcalls(sa_upcall_t *new_upcalls, sa_upcall_t *old_upcalls);


/* Provide the kernel with stacks to be used for making upcalls. 
 * 
 * It is the caller's responsibility to not pass back stacks that are still
 * in use, or that have been previously registeresd but not yet used.
 *
 * Return values:
 *   The number of stacks registered is returned (and may be 0). The
 *   stacks are used in reverse order, starting with *(stacks+num-1).
 *   The value -1 indicates an error occurred and errno is set to
 *   indicate the reason.
 *
 * Errors:
 *   sa_use_stacks() will fail if one of the following occurs: 
 *
 *   [EINVAL]	"num" is not a valid number of stacks (negative).
 * 
 * */
int sa_use_stacks(int num, stack_t *stacks);


/* Begin using scheduler activations.
 *
 * This call does not return on success. Instead, a SA_UPCALL_NEWPROC
 * upcall is made. 
 * 
 * Return values: 
 *   This call returns -1 on failure, and does not return on success.
 *
 * Errors:
 *   sa_enable_activations() will fail if scheduler activations have
 *   already been enabled.
 */
int sa_enable_activations(void);


/* Set desired level of concurrency.
 *
 * This call tells the kernel the maximum number of
 * concurrently-executing activations that it could use. The value
 * will be considered in the processor allocation algorithim.
 * 
 * Return value:
 *   Previous level of concurrency (initially 1). 
 *   -1 on error.
 *
 * Errors:
 *   sa_setconcurrency() will fail if one of the following occurs:
 *
 *   [EINVAL]	"concurrency" is not a valid level of concurrency
 *              (smaller than 1).
 *   [??????]   sa's are not in use.
 */
int sa_setconcurrency(int concurrency);

/* Indicate that this processor is no longer useful to the process. 
 * Reduces the process's desired level of parallelisim by 1.
 * Does not return on success.
 *
 * Return values:
 *  Does not return on success.
 *  Returns -1 on failure.
 *
 * Errors:
 *  - sa's are not in use.
 */
int sa_yield_processor(void);


/* Cause another activation to be preempted.
 *
 * This call causes another running activation to be preempted and a
 * SA_UPCALL_PREEMPTED upcall to be delivered. It is useful for
 * gaining control of another activation in a multiprocessor
 * environment. 
 * 
 * Errors:
 *  - named sa does not exist.
 *  - named sa is not running.
 *  - sa's are not in use.
 */
int sa_preempt(int sa_id);

