From: Tschenett Helmuth <helli@siemens.co.at>
Message-Id: <199511270910.AA184383454@ws2313>
Subject: Using pthreads
To: proven@MIT.EDU
Date: Mon, 27 Nov 95 10:10:54 MET
Mailer: Elm [revision: 70.85]

Hi Chris Provenzano!

A few days ago i dumped your thread package from the internet because
i have a C++ class library to become threadsave.

All was working fine and it seems to me that for my needs the package is
very good. I found no bugs.

But in confront with the kernel threads of SUN SOLARIS there was a 
difference in the scheduling strategy in conjunction with
mutex lock variables. I took a look at the code and i found the following:

if there are two threads making lock and unlock calls to the same mutex
variable, the threads are scheduled in an alternative manner after each
unlock call. 
This is because unlocking a mutex variable immediatly changes the owner of it
to the first waiting thread for this mutex variable. This is a possible
way of doing but it has consequences.
If the two threads have high load of lock unlock calls the time slice for each
thread to do normal work is very small. The maximum time possible is from one
lock call to the next lock call. Thus the number of context switches is 
very high.
This strategy resulted in a very poor performance for my library when using 
more than one thread running the same thread function:

		1 thread:	2sec
		2 threads:	20sec
		4 threads:	60sec

I changed the code for mutex.c and now the scheduling seems to be more 
balanced over the running threads. At least the performance for my case was
much better:

		1 thread:	2sec
		2 threads:	5sec
		4 threads:	9sec

I made the changes only in pthread_mutex_lock and pthread_mutex_unlock
and only for the MUTEX_TYPE_FAST case. But the other changes should be similar.
The policy now is, that by unlocking a mutex variable the mutex owner is set
to NULL and the first thread waiting for this mutex variable is rescheduled.
(There where strange results when the main thread makes a call to lock.)
/* ==========================================================================
 * pthread_mutex_lock()
 */
int pthread_mutex_lock(pthread_mutex_t *mutex)
{
	int rval;

	pthread_sched_prevent();
	switch (mutex->m_type) {
	/*
	 * Fast mutexes do not check for any error conditions.
     */
	case MUTEX_TYPE_FAST: 
	case MUTEX_TYPE_STATIC_FAST:
		while (1) {
			if (!mutex->m_owner) {
				/* for the main thread the pthread_run variable is not
				   set so we have to make the following if. */
				if (mutex->m_owner=pthread_run) {
					pthread_queue_remove(&mutex->m_queue,pthread_run);
				}
				break;
			}
			else {
				pthread_queue_enq(&mutex->m_queue, pthread_run);
				pthread_resched_resume(PS_MUTEX_WAIT);
				pthread_sched_prevent();
			}
		}
		rval = OK;
		break;
	default:
		rval = EINVAL;
		break;
	}

	pthread_sched_resume();
	return(rval);
}

/* ==========================================================================
 * pthread_mutex_unlock()
 */
int pthread_mutex_unlock(pthread_mutex_t *mutex)
{
	struct pthread *pthread;
	int rval;

	pthread_sched_prevent();
	
	switch (mutex->m_type) {
    /*
     * Fast mutexes do not check for any error conditions.
     */
    case MUTEX_TYPE_FAST:
    case MUTEX_TYPE_STATIC_FAST:
		mutex->m_owner = NULL;
		pthread=pthread_queue_deq(&mutex->m_queue);
		if (pthread) {
			pthread_sched_other_resume(pthread);
			return(OK);
		}
		rval = OK;
		break;
	default:
		rval = EINVAL;
		break;
	}
	pthread_sched_resume();
	return(rval);
}

Hopefully my considerations are usefull to you.

Best regards.

--
Helmuth  Tschenett                                    helli@siemens.co.at
SIEMENS AG AUSTRIA                                    
Gudrunstr. 11                                         +43-1-60171-5666 (tel)
A-1100 Vienna, Austria                                +43-1-60171-5712 (fax)