Message-Id: <199602160054.QAA13637@cygnus.com>
Reply-To: jtc@cygnus.com
To: netbsd-developers@NetBSD.ORG
Subject: NetBSD 1.1 Release Post Mortem
Date: Thu, 15 Feb 1996 16:54:40 -0800
From: "J.T. Conklin" <jtc@cygnus.com>
Sender: owner-netbsd-developers@NetBSD.ORG
Precedence: first-class

As most of you know, I never kicked off the 1.1 release post mortem.
This is bad, considering the fact that we need to start planning for
NetBSD 1.2 if we're to have a spring release.

What is the purpose of a post mortem?  Simply put, it's the time where
we measure ourselves against the goals set out before we started the
release.  We should try to learn from the occasions where we did not
succeed in meeting a goal.  We'll gain a lot by addressing them the
next go around.

I don't think we had any explict goals, so let's measure against some
goals we should have had:


* provide a quality release.

Most of the changes going into the release branch were either critical
nature or changes to the install infrastructure.  IMO, this means that
the release contents were being well managed.  It could have been 
better, the tty changes (which I think were made just before the
release branch was cut) should have been reverted.

However, as release branch gatekeeper, I feel that I had to do way too
much work.  I approved many more critical fixes than expected, some
very late in the release process.  This indicated that -current was
not really in that good shape before the release process began.

Some of the problems with late changes were due to other problems
listed below.


* provide a timely release.

We managed to get the release out in ~10 weeks from start to finish
instead of 8.  Some of the the delay was caused by:

	- sun-lamp going down

	- hardware problems with warped

	- network problems with warped

	- account problems with warped

	- release engineer taking vacation without having sufficent 
	  net access.

	- lots of last minute issues 

The hardware problems with lamp and warped is not something we could
do anything about.  It would have been better had we had replacement
hardware ready to bring on line at a moments notice, but we were not
in a position to do so.  We still are not in a position to do so, so
this could occur again.  Fortunately, warped seems to be running 
well.

The network problems were a similar case.  There was nothing we could
do to fix it.  We still hear problems from time to time about sup, ftp
not working.  

The account problem is where we should have acted instead of waited.
Since warped is Bill Coldwell's machine, I thought that he should be
the one to create all the NetBSD accounts (even though I had root and
could have done it myself).  However it took a long time to get every
one to submit an account creation request to Bill, and even longer for
accounts to be created.  In the end, I created many of the accounts
myself after all.  I should have explained that we were under strict
time constraints and asked permission to create all the needed
accounts.  

And bad things happen when the release engineer takes a vacation
expecting good net access and not having it.  Its clear that I should
have designated someone to fill in the void if I was unable to perform
my duties.  Sorry 'bout that folks.

There was a lot of little stuff like updating install notes that was
not done until the last minute.  For the next release, I'm going to
insist that the install notes be largely done at alpha release time.
I received some changes just hours before the release was to be
announced.  This is responsible for the second of the two delays.


	* keep developers informed.

Several people voiced complaints about not being sufficently informed
about release schedule stuff.  For example, sources were never tagged
which was expected (and even on the schedule).  I think we've become
better at communicating issues since the release.


	* keep users informed.

Users too were confused about lack of Alpha and Beta releases.  We got
lots of positive support when I announced the two schedule slips.  So
we get a mixed score on this.

The mirror maintainers needed to be kept informed about 1.1 too.  I
had no idea how to get ahold of them.   This should not be a problem
now that we have the mirrors database (not published yet, but I have
the raw data).


	* don't stall mainline development.

As far as I'm concerned, we accomplished this goal.  This is a benifit 
of short release cycle.


	* make process repeatable.

I started writing a release process manual.  It's still far from
complete.  I hope to make major contributions to it this during
the 1.2 release.


	* make NetBSD 1.1 the best release ever.

I've heard that NetBSD 1.1 is not as stable as 1.0, but I think it's
reasonable.  There were a lot of changes and over a year between 1.0
and 1.1, which made it more difficult to make a release.  This was 
the first release for most of the participants. 

I think we're a bit more cognzent of keeping -current in good shape,
and we have recent experience making releases.  These two facts should
help us accomplish the goal of making NetBSD 1.2 the best release ever.

	--jtc