Message-Id: <9105160738.AA14699@baby.bby.oz.au>
To: bug-at@ATHENA.MIT.EDU
Subject: Well, well, well, the redraw protocol works....
Date: Thu, 16 May 91 17:38:24 +1000
From: Gregory Bond <gnb@bby.oz.au>

Guys,

You haven't heard a real lot from me for the last few weeks.  I have
been a busy little vegemite and I finally have something to show for it.

Let me start by outlining why I have had to do what I have done.  Then
I will describe what I have done, and how well it works.  Then I will
describe what I have not done, then what is still on the "to-do" list.

I still have a number of questions, so if you bear with me to the
end...

Why I Did It.
-------------

When we first got the Plotter widget set, I was most excited.  It did
about 90% of what we needed.  There were a few drawbacks, though:

1) It was written in ANSI C.  We use the gcc compiler for development,
   but policy says all installed/delivered products must be compiled
   with the system compiler.  Fair enough too; we don't have the
   resources to properly support gcc.  The additional problem was that
   gcc couldn't generate shared library code, and we expected to be
   using this code a lot, so wanted shared libraries.

2) It was strongly directed towards "engineering" graphics (i.e. real
   vs. real).  We need more "business" graphics (i.e. data vs date).
   There was not enough flexability in the axis code to enable
   labelling by dates, at least not enough to do things like tics at
   the start of each month....  This was the real killer in our
   business environment.

3) All our existing applications stored the application data as an
   array of structs.  The plotter expected a "tree" of arrays of
   doubles.  It would have been very hard and wasteful to maintain two
   copies of the data.  (Not easy to re-cast the applications either
   as the rest of our libraries assume the array-of-structs model).
   Again, FORTRAN-like thinking from engineers!  (I can talk - I'm an
   engineer myself!)

4) It wasn't clear to me how to link multiple instances of the graph
   into one application so that common axes shared the same scale,
   min, max etc.  (It may have been possible, but it wasn't very well
   documented).

5) The refresh/redraw code was a real mess, with policy and
   implementation being strewn amongst all classes in a fairly
   haphazard manner.  There was no well-specified interface between
   plotter and axis and plotter and plot.  This made making changes a
   very dificult task.  In fact, the redrawing code had huge holes in
   it.  Often no redraw would be done when it was needed but far more
   often many redraws, each with expensive layout and scale
   operations, were required to change one simple element of the graph.
   Hence the refresh and update had a very annoying flicker.

What I Did
----------

I posted patches to fix a number of problems in the 4.0 release to
this list a few weeks ago.   What I am describing here is stuff I've
done since then.

We have had discussions on the redraw protocol earlier on this list.
That was after I had hit problem 5) above in the attempt to work out
how to solve 2).  After a few comments and a lot of thinking, I
decided to have a go at implementing this new redraw protocol.  One
thing led to another, and the short answer is I wound up rewriting the
core of the widget set.

So I have more-or-less new versions of the plotter code and the axis
code, and slightly hacked versions of the plot widget, the postscript
and text code necessary to go with all this.  Rather than hack on the
existing xyplot widget, I wrote a very very simple lineplot widget
that conforms to the new redraw protocol as well as offers support for
the array-of-structs data model.  I have not touched the xyplot, the
barchart, the countour plot or any of the subclasses thereof.  I have
not touched the textplot class either, but I suspect it would compile
and work with very minimal effort.

In the course of this rewriting, I did the following things, some of
which may be considered gratuitous changes:

1) Make it compilable with either K&R C or ANSI C, including full
   prototypes in all header files and using the fully prototyped Xt
   headers.  Rather than the fairly verbose method of compiling
   prototypes used in the Xt headers, I used this arrangement (stolen
   from I don't remember where):

	/* This bit in each header file */
	#ifndef P
	# ifdef __STDC__
	#  define P(args) args
	# else
	#  define P(args) ()
	# endif 
	#endif 

	/* This is how header files delcare routines */
	extern int AtAxisWidth P((AtAxisObject));

	/* This is how static functions are declared in each c file */
	static void funtion P((String arg1, int arg2));
	static void function(arg1, arg2)
		String arg1;
		int arg2;
	{
		/* Code of function */
	}


   This means that every function is fully prototyped.  The result is
   that it all compiled "gcc -Wall", including x header prototypes,
   with no errors or warnings (in our setup, which declares almost the
   entire C library in the stdlib.h file).

2) Changed the names of the structure members from the existing style
   ("ticInterval") to that recommended in the Xt manual (sec 1.6.1)
   ("tic_interval").  Gratuitous change, but it helped in the
   conversion process!!

3) Used AtText's to do axis numbering instead of the original string +
   XFont stuff. For consitency rather than anything else.  Chris (or
   anyone?) - Why did the original verion use the native X font stuff
   here and AtText everywhere else?  Design evolution?  End result is
   that the numberStyle resource has gone until AtTextCreate() or some
   variant supports a style parameter.

4) Chopped out a lot of unused crud - an amaazing amount!
   Unfortunately, I probably chopped out some potentially-used crud, all
   of which can be added back later if anyone is crying.

5) Implemented the new redraw protocol (and refined it quite a bit on
   the basis of the experience!).  All the logic for scaling, layout,
   drawing etc is collected in the one place, with clearly defined
   interfaces between objects.  It makes them all much easier to
   understand.  It is certainly _MUCH_ more efficient at redraws etc.

6) Changed parts of the interface to the plotter and the new LinePlot
   widget to use member functions rather than the SetValues stuff,
   mainly for the sort of things that have to be done in C anyway
   (i.e. pointers to functions and data).  This sort of change will
   mainly affect the plot widgets I haven't touched yet.

6) Used the region for handling exposure events to cut down on flicker.

7) Added a few new features, e.g. automatic horizontal sizing of the
   legend; more flexible handling of axis tics and labels;

8) Fixed a number of bugs/oversights, especially in the SetValues
   handling (most of the resources wouldn't force a redraw if they
   changed as the SetValues routines ignored them).

9) A registration function to register the relevent bits with WCL (the
   Widget creation Library -- see comp.sources.x Vol 11, Issues 13ff),
   and a new test program that is built using WCL.  This test routine
   can be very quickly changed and allows rapid testing of just about
   everything.  (It currently has 10 resources it can alter, adding
   new ones is very easy).

10) A new Imakefile for building shared libraries and the new test
    code, including strcasecmp for old C libs and the Wcl stuff.

What I Have Tested:
-------------------

The basic stuff all works beautifuly.  Layout, scaling, resize,
redraw, expose etc all happen very quickly and without flicker.  The
prostcript equivalent looks great also.

All combinations that I can think of of:
- Y and / or Y2 axis displayed
- grids
- endpoint rounding (i.e. making min/max a multiple of ticInterval)
- framed axes
- Changing colors of plots
- Changing colors of axes
- showLegend
- numberInside
- ticsInside && ticsOutside
- User tic labels
- date_labels (a new feature....)

What I Have Implemented But Not Really Tested:
----------------------------------------------

- The forceSquare and forceAspect logic.
- Log scales (in fact, this currently fails....)
- The fast update mode on the LinePlot widget.
- Copying axis tics and scales to another graph.

What I Have Not Implemented:
----------------------------

- The other plot widgets. This is pretty low on my list at this point
  due to a combination of the array vs. structure problem and the
  scientific vs business use.  I will be looking at the barchart
  fairly soon however.

- Textplot label-this-point-on-the-graph widget.  Should more-or-less
  compile and run.

- The layout and rescale callbacks that were described (in a fairly
  hand-waving sort of way) on the redraw document I posted a few weeks
  ago.

- Functions for applications (as opposed to plot or axis objects) to
  interact with the plotter, as sort-of hinted at in the redraw
  document.

----------------------------------------------------------------

Now some questions:


Firstly, I could use a volunteer anon-ftp site to hold an alpha
release. I can put a tarred compressed file of all the things I have
changed, including the redraw protocol document updated to the current
reality, into someone's ftp area if there is a volunteer to receive
it. (I don't have ftp on my machine, and neither of the machines where
I have guest accounts with ftp have anon-ftp set up).  Unfortunatly, I
haven't updated any manual pages yet, but I may have done that by the
time I get an alpha release ready.

Secondly, Chris: When I first dicussed this you mentioned the
copyright on each file was the wrong one (i.e. was the kerberos one
not the X one).  Send me the new one and I'll stick that in before I
send it all off.

Thirdly: Coding styles/formatting; I have used the GNU c-style from
emacs' electric-c for most of it.  The original was in some other
similar format.  If there is an official c-style for Athena code I
will gladly use it if someone can send it to me.

Fourthly: I guess, I should ask if anyone else really is interested in
looking at this and (perhaps) working on converting the remaining plot
widgets.  I welcome all your comments.

Fifthly: If there is interest in what I have done, do you (plural)
want to make this the basis for an "official" new release (version 5?)
or a parallel variant (that has already diverged pretty
significantly).  Apart from the unimplemented stuff, I have not put
great effort into backwards compatability at either the resource level
or the program interface level (as we don't have anything depending on
it!) so the v4->v5 upgrade would be non-trivial (but probably not too
big a job).

IMnsHO the code has benefitted an awful lot from being under just one
hand for a while, as opposed to the somewhat organic,
undergrad-project status it seems to have had earlier.  I am actually
being paid to work on this, and will have resources to put in for the
next month or so, provided I can convince the Powers That Be that the
work I am doing is useful to us.  Given that, I am more than happy to
share the results of my labor with the world in one form or another;
especially if I can get help in return!

I await your wisdom, criticism, observations, praise, Nobel Prizes,
offers of help, etc.  Even flames!

Greg.