Ideas...

=============

The standard model of software development includes a process in which
an "architect" designs a functional specification of the final product
and a team of engineers implements the design.  Testing consists of
checking to see how well the software meets the specified criteria.
Experience has shown (Myth man month) that this is unrealistic, 
particularly if the software being designed will be used by people 
in the course of their work.  In these cases, the interface 


Some software is designed to be adaptive -- in that we feed data 
to it and it changes accordingly.  However, it is not enough 
to say that this software is adaptive.

I am interested in a dynamic view of how people and technologies
work together.


===============

Provide evidence that users are affected by the software they use:
	LISP vs C vs small talk -- the software you use defines the 
		limits of your world.  What's constrained.
	users actively interact with the software they use
	influence of previous text editors  teco, edt, vi, wps  (modes)
	spreadsheets vs financial templates
	pc programmers vs dist. net programmers

Provide evidence that users change the software they use:
	zephyr modifications
	lens rules modifications
	muse modifications
	choice of when to use email

Provide evidence that the process continues.  
	Example of email -- Common Lens -- email -- Argus

Does the process stabilize or keep evolving?  It should depend on 
	the ongoing changes in:  the users, the software, the 
	dynamics between the users and the software.

What about the role of individuals in all this?  Different people use 
	the system in different ways, providing a complex net of 
	usage patterns that affects each other.

Why electronic communication?  Why is it co-adaptive?
	Users are influenced by the technologies made available to them.
	(show changes due to introduction of: FAX, voice mail, 
	zephyr, email)
	Users change the technology (cite interesting changes --
	both by end users and by programmers).  Note the interesting 
	possibilities of studying a person who is an intermediary,
	e.g. Evelyn and Muse, an active interpreter of the system.

Field studies:

	Athena -- look at zephyr modifications.  What have programmers
	done to incorporate new views?  (example of shared backgrounds --
	found a security hole too).  lots of serendipity.

	Look at email use at different places -- especially look at 
	something like All-In-One, which is designed to be flexible,
	but will look different in different organizations.

What other systems are likely to be co-adaptive:

	Ones with users who perform different tasks, that change over 
	time, whose tasks are influenced by the tools provided to them,
	and who have different ways to use the tools.  

	Management information systems
	Distributed educational software
	Financial systems
	Logistics and job tracking systems


Remaining questions:

	Is any of it predictable?
	Is it chaotic?  or will it reach a steady state?
	Is a co-adaptive system better suited to handle change?
	  If yes, how can it be justified?
	Can we measure co-adaptivity?
	  How?
	What are the effects of individual differences?
	

Experiments:

	Given a programmer and a user.  
	Guess that if they have very similar backgrounds and world views
	  they will do very similar things.  If there's no overlap,
	  the user will end up confused and in "survival mode".  If 
	  there's some overlap (especially between a theoretical 
	  push and an applications push), then some interesting 
	  innovations should occur.  Tie this to the reasoning behind
	  my model of the research group.  What kinds of innovations 
	  did they come up with?  What's the control/comparison?


Studying innovations:  the study of rare events (poisson distribution).



==================


Relate to binding constraints in operations research -- through 
	use, the effect of the software changes, because different
	constraints operate.

Is this a chaotic system?  In other words, do small changes create 
	large, unpredictable changes later.  Can we predict when 
	such changes would occur?


====================

Pet topics to incorporate -- where do they fit and why?

1.  Integrated communication system diagram -- with a discussion of
the other possible dimensions. 

2.  Educational "information access" diagram

3.  Editor tutor experiment

4.  Design of the R&D group -- a method for providing comparison, 
	a way of handling the concept of control.

5.  Pygmalion design

6.  EVA design

7.  Video prototyping design

8.  Email study

9.  Lens study

10. Argus design

11. Generalized oz

12. Carrie study:  choices of when to use which comm. channel

13. Anecdotes about user innovations

14. Anecdotes about changing user context

15. Anecdotes about repair theory

16. Anecdotes about email and SIGCHI

Categorize as:

design process
research process
interactivity
role of networks

==========================

Theoretical Models/Frameworks:

Co-adaptive Systems: claims that an organization's use of software
changes over time, because people both adapt to the existing set of
software, learning "what it is" from others and from experience with
the software itself, and they modify the software, sometimes in minor
ways (many individual differences in choices and styles of use) and
sometimes in major ways (completely using the software in an
unexpected way or changing it directly).  This challenges traditional
software development approaches, which treat the delivery of software
as a fixed event, and supports user interface development techniques
and suggests that they be incorporated into the system directly, as
part of a feedback loop.

Learning/Information access:  claims that users have different 
requirements for obtaining information at different times, and 
that different information delivery mechanisms will be more or 
less appropriate according to the changing state of the user.

Integrated communication system:  claims that users choose one 
channel over another based on a set of factors, which can be 
kept separate, and that choices are often suboptimal because 
the currently available systems have collections of features 
that were due to historical accident, not general usefulness.
If users could choose the optimal set of features, the overall
effectiveness of the system should be improved, and we should 
see some innovations as well.

Dynamic response to information:  tools to support user's management 
of electronic information will be better if they allow the user 
to take into account the user's changing relationship to the 
information.

Software development

Argus -- generalized message annotator and filter.

Pygmalion -- prototype multi-media message system, with distributed
video and smooth integration of public and private messages.

Common Lens -- multiple rulesets.  Boring rule.


Research Tools

EVA -- research tool to facilitate evaluation and analysis of video
data.  

Generalized Oz

Video prototyping


Organizational Design

R&D Group


Studies

Tutor

Diversity

Lens rules

communication channels

[Others:  lens rules by individuals, Argus study, case studies]

=============================================

@comment[Old Abstract]


Distributed networks of computers have provided new channels for
communication within organizations.  @comment[new opportunities for a
variety of different kinds of communication within and among
organizations.]  Over the past few years, numerous software packages
have been created to provide different kinds of functionality to
users.  @comment[let users communicate with different features]
The collection of programs appears random, based on historical 
accident and preferences, yet together they provide users with 
a system of communication paths and provide a range of communication 
possibilities.  (I include the telephone, FAX and other forms 
of technologically supported communication devices.)

The purpose of this dissertation is to explore the characteristics
of such systems of communication packages and to claim that 
together, they represent an excellent example of a @i[co-adaptive]
system.  I will present a framework for discribing the 
development of such systems, methods for observing them,
and examples of software that encourages it.  

@begin[enumerate]
What is a co-adaptive system?

Why is it good to have co-adaptive system?

Can we predict when they are likely to occur?

Can we encourage them?  If so, how?
@end[enumerate]

At the end of this thesis, I hope to have introduced and explored
the concept of co-adaptive systems, using electronic communication 
as a key example.  I want to use this to explore electronic 
communication itself -- how it changes from "off-line" communication 
-- both the content of what's said and the patterns of communication.
I want to create a framework for future electronic communication 
systems that permits the kinds of flexibility required for further 
communication and describe prototype software that has been developed
to demonstrate these ideas.  I would also like to propose a process
for observing, studying, developing and incorporating the 
"research process" into the development process (everyone's a 
scientist!).  Finally, I would like to take a meta-level view of 
this approach to research, which I believe is an important combination 
of techniques necessary to understand and improve the complex 
rapidly changing systems of today.  (ie. psych methods aren't 
sufficient -- need to reexamine the nature of control.)

Thus, this is a series of essays on the following topics:

@begin[enumerate]
electronic communication in the context of organizations
(observation, data, analysis)  (cost-benefit analysis --
who does what)

co-adaptive systems
(observation, data, analysis)
(what causes them to occur?)

a theoretical framework for electronic comm
(source:  examine real systems from a different perspective)

software to support the framework
(design decisions and feedback re: argus, pygmalion, tutor and its)

research methods for studying the software development process
(external observer, iterative design, general oz, eva)

incorporating the research methods into the process
(participant observer, feedback to users, video prototyping)

meta-level view of the research method
(reassess methods of control, combine lab, field, theory. Build,
test, observe, integrate)

@end[enumerate]

I am specifically interested in electronic communication for 
a variety of reasons.  One -- the Zuboff style notion of "informating".
Two -- related idea of empowering.  Three -- teaching.  Four --
increased innovation.  Five -- increase productivity.  Six -- 
increased work satisfaction.

Communication networks reflect the organizations -- very 
hierarchical ones look hierarchical.  (Do people copy every 
body up and down the line "Everyone is equal, all the way 
up and down the line" -- from a military person, the only 
one in favor of Winograd's software.

In this thesis, I claim that the organizations that used these 
nets co-opted from from their original purpose (rand study)
and that email and other stuff evolved over time.  Thus, we 
can use the history and lessons of email to examine this 
co-adaptive process and explore a few examples in depth.  then
extend it to other forms of systems in which the users and 
the technology influence each other -- propose a new model 
of describing the development of innovative technology in 
which ...


Various kinds of software have been developed to support this --
I am interested in how groups of users and individuals change 
the software (electronic communication) environment to meet their 
changing needs.  I call this a co-adaptive system -- one in 
which the user's behavior is influenced by the introduction of
new software, new features and new combinations of software and 
they simultaneously change the software.

Some of those changes are direct -- either the programs have 
features that users can change or some of the users are programmers
 or people devise clever ways of using other techniques to 
use the software.


(What about learning to program with TECO (me) -- to do data analysis ---
and SCRIBE (for steve to do his thesis).  Who learns to be a programmer?
(Any field - the common problem  for graduate students is to get 
"seduced by the computer").  What are the motivations -- rapid feedback 
and progress toward a self-defined goal.  Graphics/old computer games
drove first set of programmers.  But other flexible tools have seduced 
others.  Compare bank folks with others...

Difference betweeen software engineers and hardware engineers -- 
more like diff between scientists and engineers.  The former are 
driven by the project (hacking -- trying to test out a new theory), 
the latter are driven by getting the project out the door.  Andy's
data about the two types, with SW eng's significantly less interested 
in "getting product out the door".

I'll claim that the job is NOT to get the product out the door in 
software -- it's more of an on-going service with users modifying 
and manufacturers examining it.


Tools that support user productivity must take into account the
dynamic interaction between an individual user and the information and
how the value of the information changes over time.  ( rather than
just the content of the information.)

Networks of electronic mail and network news groups are an example
of a @i[co-adaptive system], in which the users change the system and
the system changes the user.  Other examples include: spreadsheets,
Management Information Systems, hypercard, Information Lens and
certain types of educational software.  Co-adaptive systems tend 
to reflect the organizations in which they are used and provide 
significant opportunities for user innovation, increased productivity
and enhanced adaptibility to change.

From the perspective of an individual, the relationship between 
an individual item of information (electronic or otherwise) and 
the person changes over time.  This means that tools designed to 
help users deal with information will be considered difficult or 
unsatisfactory if they cannot adapt to changes over time.

From the perspective of groups of users, the ways in which the
people use the system has a lot to do with critical mass and 
ability to share information.  Discovered that technical people 
develop "islands of expertise" -- akin to the gatekeeper concept,
which develops over time.  (Start out in an overload situation,
with too many messages, too many dl's, don't know what's important.
Over time, become interested in a few areas, learn who to ask 
when you need info about other things.  Email as a pointer to 
local experts.  Reduce info load, probably become known as an 
expert yourself, identifying certain areas and having other people 
forward things to you because they know you're interested.  Some 
people are information junkies -- others are much more focused.)

Dynamic nature:  how it affects individuals and the tools to support 
them.  How it affects organizations and how the organization adapts
to change.  Talk about Zuboff's "informated" organizations -- when 
feedback about becomes an important characteristic.

What makes people innovate:  provide feedback...

The value of a piece of information changes over time -- even if 
the content doesn't.  This relates to educational software 
and electronic communication in general.  It also says something 
about choice -- who's in control of the communication and the 
tools to control that communication.

In this thesis, I make a number of claims:


From the perspective of groups of users, the ways in which the
people use the system has a lot to do with critical mass and 
ability to share information.  Discovered that technical people 
develop "islands of expertise" -- akin to the gatekeeper concept,
which develops over time.  (Start out in an overload situation,
with too many messages, too many dl's, don't know what's important.
Over time, become interested in a few areas, learn who to ask 
when you need info about other things.  Email as a pointer to 
local experts.  Reduce info load, probably become known as an 
expert yourself, identifying certain areas and having other people 
forward things to you because they know you're interested.  Some 
people are information junkies -- others are much more focused.)

Dynamic nature:  how it affects individuals and the tools to support 
them.  How it affects organizations and how the organization adapts
to change.  Talk about Zuboff's "informated" organizations -- when 
feedback about becomes an important characteristic.

What makes people innovate:  provide feedback...

The value of a piece of information changes over time -- even if 
the content doesn't.  This relates to educational software 
and electronic communication in general.  It also says something 
about choice -- who's in control of the communication and the 
tools to control that communication.


===

The unifying theme is the dynamic relationship between people and
electronic information.  I am interested both in understanding how
people communicate in organizations and in improving the effectiveness
of the tools that support that communication.

I have created several research groups over the past ten years and
attacked the question from a number of angles.  We have studied
behavior in the field (e.g. studies of electronic mail use and how
field service technicians work) and in the lab (e.g. experimental
studies of electronic tutoring and educational software).  This work
has resulted in new research strategies (e.g. video prototyping and a
matrix approach to the design of applications and systems) and the
development of better tools for data collection and analysis (e.g.
Argus data collection, the Experimental Video Annotator (EVA) and the
generalized "Wizard of Oz").  This has lead to several theoretical
models that have been implemented in the form of software (e.g. Argus,
Pygmalion, Editor tutor, Navigation project, Interactive
trouble-shooting simulator, etc.) and tested in the field (e.g. Lens
and Argus studies).

An interesting aspect of electronic communication systems is that they
appear to be "co-adaptive".  The users of the system are both
influenced by it and actively involved in changing it over time.
Different organizations with the same software base may end up with
qualitatively different patterns of behavior and software environments
over time.  Some systems seem to be naturally co-adaptive, yet current
software development processes do not take this characteristic into
account.  I am interested in developing techniques for studying this
phenomenon and in creating processes for developing and encouraging
such systems.  I believe they present a method of encouraging
individual growth and fostering effective reactions to constantly
changing environments.

@begin[figure]
@begin[center]
@graphic(postscript=/mit/visual2/mackay/PS/coadapt.PS,
magnify=.75, BoundingBox = "2inch, 5.75inch, 5.75inch, 8.75inch ")
@comment[clock measurements:  9:00,    6:00,  3:00,     12:00]

@caption[The user both learns from the software and modifies it, 
iteratively adjusting the co-adaptive system over time.]
@end[center]
@tag[coadaptive]
@end[figure]


=============
Citing tutor work:

People are always making decisions about how to do something.  for 
example, if I have to do the same operation 3 times, I'll probably 
do it by hand.  If I have to do it 1,000 times, I'll probably take 
the time to figure out a "fast" or automated way to do it.  but 
what if I have to do it 10 times?  30 times?  100 times?  When do 
people decide to stop and "write a program" or do something special --
use a new command, etc.?  Usually, if people are worried about the 
task, they'll be more likely to do it the "tried and true" way  
because they're nervous about having it go wrong.  (Automated
procedures often go wrong.)  But if they are bored, or think they'll 
have to use this in the future, that will raise the incentive to 
try it the other way.    (When do you try it versus. ask someone vs???)

Note that these decisions differ from system to system.  A hotshot
C programmer may use a tiny fraction of the possible emacs commands.
A non-programmer may be a whiz at scribe.  What dictates this???

