

The proposed research does not easily fit into one of these 
categories.  Borrows from ethnomethodology, but also will 
also quantitative analyses as well.  Goal is not to predict 
and control, through creation and testing of hypotheses, but 
to observe, and identify general patterns that can be generalized
to other situations.

Triangulate:  take different kinds of data from different 
populations and see if the results converge.  Use "thick description"
as well as "thin" description.

Concerns:  generalization:  goal is not to find a distribution 
and a predictable pattern, which can be generalized from a small
sample to a larger population.  Instead, looking for patterns of 
behavior 

Have anecdotal evidence of this kind of sharing and that its hard
for people to customize -- this should provide a systematic analysis
to see if the phenomenon exists.  If it does, it should provide
clues as to how to improve things -- but only if the entire 
context is taken into account.

Reliability:  Not doing controlled experiments -- want to know if 
people will say the same things from one time to the next.  Answer --
no, because the context affects the answers.  

@subsection[Experimental Psychology]

We draw our research theories and methods from other, more established
fields.  However, this field has a number of very different fields to
draw from.  Experimental Psychology is modeled after other branches of
natural science in which the goal is to predict and control behavior.
See @cite[sidman] for an excellent summary of the major themes of
replication, variability, and experimental design and how they are
influenced by the forms of data being analyzed.  A typical 
experimental procedure would involve the following steps:

@skip[1] @begin[enumerate, spacing 1.5]
Observe a phenomenon that does not fit with the currently-held theory.

Develop a hypothesis to explain the phenomenon.

Create an experimental procedure that differentiates between 
the experimental hypothesis and a "null" hypothesis.

Attempt to disprove the null hypothesis in an experiment.  Control for
all variables except the one being tested.

If the data do not support the null hypothesis, the experiment has
supported (but not proved) the experimental hypothesis.

Replicate the findings, either through direct replication with a
different subject population or through systematic replication with
new but related experiments.  
@end[enumerate]

The important principles here are the notions of comparison among
alternatives and the notion of control.  Also notions 
of reliability and validity.

what are the acceptible forms of justifaction? -- are we simply 
observing?  Are we testing hypotheses?  

@subsection[Computer Science]

In computer science, the strategy is more likely to be: develop an
idea or algorithm, develop a system that embodies the idea,
and then test the system.  It should solve a previously
unsolved problem, be better (faster, smaller, etc.) than
other systems, or involve a new way of defining problems.
Sometimes just the design is sufficient, at other times, 
the design and proof that it can be implemented is 
necessary.

@subsection[Economics]

Economists have a different approach.  They develop theories and
mathematical models that can be tested by examining historical data.
They create operational definitions to measure the data.  


@subsection[Organizational Behavior/Ethnography]

Observe behavior of people in the field.  Identify phenomena
or principals, test by observing them elsewhere.

Ethnographic studies:  (Anthropological influence)

Go into an environment with an "open" mind, and observe.   Draw
conclusions and check for additional instance.  Assume a bias.


In each case, we need to abstract from the complexity of the real
world in order to successfully approach problems.  The nature of the
question and the form of the data available both affect the research
strategy.  Of course, the traditional research strategies in each
field affect future research methodologies.


Rather than assume that the goal is to "control" for organizational or
technological change, the research methodology must include these
changes as part of the phenomenon to be studied.

I propose to conduct an ethnographic-style study, in which I observe
individual behavior from within the organization context.  I will also
collect additional data, in the form of answers to a questionnaire and
records of actual use and modification of customization files.


Why is a questionnaire alone inappropriate?  Don't have a clear 
enough idea of the variables that are operative.  Need to talk 
to people.


Why is a lab study inappropriate?

Some phenomena can be isolated in a laboratory and studied through
systematic comparisons between control and experimental conditions.
However, when all of the variables are dependent upon each other,
different techniques are required.  These phenomena are embedded in
the organizational and technical contexts in which they occur.
Laboratory studies, especially at this stage, are unlikely to be
useful because of the nature of control required: some of the elements
must be held constant while others vary.  Yet one of the claims of
this dissertation is that neither the technology or the behavior
patterns of members of an organization can be treated as independent
variables: each action is both influenced by and may influence both
the technology and the accepted norms of behavior.

This perspective does not obviate the need for careful comparison;
however it changes the basic research paradigm.  I will be influenced
by an ethnographic approach, which describes both the organizational
and technological contexts in which the behavior occurs.

The phenomena I am interested cannot be studied in the laboratory 
for two reasons.  First, they occur over significant periods of 
time.  Second, they are "situated" in the context.  The particular
behaviors of individuals are affected by external events, the 
organizational context, individual factors and ...

===Generalization=====

What about the problems of generalization?  This is not the issue of
generalizing from a small sample to a larger population.  Instead, the
idea is to identify certain phenomena that occur in organizations and
use these to explain problems... 

It is important to distinguish between theoretical generalization 
and generalization from a sample.  At a deep level, a phenomenon
can be generalized.  Thus, we can generalize to other users in 
similar settings.

People appropriate and customize, but they don't do it randomly -- 
it's somewhat "sticky".  They save time, etc. by doing things in 
more constrained ways.

===================

Many insights about the factors that influence a user's choices
about customization or sharing of customization files are quickly 
forgotten.  For example, a user may suddenly run into a problem
and think about fixing it.  At that point, the user may be aware 
of the factors that are relevant, but he or she will quickly 
forget.  A laboratory experiment might be able to examine such 
phenomenat, but is unlikely to identify them.  Questionnaires 
will not do it...  Must watch users as they work and observe.

It's important to understand the technological environment as well
as the organizational environment.  I have conducted many interviews
over the past four years and have learned to use the technology 
myself.  However, this is not just my experiences at MIT.

===================


If it were possible to treat either the techology or the organizations
that use technology as static, independent variables, then it would be
possible to construct research questions that measured the effects of
the independent variable on one or more dependent variables.  However,
since one of the phenomena of interest is how these variables affect
each other, a different research paradigm is required.  As discussed
earlier, researchers in the field of information technology are
struggling with developing a research paradigm (in Kuhn's (1970)
@citemark[kuhn] sense of the term).  As an interdisciplinary field, we
can draw upon several existing paradigms from Experimental Psychology,
Computer Science, Economics, and Organizational Behavior.

Reductionist perspectives from the natural sciences cannot
simply be used as models for this type of research.  The basic
conditions under which research is done, i.e. that the independent
variable is held constant and the experimental variables are
controlled systematically, is rarely appropriate.  Because technology
is made by people and changes through use, it cannot be studied
independently of human action. [Some fields in natural science are
struggling with a similar paradigm shift.]

What are the basic assumptions in a reductionist research 
perspective?  That the whole is equal to the sum of the parts,
no more and no less.  That the individual variables can 
be held constant for the term of the experiment 

When these assumptions hold, and when the questions match, then 
the research techniques of experimental science can be 
extremely productive.  However, if the assumptions no longer 
hold or if the questions of interest differ, it is a mistake 
to blindly apply these research methodologies.  If we have 
biases in our observations, then we should include them, not 
ignore them.

===============
