17.309/ESD.082/STS.082  Science, Tech, & Public Policy

If you have received permission to take a conflict exam or make up exam at any time other than Monday 12/16 1:30-4:30 PM, please email oye@mit.edu with details.  

There is at least one conflict exam scheduled for E53-485 at 9 AM on Monday 12/16. 

Q1 ADVERSARIAL VS TECHNOCRATIC :  Do you favor adversarial or ostensibly neutral technocratic approaches for assessing knowledge under uncertainty and controversy?  As an alternative, specify the conditions under which you would go with adversarial approaches and the conditions under which you would go with technocratic approaches to knowledge assessment.  Please develop your argument with at least two examples.

BONUS:  Discuss the emotional appeal of technocratic approaches and the emotional distaste for adversarial approaches among your classmates.  Is a commitment to reliance on expert systems of evaluation naïve or admirable or both?

 

Q: With reference to the second sentence, what is the alternative?

A:  The first sentence asks you to accept a dichotomy and select either an adversarial or technocratic approach.  The second sentence lays out one alternative -- specifying the conditions under which you would favor a technocratic approach and the conditions under which you would favor an adversarial approach.  You are free to develop other alternatives, such as  defining an approach combining features of technocratic and adversarial approaches.

 

Q: Do examples have to come from class?

A:  You are welcome to use examples that we did not treat in class. If you do so, please provide enough information for the readers of the exam to understand the example.

 

Q: What about the bonus?  Should I write an integrated answer that addresses both the main question and the bonus?  Or should I keep the main response and the bonus separate?

A:  Please answer the main question first, then address the bonus.  You may find that the points raised in the bonus are of relevance to your main essay.  If so, you may wish to reference your separate bonus answer in your main answer.

 

Q: Why emotional appeal matters?

A: How many of you favor technocratic approaches? 

Q: We are engineers.

A:  Would the ratio favoring technocratic approaches be the same at the Kennedy School or a law school?  Why is the ratio here different?   Are these differences grounded in reflexive biases or reasoned positions?   This bonus question exists is to encourage reflection as you develop responses to this question as a whole.  

Q: Does the question ask me to reveal my biases?

A: The bonus does not ask for your position.  It asks you to describe and explain the appeal of technocratic approaches among your classmates.  

 

 

 

Q2  THE BEST AND THE WORST : Choose a policy area* with significant scientific content that is also marked by notable variation in the quality of policy.  Choose examples of the best and worst in this policy area, and then write an essay with attention to the following:

A. Explain why you selected these cases as the best and worst.  What are your criteria for evaluating cases and how do the cases map into your criteria.   Define your standards for evaluating performance and apply them in these two cases?

B. What factors contributed to success and failure of outcomes in these two cases?

C. Would changes in knowledge assessment institutions and practices or other political institutions and processes have improved performance in the “worst” case?  

*Note: Please clearly specify your policy area (ie. pharmaceuticals, defense, environment, energy, consumer product safety, or another area of your choice) and limit the scope of your evaluation to that area. In other words, favor depth over breadth in this essay.

 

 

Q: With reference to A, what is the difference between the two questions that are posed?

A:  The second question was intended to reinforce the first question – not to raise new issues.   One of the TAs recommended adding this language to clarify. 

 

Q:  So the core of part A is to specify evaluative criteria?  Not to specify examples?

A:   The question asks you to specify evaluative criteria and to apply them in two examples within a policy area.

 

Q:  Should the best and the worst come from the same policy area?  Not from different areas?

A:  Yes.  The wording introduces a constraint by asking for consideration of good and bad outcomes within a single policy area, and then to figure out what is driving differences in outcomes within that area.

 

Q:  With reference to C, what should we do if there no reform was attempted in the examples?

A:  The question does not assume that a reform was attempted.   It raises a counterfactual on potential changes that might offer a plausible prospect for a better outcome.

 

 

 

Q3 SUPPLY OF KNOWLEDGE ASSESSMENT :  Please prepare responses to all options below.  Only one option will appear on the exam.

Option A . Diogenes Associates is a new international consulting firm founded to provide credible and honest assessments of knowledge on scientific and technical issues in areas of controversy.  The partners are struggling (earnestly) over several critical strategic issues.
        Define market niche -- assess potential customer base, in  the US, in another country, or in several countries;
        Define guidelines on what topics the firm should and should not take on;
        Define  rules on how to fund assessments;
        Set  procedures on how to conduct assessments; and
        Define personnel needs and recruiting appropriate analysts.

Write an advisory memo to the Diogenes partners, supporting your points with examples drawn from this course or other sources.   
Option B. Prevaricor Associates is a new international consulting firm founded to provide credible assessments-made-to-order of knowledge on scientific and technical issues in areas of controversy.  Prevaricor quietly assures customers that its assessments will fit customer preferences.  The partners are struggling (cynically) over several critical strategic issues.
        Define market niche -- assess potential customer base, in  the US, in another country, or in several countries;
        Define guidelines on what topics the firm should and should not take on;
        Define rules on how to fund assessments;
        Set  procedures on how to conduct assessments; and
        Define personnel needs and recruiting appropriate analysts.
Write an advisory memo to Prevaricor partners, taking care to support your points with examples drawn from this course or other sources.   
Option C. McCray Associates has been commissioned by the White House OSTP to perform three tasks:  (a) to analyze the long term viability of Diogenes relative to Prevaricor given current executive, legislative and judicial knowledge assessment practices; (b) to suggest changes in public policies to improve the long term viability of firms like Diogenes relative to firms like Prevaricor, and (c) to identify ways of forming a coalition on behalf of such reforms.  Please write a memo for McCray Associates addressing these issues with appropriate examples to support your points.

 

Q: Who is Diogenes? What is Prevaricor?

A: Searcher of truth.  Diogenes sought the last honest man.  Prevaricor is based on the root of prevarication. 

 

Q: What is the connection across these options? 

A:  You could address option A or B without answering the other options.  You cannot do option C without doing A and B.  Note that option C asks for first, long term viability of the firms, second, how would you change policy to improve Diogenes prospects, three, how to build a coalition that would support it.

 

Q: Do these fictional firms have real analogs? 

A: I can think of organizations that had the characteristics of Diogenes, such as OTA, that used to serve the Congress.  What happened?  It is gone.  The life expectancy is shorter than I would like.  Some firms look like Prevaricor.  Many live around the DC beltway and provide made to order assessments.  The question addresses long term viability. 

 

Q: I am not from the US…..

A: If you would like to address these issues in a non-US context, go right ahead.  We would love to see what you produce.  

 

 

 

Q4 : WHISTLE BLOWING AND KNOWLEDGE ASSESSMEN T : Defend one of the positions below, supporting your view with examples from class cases or other sources .  Alternately, develop and defend your own position.  

 

“A market economy rests on informed consumers and a democratic polity rests on informed citizens and legislators. Private sector employees have a duty to inform journalists, regulators and publics of evidence contradicting assurances of product safety.  Government employees have a duty to provide journalists, publics, inspectors general, and legislative committees with evidence that contradicts false official representations of environmental, health and security risks.  These duties trump confidentiality pledges and classification systems.”          

 

“Whistle-blowing erodes the confidentiality and privacy necessary for societal, political, military, and economic institutions to function.  This is even more the case in the digital era, where low barriers to information access and an ideology of transparency result in too much whistle-blowing and not enough attention to less damaging channels that could resolve the problem.  Societies should be looking for solutions that permit responsible disclosure at a level to be determined by governmental authorities and preserve the confidentiality necessary for institutions to function effectively.”    

 

Q: Is whistle-blowing limited to disclosures to the media?

A: No. Whistle-blowing can involve going to prosecutors, committees, or even up the ladder. The question can be answered well with reference to restricted or expansive definitions of whistle-blowing. 

A: This question emerged from discussion among the teaching staff.  Thisis what came up at end of week.  The positions are not fictional.

 

Q:  The second position is more nuanced.  The first part seems like you cannot change parameters, the second part seems as if you could change many.  I like the first position but would like to discuss variations on that theme.

A:  This exam permits justification of principled modifications of questions and this might be such a case.

 

Q: Can we reformulate the sentence on pledges and classification? I would like to get rid of it.

A:  Typically, when someone goes against their company or government line, they are acting in violation of an explicit contract or implicit pledge.   So please do not eliminate this element of the question just to make life easier.  

 

Q: Does this question just apply to private firms?  It also references societal, economic, and military settings. 

A: This question explicitly asks you to consider a broader range of settings because you may not reach the same conclusions in each.

 

Q:  I do not understand what is meant by less damaging channels.

A:  Are there formal or informal mechanisms that the whistleblower could have used to correc the record and get information out?  For example, could Snowden have realistically gone to alternative channels?

 

 

Q5  PERFORMANCE INDICATORS : The Reiner chapter and lectures considered three classes of indicators of performance:

·         Behavioral – concrete actions that facilitate assigning obligations to specific actors for specific actions

·         Intervening  – variables that may sit on causal paths between actions and objectives in the process under examination

·         Consummatory –  goals defined in terms of proxies for ultimate values

These indicators of performance are often selected without adequate attention to underlying causal structures.  As a consequence, “improvements” in terms of measures of success often come at the expense of perverse selection effects and/or inefficiencies.

A Provide an example of an issue area where a set of performance indicators appears to elicit exceptionally perverse effects.  Then lay out mechanisms linking indicators, incentives and effects in the case you have selected

B Define a set of performance indicators that should work better in your cases, then lay out mechanisms linking your revised indicators, incentives and effects

C. Discuss potential sources of support and opposition for redefining indicators in the manner that you have recommended.

D. What knowledge assessment problems may be associated with your choice of indicators?  How might these problems be addressed?

 

 

Q: SAT scores were used as an intervening indicator in the slides, but we've heard other example where they are consummatory indicators. What type of indicator is an SAT score?

A: SAT scores can serve as intervening variables or as proxies for consummatory variables depending on what you are evaluating. Focus on the air pollution examples (below) if you seek examples of more concrete indicators. 

Q: Would you provide examples of behavioral, intervening and consummatory indicators? 

A: The air pollution examples work well.   Installing a catalyst is behavioral.    Human health is consummatory.  Emissions of nitrogen oxide from tailpipe and ambient ozone levels are intervening variables that are presumed to link behavior to ultimate outcomes.   

 

 

Q6 LEARNING AND ADAPTATION: Public policies may be viewed as experimentation under conditions of uncertainty.  Ideally, decision systems should recognize emerging information on the consequences of policies and incorporate evolving scientific and technical understandings.  Ideally, policies should adapt to emerging information.  Unfortunately, governments tend to lock into positions. Incentives for harvesting information after positions are formed are weak, and incentives for using newly harvested information to correct and adapt are even weaker. 

a. What are key sources of lock-in?  Please illustrate with relevant cases from this course, or other examples as appropriate.

b. What changes in institutional arrangements or processes would you suggest to improve the adaptive capacity of government? How may incentives for surfacing and using information be improved? Please illustrate with relevant cases from this course, or other examples as appropriate.

BONUS: If a little more adaptation would be good, then more adaptation and self correction would be even better…… or would it?   Discuss problems that would emerge if policy systems were to become highly sensitive and highly adaptive. 

 

 

Q:  Should we focus on regulations or statutes? 

A:  Should you focus on statutes regardless of implementation?  Or regulations that have interpretations that vary widely?  You have statutes that are fixed, you have regulations that are rigid, but in practice behavior may be responsive to context and modify over time.  In practice, the changes in behavior can be seen as adaptive.  This is what Jesse would refer to as informal processes.  Do not be too fixated on black letter law, you want to see how the system really behaves.

 

Q: Should we focus on regulations or statutes?

A:  Are you talking about laws, regulation, or behavior?  Do not make the mistake of assuming that just because the law doesn’t change, there isn’t adaptation within behavior through observations and changes.  The difference is that the focus on laws and statutes can put blinders on.

A: The magic word is discretion.  When evaluating rules, many analysts (the language here being drawn from legal theorists) look to the spirit of the rule to evaluate what the framers intended.  If one evaluates only the black letter, then the rule appears rigid and fixed, hence the blinders.  Rather, many rules include subtle points of discretion, many identified in course of application, that accommodate a variety of unanticipated behaviors---how that discretion is leveraged may be a form of intended adaptation, a consequence unintended by either the framers or those applying the rule, or may be a perverse abuse of the spirit of the rule such as gaming.  The issue then becomes how one uses the information at hand to evaluate the range of behaviors admitted by different interpretations to improve our understanding of whether a rule will yield beneficial adaptations or is susceptible to unintended consequences.

Q: What does "...incentives for surfacing information..." mean? 

A:  "....cncentives to reveal rather than conceal information...."

GENERAL QUESTIONS

Q: Would it be in our interest to answer the questions that have a bonus to maximize the number of possible points we can get?

A: Yes, answering the bonus questions is more advantageous than answering non-bonus questions.

Optional review sessions will be held:

Wednesday 7:30 PM E51-335 (Ken and others)

Thursday 1:00 PM E17-133 (Nicholas)

Friday 11 AM E17-136 (Jesse)

Friday 3 PM E17-136 (Amanda). 

Students are not limited to their TA and are free to attend any or all sessions. 

END OF TERM SCHEDULE

Sec Tu-W    Reconciling Secrecy-Accuracy-Honesty-Credibility?

L25 W 12/04 Jim Walsh: Covering Fukushima; Designing Fuel Cycle for Security

Ken Oye: Overview and Conclusions on Knowledge Assessment

    W 12/04 Ken Oye open office hour 3-4

    F 12/06 Final knowledge assessment paper due

L26 M 12/09 Evaluating Policy: Indicators of Performance

Sec Tu-W    ID best and worst cases? ID sources of failure?

L27 W 12/11 Improving Policy: Prediction, Adaptation, Selection

    W 12/11 Final Exam Essay Pool Posted

    W 12/11 TPP Holiday Party 3-5 E40-380 / Ken Oye open office hour 5-5:45  

    W 12/11 Optional Review on Essay Pool 7:30-8:30 E51-335

    M 12/16 Final Examination 1:30-4:30 E25-111

Jim Walsh will be joining us for our Wednesday 12/4 lecture.  He will be speaking on two topics -- on covering Fukushima for CNN and on the design and demonstration of a proliferation resistance nuclear fuel cycle.   Monday class will treat other security related issues.