Chapter 10
This chapter is a piece of work written 2010 mainly by my colleague Elisabeth Gulbrandsen with myself as a co-writer. The positions Elisabeth and I are taken are rooted in our experiences and mutual interests of needed transformative processes in contexts of academia, research policy and distributed knowledge production. I am most grateful to Elisabeth for letting me include this text in the book.
Scientific excellence has become the catchword for research politics, manifesting itself in the establishment of centres of excellence all over Europe as well as in the European Research Council. What is the meaning of this phenomenon? What will we find when trying to open up the black box of “scientific excellence”? The need to question evaluation processes seems intrusive, when we realize the increasing demands on knowledge production to be societally robust, not the least within the energy and environmental sector. How do we move towards a situation where scientific ‘excellence’ is automatically taken to include reflection and wider engagement concerning societal dimensions?
This chapter amalgamate a number of interrelated issues supporting the aim of this book.
Re-thinking Excellence; getting smart between the no longer and the not yet - comments on the convergence of knowledge and politics
Introduction
The article “Authority in Transformation”121 Elisabeth and I produced in the middle of the 90s, carries a message that has become more urgent with the passing of time. The quote from Sandra Harding’s edited volume The ‘Racial’ Economy of Science reminds us that
…the declining authority of the West to determine how the rest of the world shall live requires a rethinking of the past, present, and future of Western sciences and their technologies no less than other important Western institutions and practices. (Harding, 1993)
In a comment that we at the time “censured” into a footnote, we contended an understanding implying that we ‘in here’ have developed something valuable that we want to give to you ‘out there’, to be fairly widespread among Nordic feminist research. At a Nordic conference on ‘Women, Development and Environment’ in Oslo (autumn 1990) the Indian feminist / activist / researcher Vandana Shiva responded to our wanting to export our knowledge to Indian rural women by asking back “Who appointed you God?” As we find that it still is the scent of ‘technologies of hubris’ and not ‘technologies of humility’ – as figured by Sheila Jasanoff (2003) – that make us mark something as excellent research, we find it appropriate to keep writing the same article. Here we copy and paste the 2nd paragraph from “Authority in Transformation”.
We argue that it is high time we make a shift. A shift that may seem simple but, as our own ‘trying transformations’ tell us; it is certainly not easy. Time is ripe for us, as partakers in the modern research complexes, to develop a readiness to think and feel ourselves as part of the problem, and learn how to use our implicatedness as a resource for transformatory projects. This shift represents our headline challenge. In this article we aim to expand on this challenge by spelling out some of the motivations for and implications of the shift, as well as pointing to conditions for carrying it through.
We are still struggling with shift and with the learning. Being involved in more horizontal partnerships for learning and development is still quite challenging for expert systems in general, and perhaps especially so for academia, which have to shed its cloak of assumed neutrality and objectivity and create for itself a new role as a societal actor. This challenge is partly a consequence of research’s growing impact and “success”. Research is increasingly involved in every aspect of life. There can be no doubt that research plays a crucial role in the development of industry and commerce, it affects our decision-making processes, it colours our culture and steers the development of society. However, research and technology not only have an integrating effect on the development of society, research and technology are also always already integrated in the development of society in general. Or to put it another way, society also influences the processes of developing research and technology. Focusing on “society in science” will thus be at least as important as “science in(to) society”122.
Co-evolutionary approaches
At this day and age – following a mounting environmental and poverty crises – there is a lot more unease with presenting research as the solution than in the mid-1990s. The weft of science increases in everything that surrounds us and it is at the same time possible to ask: What is progress these days? And how to measure it?123 Increasingly open systems for knowledge production require a focus on the direct reality-producing effects of research – its “context of implication” (Nowotny et al. 2001). It is these features of the development that made Ulrich Beck query whether representative democracy is collapsing through development of the modern research complex as a separate policy area: Politics breaks out in a new and different way, beyond the reach of formal responsibilities and hierarchies. So we are looking for politics in the wrong place, with the wrong concepts, on the wrong floors, on the wrong pages of the daily newspapers (Beck, 1996, p. 24). We want to position our ambitions to promote more complex and integrated understandings of the relationship between research and society, in this grey area that Nowotny et al. (2001) ascribe to a dedifferentiation of the societal spheres of modernity.
The boundaries between politics and research are not straightforward and clear in a society that depends on research and knowledge. Nowadays it is even claimed that research and society are co-produced or co-evolve (see chapter 11), which is a long way from the simple, linear understanding of this relationship that has dominated research policy hitherto. Research is no longer merely a means to realise goals in other policy sectors. Research is becoming a policy sector in its own right. It is in the fields of technoscience (information and communication technology, bio- and gene technology and material technology) that scientists are most clearly pushing the boundaries between science and society, research and politics, thereby illuminating the obsolescence of a linear understanding (Gulbrandsen 2004).
‘Innovation system’ was one of the first concepts put forward as an interactive alternative to the linear model. Strategic research, post-normal science, triple helix, mode 2 and agora are other examples. The term innovation system is in widespread use in the Nordic countries. Finland is usually held up as the paradigmatic case because of its “founding” of the concept ‘national innovation system’ (NIS). Reijo Miettinen’s analysis of how the NIS have developed in Finland can also be called paradigmatic because of his focus on the role of the NIS as a mobilising metaphor (Miettinen 2002). Miettinen talks about a double development in that it has become a scientific term and a political term. He introduces and develops …an epistemology of transdiscursive terms that are simultaneously and interactively used both by scientific communities and in policymaking (p.17). We believe that this is a perspective that can provide our tranformatory efforts with better tools to process changes in the relationship between research and society or science and politics, to produce more substantial, complex and integrated understandings and images of this relationship. By exploring other figurations like mode 2, the agora, post-normal science and technoscience as transdiscursive terms, we might be able to improve our understanding of the convergence between research questions and policy questions.
Miettinen discusses the extent to which Nordic social democracy and its political culture predestines political decision-makers and researchers alike to apply technocratic and pseudoscientific interpretations of the concept of NIS. However, it does not have to be so. Miettinen argues for a more modest way of relating by emphasising reflexivity, learning processes and contextual knowledge production. This is an echo of Haraway’s situated knowledges (1988) and Jasanoff’s technologies of humility (2003), rather than seeking mastery and control, we should focus on collaboration with ambitions of developing modulations in the diminishing gap between variation and selection or between promotion and control/regulation (Rip, 2002a). This corresponds well with discussions in policy studies concerning how a “governance by design” mode of working needs to be supplemented by a “governance through dynamics” approach124.
The regime of collective experimentation
Even if the call for such co-evolutionary approach to science, technology and innovation (STI) is often heard, it seems hard to realise in practice and as culture. The so-called “regime of collective experimentation” suggested in an EC-report from an expert group on science and governance (Felt, 2007) is an articulation of this challenge. How to identify potentials for, how to design instruments for, how to promote, manage and evaluate productive interactions between “science and society” or between science, technology and the market? The report collects examples featuring the recent shift from the idea of centralized organization of innovation to explicit recognition of the importance of distributed and more diverse innovation. Referring to John Dewey’s conception of policy as collective experimentation, the authors of the report contend that: … the experimentation is now at the technological level as well(p. 26). This move is inspired by experiments with “open innovation” in the business sector, and connects to the range of suggestive figures from the history of science policy such as mandated science, strategic science, triple helix, mode 2, post-normal science, agora. Still, it seems hard for science as well as for policy organisations to see themselves as involved in governance and steering issues, as societal actors in more horizontal partnerships, as key players amongst other key players. How come?
Figurations associating co-production of science and society indicate that such intimate interaction between science and society can further more societally robust science and technology. We must strive to open up a ”reflective conversation with the situation” as Donald Schön phrased it in his influential work The Reflective Practicioner: How Professionals Think in Action (1983). This argument may still be felt to be provocating in some corners of research. How far into research will the arena for co-production extend? And what will the interaction be concerned with? Quality, relevance or both? To develop a role as co-player seems to be dependent on a mode of humility and acknowledgement of limits in singular positionalities, that can be hard to find. (Jasanoff, 2003, Felt, 2007) It might even be felt to be ”contraintuitive” for researchers to move away from a ”mastery and control” mode in order to ask for help and to open up for input from and collaboration with others.
Citizen scientists
Helga Nowotny, the central figure in European research policy and the former head of the European Research Advisory Board (EURAB), has for some time been calling for a greater degree of reciprocity in the relationship between research and society, arguing that this partnership presupposes more transparency concerning the processes involved in research and technology. Nowotny argues that it is the research system that need to be opened up, and she believes it is particularly important to be able to communicate “uncertainties, contradictions and contingencies” – everything that cannot be assured as “scientifically” proven and which therefore turns the spotlight on the idea of science / research / technology as based on neutral and to a certain extent “objective” knowledge processes. “A new kind of more mature partnership” needs to be developed, Nowotny claims, and this can only be achieved if the processes whereby research and technology are developed are opened up.
Science can no longer expect unconditional support on the part of society for whatever it wants to do, nor unconditional acceptance of its authority. Society will have to become more involved in understanding better how research actually functions and why it is important. (Nowotny 2005c)
The same tendency is also evident in the United Kingdom, one of the leading countries in Europe in terms of development of the dialogue between research and society. Here, focus is increasingly on the process of developing research and technology. This shift is described as “upstream”, and Demos positions the challenges thus in The Public Value of Science (Wilsdon, Wynne & Stilgoe 2005):
Those who see upstream engagement as a means of providing earlier and better predictions of risks and impacts are missing the point. It is not a matter of asking people, with whatever limited information they have at their disposal, to say what they think the effects of ill-defined innovations might be. Rather, it is about moving away from models of prediction and control, which are in any case likely to be flummoxed by the unpredictability of innovation, towards a richer public discussion about the visions, ends and purposes of science. The aim is to broaden the kinds of social influence that shape science and technology, and hold them to account. (p. 34)
“Upstream engagement” refers primarily to the reflexivity of the research and technology systems, according to Brian Wynne. The requirements that knowledge must be reflexive and societally robust will only continue to grow in the years to come.125 The conditions necessary to create a constructive dialogue with society seem to be rooted in the increased ability of the research system to open up and admit the limits of its knowledge. This is necessary in order for research to be able to invite collaboration with other social institutions. The same demand to be able to open up and acknowledge one’s limits also applies to interdisciplinary work. One of the main challenges facing efforts to nurture interaction between research and society relates to inviting other parties to participate in dialogue in ways that make it possible and interesting for them to be involved and engaged. This requires what a report from the EC (Markus, 2009) calls the development of “further skills” by researchers, as they must be able to explicate their premises, conditions of validity, uncertainties, areas of ignorance, values and conditions of applicability to certain contexts. Because “Involving publics …, can be more productive if not only the knowledge at the object level is presented and discussed, but also the related metaknowledge.”(p 14-15) Developing the dialogue with society thus requires major changes in expert systems in general and the research system in particular. One of the challenges lies in “bringing out the citizen in the researcher”. Wynne (in the preface to Weldon, 2004) points out that this kind of understanding is just hatching:
The only recently recognised challenges of two-way understanding between science and its publics, replacing one way understanding of science, are in their very earliest days. This is emphatically a long haul, of nurturing not merely policy shifts, but profound cultural change in such science fields, their policy and technological uses, and the assumption underpinning them… .The bottom line issue in the new climate of “public engagement” is not just seeking earnestly for ‘public inputs’ – preferences, values or knowledge. It is being encouraged, by public dialogues and questions among other things, to question the validity of our own scientific-institutional taken-for-granted assumptions and routines. (p. I)
If research has an impact on society and interacts with other research in ways that are not linear, it becomes necessary to address the legitimacy and responsibilities of research on a broader basis than merely through reference to the fact that public research grants are used and distributed by institutions and allocation mechanisms that follow strict internal quality requirements and professional norms. Helga Nowotny et al.(2001) stress that the dialogue with society must necessarily be an ongoing process:
That the authority of science in the future will have to be established in an ongoing process that needs to be worked out again and again in each concrete situation is the meaning of the somewhat aphoristic title of this final chapter of the book, that re-thinking science is not science re-thought. (p 249)
Challenging research quality and excellence
Interactive policy models entail changes in the concept of quality. Contributing to productive collaboration and co-production between science and society, becomes an important mark of quality:
Recent discussions of Mode-2 science (Gibbons et al. 1994; Nowotny et al. 2001) has pointed out that ways of producing technoscientific knowledge already extend well beyond the classical ’independent’ mode of basic science. Stronger roles of applications contexts and imaginations in the very production of knowledge, transdisciplinarity, and socially as well as epistemically extended peer-review are but a few elements which indicate much broader social involvement in how knowledge is produced and validated. This co-production of science and society changes the very meaning of notions like objectivity and rationality. (Felt, 2007, p. 77)
It no longer suffices only to identify thematic priorities or societal challenges ”upfront”. We must also explore how research processes can be developed as productive interactions between different actors, how to develop and how to evaluate them as societally robust processes (Voß et al, 2006). A more complex, dynamic and open understanding of the relations between science and society, asks for the development of new competencies and skills in the research system. The challenges are of an institutional as well as of an individual kind, and they seem to touch especially raw nerves, maybe because assessing the quality of research relates to heavy investments (institutionally as well as individually) in specific forms of rationality. Indicating that we may have some triple loop learning126 to do, can be provocative regarding our professional identities. At the same time, if we are not able to discuss and explore alternative figurations of quality, the recourse to traditional academic standards will be imminent.
One of the more promising attempts to meet these challenges is situated in the Netherlands, still being developed by a network involving several policy organisations127. Their approach is called Evaluating Research in Context (ERiC). The comprehensive method that they propose takes into account the fact that much current research is produced in a complex socio-economic context, in which demands are made by a variety of social actors. Moreover, research that addresses complex questions (for example aids, global warming, migration, renewable energy) is often multi-, inter- and/or transdisciplinary and is conducted in a context in which experts with different backgrounds, knowledge and expertise operate and different demands and interests have to be negotiated. This complexity requires a different approach to evaluation than traditional peer review that mainly emphasizes scientific excellence and relies on publications in high impact journals for its primary indicators. Since quality in the ERIC-approach is defined as a multidimensional concept which includes the expertise of stakeholders in different social domains, they elaborate on the concept of quality by looking at these different dimensions, distinguishing in each the modes of production and interaction of researchers and a variety of stakeholders. This is the how they present their approach to evaluation (Spaapen et al, 2007):
Evaluation is not the same as accounting and control; that is, the evaluation of output in terms of certain benchmarks and indicators. The method we propose aims to include a form of second order learning that also put the meaning of benchmarks and indicators that are used into question. It therefore stimulates not only first order but also second order learning processes by way of reflection, debate and ongoing iteration between goals and methods. (p. 29)
It is a major challenge – in changing times – that the models deeply inscribed in the statistical practices underpinning our monitoring and governance activities, are so hard make explicit and to put into play. The ERIC-network underlines the importance of paying attention not only to the input in research (people, money apparatus), and its output (publication and other products), but also to the ‘throughput’. By this they mean the processes to mediate with the environment, for example co-operation and strategic alliances. This implies discussions about the strategic positioning of a research program, thus giving deliberation about goals and public methods weight. Taken together, these principles form a program that combines some of the lessons of classical pragmatism (notably the anti-dualism) and new governance policy-techniques; especially the mechanism for co-ordination and co-operation, that share a focus on ‘learning processes’.(Spaapen et al, 2007, p. 29) We include a “conclusion” that they arrive at 28 pages later:
The above lead us to the conclusion that we are not looking for an instrument to evaluate a specific research group or a program, but a process of interaction. And we are not so much looking for indicators that can tell us how good or bad the ‘quality’ of the research is, but we are looking for indicators that can tell us whether the group succeeds in fulfilling its mission in a relevant context. (p 57)
As hinted at, the emerging acknowledgement of “unintended consequences” was stressed as motivating transformatory action and experimentation in STI. Another way of approaching this may be through the discussion initiated by Sandra Harding in her introductory chapter to The “Racial” Economy of Science; Toward a Democratic Future (1993). Here Harding re-interprets ‘scientific illiteracy” as pertaining not to “humanists or … the working classes”, but to “many scientists, policymakers, and other highly educated citizens”. She contends that: … most scientist are not in a position to evaluate in a maximally objective way important parts of the evidence that they use in arriving at their results of research, nor is the educated public provided with the information and skills it needs to detect such a problem. This happens because …elite science educations rarely expose students to systematic analyses of the social origins, traditions, meanings, practices, institutions, technologies, uses, and consequences of the natural sciences that ensure the fully historical character of the results of scientific research (p. 1). In her Reflections on Gender and Science book from 1985 Evelyn Keller comes close to a similar description of the challenges: Yet, while our sensitivity to the influences of social and political forces certainly has grown, our understanding of their actual impact on the production of scientific theory has not (p. 5). It was Keller, who some years later, contended that scientists had to supplement the assessment that “it works” with questioning “what it works at” as well as “how it could have worked differently” (Keller, 1992, pp. 74). Sandra Harding’s diagnosis relates to natural science, but is echoed by Brian Wynne’s concerning the social sciences in the Afterword to Governing at the Nanoscale from 2006:
The mode of social science presented here involves more than intellectual dimensions alone. It also involves learning new relationships and responsibilities, with ‘the public’, with the natural sciences and with policy. And it involves social sciences becoming actors in those worlds as well as commentators.
However, this leaves a continuing issue unresolved. If we are to engage in these more politically immersed relationships, and leave behind our well-bounded peer cultures, how are we to ensure that the knowledge we generate can claim validity? (p. 77)
If we want to move from “speaking truth to power to making sense together” as Robert Hoppe (1999) has suggested, it entails exploring how we can evaluate research and technology on the move – between the no longer and the not yet.
In “The Agora and the Role of Research Evaluation” (Frederiksen et al, 2003) the three authors from Copenhagen Business School, note that the evaluation of research is undergoing change and that they want to … investigate how recent societal developments – epitomized by the concept of the agora – influence research evaluations. The ‘agora’ here denotes co-evolutionary figuring of the relation between science and society. In summing up they contend that:
The trust in science has traditionally been and to a large degree continues to be based on institutions that are attached to the idea of an autonomous and disinterested science (universities and the peer review system). If science is to engage in the developing and changing relationships with society and face the financial interests and power games and at the same time retain the public’s trust, demands for a radical change of perspective and implementation of new methods or procedures in relation to the evaluation of scientific knowledge are unavoidable. (p. 166 – 1667)
The issue of stimulating and developing conflicting and contested perspectives, is also part of what Arie Rip (2003) find is important in realising socially robust science and technology through 4th generation research evaluation. It figures quite prominently in the section “Knowledge production and assessment” in the before mentioned Reflexive Governance for Sustainable Development (Voß et al, 2006), especially in the contributions by Katy Whitelegg and Marie Celine Loibl. They are more focused on the processes of knowledge production than on the assessment of it, but it can be argued on the basis of their texts, that production processes and assessment or evaluation, should be closely interlinked, reference also to the ERiC approach as well as to the weight placed on reflexivity or triple loop learning in the discussions we have referred to in this article. This is a point brought forward by Arie Rip in a report to EC, June 2002; “Challenges for Technology Foresight/Assessment and Governance”:
The key point, however, is to move away from a focus on our limited knowledge of the nature and extent of impacts (which will remain full of uncertainties) to the process by which they come about, starting with the here and now. The question of technological innovation and its impacts is a complex and real-time challenge for the actors. Prospective technology analysis must therefore also be “real time”, and formative (a term from evaluation studies, where real-time evaluation informs, and thus helps to form, subsequent reflection and action). Anticipating outcomes (including impacts of the technology on society) must be an ongoing concern, rather than ad hoc efforts to persuade a sponsor or regulator that the innovation journey can continue. (Rip, 2002b, p. 52)
In contexts of scares resources the quality issues in STI get explicit and easy to understand in e.g. research linked to income generating activities and solutions encountering fundamental needs for people and society like energy, food, water and communication possibilities. The robustness can be recognized on the ground by the stakeholders and people involved. Excellence in this context corresponds to concrete relevance, results and sustainability. Excellence in a Nordic, academic context corresponds more likely to peer reviewed publications and later e.g. proofs of concept to be piloted128.
Government and science as key players amongst other key players
Co-evolution in a non European context e.g. in an East African context make sense in a very explicit way where the mission of the national universities existing and the Governments coincide in the main objective of poverty reduction. When faculties of technology and engineering position themselves as relatively equal partners with entrepreneurs in society for development in crucial production sectors, we find examples on how the Governments, no matter if local, regional or national, see the relevance and join in co-evolution processes129.
Another indication that co-evolutionary and networking models are gaining ground can be found in the changes pertaining to so-called ELSI-research (Ethical, Legal, Societal Implications of new and emerging technologies). In the second phase of ELSI-research - being developed in US, UK, Canada, the Netherlands and Norway – ELSI was challenged to integrate its activities into technoscience, not to function as a way of outsourcing such concerns from the technological development processes ”proper”. The Netherlands, UK as well as the Nordic countries, have put much effort into inviting ‘society’ to speak back to ‘science’, of experimenting with different types of stakeholder involvement in order to establish the much sought for two-way dialogues and the productive interactions between science and society. The re-thinking of stakeholder involvement that we see e.g. in EC and UK, point out how the infamous ‘deficit model’ is simultaneously laid to rest and resurrected in these experiment. And they point towards a lesson, namely there seems to be a continuing failure of scientific and policy institutions to place their own science-policy institutional culture into the frame of dialogue, as a possible contributory element that hinders a genuine two-way dialogue. As Brian Wynne puts it, we are ‘hitting the notes, but missing the music’ failing to acknowledge the deeper challenges of opening up our institutions and assumptions to critical debate. The reflexive capacity to acknowledge that one’s framing of a problem is positioned and partial, and thus open to challenge from other perspectives, needs to be enhanced as well as assessed as a vital marker of scientific excellence.
Re-thinking excellence
This chapter is aimed to bring out a number of discussion threads on how the claim for excellence in research has been emptied of its substance. It is not enough to claim excellence out of the already existing normative machineries at work in the academic world. The question is what kind of quality in academic knowledge production that is relevant in what context and for what purpose. Both researchers and policy makers have to create and enter a joint learning space with a learning mindset in order to be able to tackle this never ending question. If we have learned anything from our experiences in cooperation with colleagues in developing countries, it is that research and politics are deeply intertwined and constitute conditions for innovative processes. One excellent skill here is how to navigate in a more or less totalitarian political system AND at the same time keep the university as autonomous as possible through learning alliances in-between scientists and citizens, official and local experts.
We are discussing transformation processes – that we realize and try to take part in - as processes of mode 2 and transdisciplinarity. Ulrike Felt is exploring the changing academic research environments in a European context and how researchers encounter, transform and oppose these changes. Felt is emphasizing the issue of creating and inhabiting what she calls epistemic living spaces (Felt 2009). We recognize these discussions as an important prerequisite for ourselves and other inhabitants in the academic world in order to feel intellectually and socially ‘at home’ (Felt 2009 p. 231) and for becoming answerable for what we learn how to see (Haraway 1991).