Author's accepted manuscript. The published version is at doi.org.
John Elliott and Dominik Lukeš
Epistemology as ethics in research and policy: Under what terms might case studies yield useful knowledge to policy makers
JOHN ELLIOTT AND DOMINIK LUKEŠ
This paper examines the ethnographic case study in education in the context of policy making with particular emphasis on the practice of research and policy making. The central claim of the paper is that it is impossible to establish a transcendental epistemology of the case study on instrumental rationality. Instead it argues for the notion of situated judgement that needs to be made by practitioners in context, practitioners being both researchers and policy makers. In other words, questions about the level of confidence or warrant that can be placed in different sorts of research evidence and findings cannot be answered independently of forming a view about the appropriateness of the policy culture that shapes political decision-making.
The paper draws a distinction between the general which is internal to the data as construed by a particular discipline and the universal which is the result of embedded human deliberation. This applies to all research findings and not only to case study, although since case study has had to long defend itself against accusations of the lack of generality, it can be a useful starting point for the discussion. This paper is not meant to be yet another defence of the case study research genre, although a summary of other defences is offered. Rather it focuses on how the case of the case study points to the limits of epistemology as rationality and offers the view of epistemology as ethics.
INTRODUCTION
In considering the issues surrounding the question of case study epistemology we were mindful of the following points made by David Bridges in his proposal to the Economic and Social Research Council (ESRC) for the Teaching and Learning Research Programme (TLRP) for the project on The Epistemological Basis of Educational Research Findings, defining the central questions to be addressed as:
... to do with the warrant or level of confidence which educational research (of different kinds) can provide for decisions about what should be done in terms of general educational policy across a whole system or practice at classroom level.
It will, therefore, Bridges argues:
... address specifically the claims made by contemporary discourses of ‘evidence based practice’ and ‘what works’, but it will locate these within a wider framework of literature which examines the relationship between research, policy and practice (including considerations to do with the derivation of practice from policy as well as policy and practice from research).
The task assigned to us was specifically to address issues about the level of confidence one might place on case study research as a basis for both policy and practice in the field of education.
What kind of confidence can we have in the ethnographic study and construction of a single case (or cross-site analysis of a limited number of cases) as a basis for policy and practice? What is the nature of the inference between evidence and recommendations for policy and practice in such studies?
In referring to ‘case study’, it is all too easy to engender terminological confusion. We will not attempt to provide a generative definition of ‘case study’, an exercise that has been well rehearsed (see Adelman, Jenkins and Kemmis 1976, and Stake, 1994 &1995 for more in-depth discussion). However, we believe that definitional questions are appropriately about how one characterises ‘a case’ as an object of inquiry and not about which method(s) are appropriate. Attempts to define ‘case study’ in methodological terms are all too often symptomatic of unproductive paradigm wars. Pole and Morrison (2003, p. 9) point out, that in the educational context “case study” is frequently a “synonym for ethnography”. Indeed Bridges treats it as such in his proposal to the ESRC (see above). The term ‘case study’ appears to have been captured by ethnographers working in the education research field and cast as the bearer of an alternative qualitative methodology to quantitative methods. We see educational case study as a form of inquiry into a particular instance of a general class of things that can be given sufficiently detailed attention to illuminate its educationally significant features (see e.g. Parlett and Dearden 1977, Simons (Ed) 1980). Such a view of case study is methodologically open. Methods need to be justified pragmatically in terms of their ‘fitness for purpose’ rather than in terms of a priori principles derived from a theory of knowledge. For some purposes ethnographic methods may illuminate the case, while for others narrative or even quantitative methods may be more appropriate (See the contribution by Griffiths and MacLeod in this volume).
Just as case study cannot be cast as a ‘method of inquiry’ nor can it be cast in terms of the scope, length or expense of the inquiry. The latter are dependent on the ‘needs of the case’. The idea that case studies are always small-scale inquiries compared with sample studies and can therefore be carried out in a shorter time-period at less expense is biased by assumptions that shape the latter. Case study research in education was initially developed in the UK and USA to inform decision-makers about the way an educational change proposal shapes up in particular context of action. It strove for an increasing particularization of description in order to illuminate the complexity of the interactions involved to shape actions to effect change and their consequences. Such case studies were extremely detailed and time consuming to produce. For example, the six case studies carried out in the context of the Cambridge Accountability Project (See Elliott, Bridges, Ebbutt, Gibson and Nias 1982) took five researchers 18 months to complete. Moreover, as this example illustrates, much case study-based research in education was designed to address the information needs of decision-makers beyond the confines of the school, such as members of the wider policy community. From the 1970’s, until quite recently, multi-site case study research - deploying a range of qualitative methods - emerged as a complementary and increasingly alternative paradigm of educational programme evaluation. This development was given considerable impetus by a group of evaluation specialists from the UK and USA who from 1972 onwards convened occasional meetings in Cambridge U.K (See MacDonald and Parlett 1973, Hamilton et al 1977, Elliott and Kushner 2007). A digital archive of case studies drawn from the work of members and associates of the Cambridge Conference, together with contextualising interviews, can now be accessed (http://www.caret.cam.ac.uk/tel). It was developed as part of an 18 month RDI project funded under the ESRC Qualitative Archiving and Data Sharing Scheme (QUADS). The impact of particular case study based evaluation and research projects and the issue of the generalizability of case studies are amongst the themes explored in the contextualising interviews,
It should also be noted that uses of the term ‘case study’ to refer to subject presentation in business and professional training and to problem-based learning or to purely quantitative pilot studies using smaller data-sets, are quite different to the use of the term to depict a form of research, as in this paper.
Questions of warrant, confidence1 and verifiability in the field of educational research generally are not new, or peculiar to case study. They are posed by the inevitable struggles for legitimacy of research paradigms in and across particular disciplinary contexts; the constant tug of war between qualitative and quantitative outlooks on the gathering and interpretation of data. A frequent source of confusion in these discourses is the use of the statistical metaphor to understand ‘generalizability’. In large-scale population studies, the determination of whether a particular description is general, i.e. valid for the entire population, is a matter of well-understood mathematical procedures that have to do with properties of large random cohorts. Although the limitations of this approach are known (see below, and in more detail Smeyers, this volume), qualitative researchers are called upon time and again to demonstrate how the results of their inquiry can be verified with respect to a larger set. However, this is not the focus of our paper. A near-well definitive treatment of how the validity of individual case study can be viewed with respect to the ‘statistical metaphor’ was provided by Kennedy (1979) who outlined how criteria for sample attributes in quantitative studies can be applied to individual cases and linking them to questions of verifiability and validity. A similar, albeit less formalized view, was expressed as early as 1961 by Becker et el. (1992). From this perspective a case study is valid if it illuminates certain general features of the class of objects to which the case belongs (See Simons, 1996, for a critique of this perspective). In fact, it is astonishing how little has changed in both the rhetoric and the institutional context of case study and qualitative research in general. Justificatory tropes, dignified as ‘methodology’ and aimed at linking the particular to the general, can be found as a matter of course in research publications of qualitative research.
One area we believe has not been as thoroughly investigated is the situated ethics of the researcher and the research-user (see Pole and Morrison 2003). One exception is in the field of educational programme evaluation (see Simons 1971 & 1987, Parlett & Hamilton 1972, MacDonald and Walker 1975, MacDonald 1976, Simons (Ed.) 1980, Kemmis 1980, Adelman (ed.) 1984, Elliott 1984 & 1990, Kushner 1993). While this is deeply intertwined with questions of verification and replication, we claim that any attempt to treat questions of warrant as transcending the individual and his or her embeddedness in their context are doomed to failure, and in effect, epistemologically naïve (See Elliott 1984).
STENHOUSE ON VERIFICATION: FROM THE CASE TO THE CASE RECORD
One of the more explicit treatments of questions about the verification and warrant of case study findings can be found in the work of Lawrence Stenhouse in the late 1970’s and early 80’s, before his untimely death in 1982 (See Stenhouse 1978, 1979 {a,b,c}, 1985). Stenhouse was keenly aware that in the UK case study had emerged in the fields of educational research and evaluation as an alternative to the then dominant psycho-statistical paradigm. In a lecture given at the Scottish Educational Research Association in 1979 he argued that:
The illuminative tradition... now seems to have got off the ground both in research and in evaluation. It no longer needs to fight to establish itself as an alternative to the ‘psycho-statistical paradigm’ as worthy of consideration. (1979a, p.1)
Stenhouse pointed out that illuminative research and programme evaluation2 was associated with the study of cases rather than the study of samples, as was the case with the psycho-statistical paradigm (p.3). Although sympathetic to illuminative research Stenhouse expressed concern about the extent to which such studies conformed to his understanding of research as “systematic inquiry made public” (p.4). At the heart of this concern lay two major problems, those of the verification of interpretations and their utilization by others operating beyond the boundaries of the case. For Stenhouse, utilization depended on cumulative case studies such that a researcher undertaking a study of a comprehensive school could draw on other studies of comprehensive schools. From his point of view the study of individual cases only constituted research if it yielded insights that could be generally utilized in understanding to other cases that belonged to the same class of objects. Here Stenhouse was anticipating a problem about case study that is frequently posed from within and without the educational research community, and indeed by members of the policy community, who argue that one cannot use case studies as a basis for generalization beyond the boundaries of the case. What they appear to be claiming is that one cannot use case studies to cumulatively generate knowledge about education through research, implying that one cannot apply knowledge of particulars with confidence to general problems of practice and policy. On this view, case studies do not fit the knowledge requirements for effecting solutions to such problems.
Stenhouse’s emphasis on the accumulation of knowledge through successive case studies is an issue that should not be left unexamined, mostly because of the great potential for misunderstanding. The cumulative nature of knowledge as such is both epistemologically and sociologically questionable.3 While on the one hand knowledge does seem to accumulate over time in a society and an individual, significant portions of it also seem to disappear.4 When case study (or any other research data) is being used to make policy and other practice decisions each individual teacher or policy maker as a practical philosopher (see below) needs to disaccumulate knowledge as presented to them in research and apply it to their particular problem. The knowledge that is available to them at the moment is highly integrated and compressed rather than unproblematically accumulated. Indeed, it could be argued that it is the ability of the individual to reconstruct their ‘cumulative’ knowledge into workable parts that makes for an effective practitioner. Kennedy (1979) makes a similar argument when she postulates “disaggregation” as a prerequisite to functional analysis. However, there is a very unproblematic sense in which Stenhouse’s aspirations for “cumulation” can be construed, and that is simply in the sense of assembling archival material for access and verification in the future. In that case, an easily accessible store (inventory) of past case study research can hardly be argued against. But the assumption that such an archive would serve as an unproblematic source of knowing about the world of education that is different from that available through the reading of a single case study is something that should receive more careful examination. It bears mentioning that one of the stronger arguments for indeed maintaining such a store of case study material is the education of qualitative researchers.5 Also a closer look at the US Education Resources Information Center ( HYPERLINK "http://www.eric.ed.gov" \o "http://www.eric.ed.gov" ERIC) and its real use by researchers, practitioners and policy makers, could prove revealing, particularly as it pertains to the utilization of case-study data.
Underpinning the issue of case study and generalization is a strong epistemological boundary between knowledge of particulars and knowledge of general laws. It is reflected in the historian Wilhelm Windelband’s distinction between ideographic and nomothetic methods. In this respect he was attempting to draw a boundary between the natural sciences that are concerned with formulating general laws and the human sciences, such as history, that he contended had their own methods and discipline (see Blackburn 1994, p. 185). Hence, one can argue that case study methods are ideographic and concerned with generating knowledge that takes the form of particularizations of descriptions of things and events, while policy makers need nomothetic knowledge in the form of general principles governing the relationship between causes and their effects. Only on the basis of the latter some will argue can the policy community formulate rational policies that are instrumentally effective in achieving desired educational outcomes. If one is justified in drawing a tight methodological boundary between ideographic and nomothetic methods then one can conclude that, while case studies may support judgment and action within the particular circumstances depicted they cannot be accommodated as evidence in which to ground policy making. Such an argument is underpinned by certain beliefs and assumptions about the policy context; namely, that instrumental effectiveness is the primary criterion for evaluating policy decisions and that this implies a clear and measurable specification of desired outcomes in the form of ‘targets’. We will return later to the question of how the policy culture shapes the level of confidence and warrant that can be placed on different kinds of research evidence and findings.
Let us now return to Stenhouse’s reservations (1979a) about the illuminative case study paradigm of educational research that had emerged in the wake of the critique of the psycho-statistical paradigm. Stenhouse was concerned that doctoral students and some post-doctoral researchers might declare themselves to be doing illuminative research in order “to escape the pressure of standards” (p. 5). He was not convinced by arguments from within his own centre (CARE) at the University of East Anglia, that case studies are analogous to works of fiction and therefore a kind of story that sheds any appeal to authenticity other than its power to emotionally engage the reader. For Stenhouse fiction aimed at a fuller truth about human experience than illuminative research which could only be realized “by shedding the burdens of authentification” (p. 4). Good fiction for him was the creation of exceptional individuals, but he wanted good illuminative research to be the product of “honest hard work because it has the support of a discipline” (p.4). The idea that illuminative case studies should aspire to ‘best seller status’ was not one Stenhouse endorsed. Nor did he support a tendency to cast the role of the illuminative researcher in heroic mould, as ‘speaking truth to power’. Some of his colleagues began to view illuminative research as analogous to investigatory journalism. He argued that such an analogy is only rarely appropriate. It presupposes a context in which hidden truth has to be exposed in the face of a powerful adversary who is attempting to suppress the evidence. Much illuminative research Stenhouse argued does not presuppose this kind of adversarial context, typically depicted as a situation in which the policy community abuses its power by suppressing evidence about the facts (See p. 4).
Stenhouse was anxious to build an illuminative research tradition that was founded on clear procedures for verifying case studies and utilizing their insights beyond the boundaries of the case they applied to. He proceeded to specify verification procedures in the form of constructing case records of evidence gathered from a range of sources in a variety of media (See Stenhouse 1978, 1979a). The function of the case record was to enable the interpretations in the researcher’s case study to be replicated through an appeal to a systematically organized body of evidence – interview excerpts, observational notes, documentary evidence - that the researcher had drawn on in constructing the study. In this respect Stenhouse looked to history for a method of disciplining illuminative case study research (See 1978 pp.21-39, 1979a pp. 5-9). However, he avoided implying that the use of historical case study methods excluded a concern for generalization. Generalizations can be cumulatively constructed from cases retrospectively rather than take the form of general principles that enable people to predict in advance how events will unfold in the cases they cover (1979a p. 7). Stenhouse’s retrospective generalizations are rather like the kind of ‘universal principles’ depicted by Nussbaum (1990). They link cases together by discerning their similarities without, in Nussbaum’s words, “dispensing with their particularity.” Retrospective generalizations constitute cumulative insights based on cross-case comparisons. They are couched in the form of summaries of judgment, which enable people to anticipate rather than straightforwardly predict possible events. Both Stenhouse (1978) and Nussbaum (1990) view their retrospective generalizations (Stenhouse) and universal principles (Nussbaum pp. 67-69) as supporting rather than displacing professional judgment.
Stenhouse refuses to accept that case studies cannot be used to construct general insights in the field of education. If he is right then there are no good epistemological reasons for claiming that case studies cannot be used to inform the judgments of policy makers. Indeed he argues (1985) that educational policy in the UK tended to be crude inasmuch as it is “formulated at the expense of much thought” (pp. 263-264) on the assumption that it applies to all cases. An exclusive reliance on the findings of psycho-statistical research is unlikely to challenge this assumption since such findings will tend to be used to justify pre-formulated policies. Stenhouse argues that in case study research, as opposed to sample studies, the relationship between the case, or a collection of cases, and any population to which similar meanings may apply is a matter of discernment and judgement. Such general insights are therefore the product of reflective thought. Stenhouse is careful to distinguish the ‘insights’ that emerge from the comparative study of cases from the general law-like principles derived from the study of samples. The latter may appear attractive to policy-makers as a basis for confidently predicting the outcomes of policy, but in the realm of human affairs they promise more than they can deliver (See Stenhouse 1979c, p. 5-6). Insight rather than law Stenhouse argues is a more appropriate basis for understanding. The policy-makers grasp of actualities is not so much enhanced by the straightforward application of general principles as by judgements that are tutored by the comparative study of cases. We will return later to Stenhouse’s account of the distorting effects of an over-reliance on the findings of sample studies on education policy and practice.
At the time when Stenhouse was writing about the problem of disciplining illuminative research in the field of education many who subscribed to this approach appealed to ethnographic methods developed by social anthropologists. These included participant observation, through which the researcher became familiar with the meanings that participants attributed to the social setting he or she sought to understand, and grounded theorizing through which theory building proceeds interactively with field observations and the constant comparison of cases. Stenhouse (1979a, pp. 6-7 & 1985, p. 266) expressed worries about the use of ethnographic methods in the context of educational research on a number of counts.
Firstly, ethnographic methods presuppose a particular standpoint on the part of the researcher: namely, that (s) he is unfamiliar with the kinds of situations (s) he set out to study. Educational researchers, on the other hand, and their readers have a lot of first-hand experience of life in classrooms and schools as pupils. They have already absorbed many of the meanings that shape the transactions between teachers and their students in classroom and school settings. Illuminative Educational Research will throw new light on familiar educational situations as opposed to throwing light on the unfamiliar.
Secondly, the ethnographer will tend to employ second order theoretical constructs from her discipline - ‘ritual’, ‘rites of passage’, etc. - to render the social experience of participants intelligible to her peers. Hence, Stenhouse tends to go along with the view that the ethnographic methods of the social anthropologist tend “to increase the power of the community to which the ethnographer belongs without making a similar contribution to those who are studied.” In the context of education the outcomes of ethnographic research would tend to be inaccessible to practitioners. Stenhouse argues that illuminative research which supports practical judgment “calls for parsimony of theory, and theory which is within the literacy of the actor.” In other words theoretical insights should be expressed in the vernacular non-technical language of participants if they are to inform judgment. They may build out and extend this language but in terms whose meanings are continuous with and fit into the network of meanings it consists of.
Thirdly, ethnographers tend not to make their field notes available, and for Stenhouse this constitutes a problem for verification. He suggests that verification can be tightened up in ethnographic research by preserving the field–workers’ records. However, such records present an issue. Many of them are analytic records containing data that has already been processed in the course of field research, and which do not separate data from analysis. Hence, it is not clear how exactly the inclusion of Field-Notes tightens up the verifiability of ethnographies, since the latter would imply separating ‘interpretation’ and ‘evidence’.
The verification problems of ethnography are, according to Stenhouse (1979a), compounded by the difficulties of replication. It is rarely possible for a study to be replicated by other ethnographers, so that one can compare the consistency of interpretation/analysis across studies of the same situation or setting. Situations change or die away over time. This is why he looks to historical methods as a source of standards for illuminative case study research. The verification of case studies that follow the historical tradition would rest on the meticulous keeping of records of evidence, that were relatively free of the researchers own interpretations. In the interpretative case study the researcher would footnote references to the relevant sources of evidence (both primary and secondary) that are held to warrant a particular interpretation of a situation or event. These sources would then be systematically organised in the form of a case record for the purpose of enabling others to critique the case study and to judge the extent to which it could be verified as a reasonable interpretation of the available evidence. Stenhouse defined the case record as a selection from the totality of the sources gathered, the case data, for the purpose of enabling others to verify the study. Of course, the researcher would also list all the case data gathered to enable others to check the sample selected as the case record for bias.
Stenhouse claimed that an appeal to the kinds of verification procedures embedded in the historical tradition of inquiry helped to resolve the problem of utilizing case study accounts beyond the boundaries of the case. They enabled generalizable insights across cases to be cumulatively constructed on the basis of verifiable case studies in a way he believed was problematic for ethnographic case studies, unless they incorporated elements from the historical tradition. The existence of records of evidence ensures that people can have confidence in a case study as a source of insight into present circumstances, in spite of the fact that the situation it depicts may have changed and sometimes changed beyond all recognition. There may still be lessons that can be learned from such a case study, but this will depend on the trustworthiness of the account. Hence the significance for Stenhouse of the case record.
Becker et al. (1977 [1961]) provide a counterpoint with the claim that “the analysis of participant observation data can, and usually does, proceed in a careful way to test hypotheses in ways that can be communicated and replicated” and that case study often provides “evidence for conclusions in field research [that] is usually better than critics think.” (p. 30) This does not necessarily invalidate the concerns of Stenhouse and others but, if correct, it should put their weight in a proper perspective.
One advantage for Stenhouse in conducting educational case studies in the historical as opposed to the ethnographic tradition is that they will present education “in a language comprehensible to the educator”(1979a,p.7). This is because the researcher gathers documents or creates them through interviews with “a parsimony of interpretation in the field” (p.8). This enables the educational researcher to engage in a reflective interpretation “of a second record” (p.8) that can be shared with his audience, which Stenhouse believed should largely consist of educators and not merely academic educationalists.
CASES VS. SAMPLES
Stenhouse (See 1979b, p.3 {1980}) regarded case study research as complementary rather than an alternative oppositional paradigm to the study of samples. There was room in educational research for both the study of samples and the study of cases, and he wanted the proponents of each to talk to each other. For him the discourse of educational research should be woven in terms of this distinction. He regarded it as of greater significance than distinctions between ‘quantitative’ and ‘qualitative’, ‘psycho-statistical’ and ‘ethnographic’, ‘positivism’ and ‘humanism’. He was particularly concerned to leave space for quantification and numbers in his account of educational case study research.
Stenhouse spelled out very clearly what he regarded as the limitations of sample studies in educational research, and argued that they were not simply technical limitations that could be overcome. The problems went deeper than that. Hence,
Statistically significant preferences for one treatment as opposed to another generally meant that in a substantial minority of cases – as many as 40% it could be - the treatment which showed better overall was in fact worse. What was sauce for the goose proved not to be source for the gander! (1979b, p.5 {1980})
Others concur. In fact, as Becker (1992) suggests, a complete symmetricity and generality (of 100%) in quantitative research is an indicator of an imperfection (either in measurement or the mathematical procedure applied). It has even been used to uncover scientific fraud (see Gould, 1996). Therefore, the psycho-metric approach to research is just as much compelled to defend its generality as the case study approach.
Stenhouse argued that sample studies in education based on probability statistics could provide guidance for decision-makers but that:
To make refined judgments about what educational action to take in particular cases lodged in particular contexts, we need much more information than can at present be reduced to indices and we need to present our conclusions in a way that feeds the judgment of actors in the situation, a way that educates them rather than briefs them (1979b, p.6 {1980}).
The role of educational case study research was therefore to complement rather than supplant the study of samples. It was to contextualise the findings of sample studies in ways that educated the judgment of professional educators.
Stenhouse’s view echoes that of Cronbach (1963) in his seminal paper on ‘Course Improvement through Evaluation’. Cronbach argues that the improvement of educational programmes depends on gathering kinds of information which depart “from the familiar doctrines and rituals of the testing game,” (p. 672). He claims (pp. 672-683) that many varieties of information were useful to decision-makers in effecting improvements to a programme, and that standardised achievement testing was of limited usefulness in this respect. Programmes have multiple effects which vary according to context and decision-makers need to know what these are and how they are produced. Information gathering required the educational researcher to deploy a variety of different research measures and instruments to capture effects and processes. Cronbach, the measurement expert, critiqued a particular ideology governing the use of measurement in educational evaluation (see Nussbaum’s {1990} account of the ‘Science of Measurement’ below) rather than the use of all measurement. The critique was very influential in paving the way for the development of Case Study research in the field of educational programme evaluation.
The implication of Stenhouse’s view of the relationship between the study of samples and cases is that government has tended to place too much confidence in sample studies as a basis for predicting educational outcomes.6 The tendency to use findings from such studies as a basis for prescribing educational action and marginalising professional judgment in the process cannot be warranted by any defensible theory of knowledge. Educational policy may profitably use the findings of sample studies to frame agendas for case study research that can yield cross-case comparisons capable of educating professional judgment. Such evidence-based frameworks for case study research may do more to improve educational practice than the current standards driven reforms that exclusively rely on ‘the science of measurement’.
Helen Simons has been somewhat critical of efforts to integrate case study into large scale research designs using a sampling framework. For her such efforts go too far in accommodating policy-makers views of their information requirements. Of case study research she writes that:
“...along with its acceptance and widespread use in a number of fields... has come a weakening of the very characteristics of case study which prompted its emergence in the first place... one of the reasons for this has been the ever present worry over generalizations... and the pressure in current times from sponsors who do not value qualitative data, let alone that stemming from a single case” (Simons 1996, p. 225).
Simons sets out a rationale for the single case study as a form of research in its own right. As a research genre, she claims, the value of case study research does not solely depend on its use to complement large-scale sample studies. A single case study she claims can paradoxically generate both unique and universal understandings (p.225), a distinction that has more recently been contested by Evers and Wu (2006), who argue that:
“case study research is not different in kind in its use of the epistemic resources we regularly employ to successfully understand and navigate our way around in the world.” (p. 524)
Simons on the other hand argues that the use of a single case study within her community of educational programme evaluators has been a means of establishing claims to knowledge that cannot be accommodated in traditional paradigms of evaluation. Such claims refer to universal understandings of human situations and events that can only be evidenced by holistic portrayals of the unique features of a case. Such portrayals, she argues, “strive to know and represent complex educational situations” and thereby constitute a form of understanding that matches the decision-makers ‘vocabulary of action’ (p. 226). Simons is articulating a tradition of case study research (See Simons, H {Ed.} 1980) that transcends the boundary between art and science. She claims that “this form of knowing is not divorced from science nor is it to be seen as the antithesis of it” and cites MacDonald and Walker (1975{1977}):
Case study is the way of the artist, who achieves greatness when, through the portrayal of an instant locked in time and circumstance, he communicates enduring truths about the human condition. For both the scientist and artist, content and intent emerge in form (1975, p.3).
Simons points out that MacDonald and Walker are very aware of the implications of their definition of case study. The choice of the word ‘instance’ for them implies a goal of generalisation. Moreover, as Simons points out, the researcher who played a leading role in forging the portrayal tradition of case study, Robert Stake, has argued that such generalizations are ‘naturalistic’ rather than ‘formalistic’ (See Stake 1978, reprinted in Simons (Ed) 1980 p.64). It is not the researcher that does the generalizing but the reader. The good portrayal provokes its readers into a new way of understanding their own practical situation. The claim that such situational understanding also embodies knowledge of universal truth is validated when readers of a case portrayal come to think differently about their own experience. It renders the familiar qualitative world they live in strange. This is why programme evaluations that relied heavily on thickly descriptive portrayals of cases became known as ‘illuminative evaluation’.
We have already noted Stenhouse’s reservations about the case study research that fell into what he and others called the ‘illuminative’ paradigm’. It is this paradigm that chiefly emerged in the context of educational programme evaluations that Stenhouse regarded as insufficiently disciplined and rigorous to warrant confidence in their interpretations and findings. He was sceptical even about attempts to inject methodological rigour from qualitative social science methods like ‘ethnography’ into social science methodology; hence, his search for rigour in historical method. Simons, however, has persisted in defending case study as a naturalistic form of educational inquiry that does not depend on methods derived from the academic disciplines as a guarantor of truth. Indeed she points out that portrayals of cases avoid that distancing from experience involved in the use of objectivist methods of data analysis.
The philosopher John MacMurray (1957,) lends support to the portrayal tradition of case study. MacMurray distinguishes between the intellectual and emotional modes of reflection (pp. 198-202). The former implies a certain ‘analytic distancing’ from direct experiences of situations and events, while the latter involves an attempt to understand what is being directly experienced. The emotional mode seeks to understand a practical situation as a “system of new possibilities for action”. The greater the particularisation of the understanding of the situation the more the complexities of making wise judgments and decisions are taken into account and it’s practically relevant features discerned and discriminated. MacMurray echoes the psychologist Rollo May (1994), who is cited by Simons (1996) as providing an account of the kind of ‘creative understanding’ involved in case study portrayals. May she points out argued that “... reason works better when emotions are present; the person sees sharper and more accurately when his emotions are engaged.” (p.49) The philosopher, Joseph Dunne (2005), also lends support to the portrayal tradition of case study research when he contends that the ‘situational understanding’ which stems from direct experience - what Aristotle called phronesis - fuses a grasp of the particular with the discernment of universal truth (p.375), and he argues for “richly descriptive studies” that possess “epiphanic power” by “illuminating other settings” (p.386) as resources that support the development of understanding.
If holistic case study portrayals of the unique features of situations and events match the policy-makers ‘vocabulary of action’ then how do we explain their rejection by the policy community? Simons (1996) argues that the reasons why educational programme evaluator’s utilised case studies in the 70’s are now being eroded in a contemporary policy context that is less concerned with understanding the complexities of effecting educational change in particular contexts of action, and more concerned with a search “ for generalisation, in the formal sense, for comparability, for certainty” (p. 227). This suggests that the level of confidence policy-makers do in practice place in case study findings and evidence may change considerably according to the prevailing policy culture.
MEASUREMENT VS. PRACTICAL PHILOSOPHY
For example, the prevailing culture to-day is frequently depicted as a ‘target culture’ and is framed by a view of practical reason that Martha Nussbaum (1990) characterizes as ‘The Science of Measurement’ She breaks this ‘scientific’ conception of practical reason into four constitutive claims (pp.56-57). First, there is the claim of Metricity, that “in a particular choice situation there is some one value, varying only in quantity that is common to all the alternatives”. The rational chooser is one who uses this single standard as a metric to weigh each alternative and thereby determine which will yield the greatest quantity of value. Secondly, there is the claim of Singularity, that one and the same metric or standard applies in all situations of choice. Thirdly, there is the claim of Consequentialism, that the chosen actions only have instrumental value as a means of producing good consequences. Fourthly, by combining each of these claims we have the principle of Maximisation, “that there is some one value, that it is the point of rational choice, in every case, to maximize”.
Policy-making and research in the field of education has become increasingly shaped by this so-called ‘scientific’ picture of practical reasoning (see Schwandt 2005, pp. 294-295 and Elliott and Doherty 2001, pp. 209-221 in relation to the US and UK respectively). Hence, the educational reforms initiated by many governments are viewed as devices for ‘driving up standards’ conceived in terms of metricity and singularity. Research increasingly takes the form of school and teacher effectiveness research aimed at determining how schools and teachers can ‘add value’ to students learning. The maximisation principle is embedded in the idea of ‘value added’, which presumes that the practices of schools and teachers only have value if they produce good consequences that can be quantified in terms of a single metric that applies generally across the system. Hence, the widespread use of standardised testing in school and teacher effectiveness research. The findings from such research are now widely regarded as furnishing the rational basis for modernising educational practice in schools and classrooms. Hence the currently fashionable, widely circulating notion in many policy contexts of ‘evidence-based practice’ (See the paper by Pring and Oancea in this volume for a full discussion).
Nussbaum (1990, p.55) claims that contemporary conceptions of practical rationality in “almost every area of social life” are so dominated by the “science of measurement” that we are in danger of losing sight of the Aristotelian conception of practical rationality as phronesis. We agree. The inroads that the evidence-based practice movement has made within the field of education in western countries, and even more globally, is testimony to this. However, this ‘scientific’ picture of rationality is also making it increasingly difficult to imagine any other relationship between science and practical reasoning. We will now argue that some policy contexts still leave space for a view of practical reasoning as phronesis and of the policy-maker as a ‘man/woman of action’ rather than a ‘technocrat’. This implies a different relationship between science and practical reasoning. We will conclude that questions about the level of confidence or warrant that can be placed in different sorts of research evidence and findings cannot be answered independently of forming a view about the appropriateness of the policy culture that shapes political decision-making.
In some contexts findings based on comparisons between cases within a large-scale sample frame, and couched as formal generalisations, may command a certain level of confidence. Stenhouse’s concern with issues of verification and utilization appears to address such contexts. In other contexts a single case study may challenge existing policy frames, and open up for policy makers new possibilities for action to effect change along lines suggested by Simons (1996, p.231). In what kind of policy context might the policy-maker find single-case portrayals useful resources? We would suggest that it is a context that leaves cultural space for phronesis as a mode of practical reasoning. It is a case-focused rather than sample-focused mode of reasoning although it may eclectically draw on sample studies in the process. On the other hand we would suggest that Stenhouse’s conception of case study fits a context in which space for phronesis cannot be presumed but needs to be opened up.
CASE STUDY AS GENRE AND AS A MODE OF REASONING
One of the problems with the assumptions around the generalizability of case study is that it suffers from a number of conceptual conflations both in the collective discourse and the individual practices in educational research. On the one hand, there is the case study as a research genre with its conventions and morphological properties that serve to identify it as such to participants in research discourse. In other words, in order for a case study to be (accepted) as a case study, it must “look” like a case study. On the other hand, there is the notion of case study as a mode of reasoning (case-focused reasoning) that is seen as qualitatively different from the deductive reasoning assumed to be the exclusive domain of large-scale probabilistically controlled trials. Stake (1994) argues that the “utility of case research to practitioners and policy makers is in its extension of experience” (p. 245) and claims that case studies provide both “propositional and experiential knowledge”:
Certain descriptions and assertions are assimilated by readers into memory. When the researcher’s narrative provides opportunity for vicarious experience, readers extend their memories of happenings. Naturalistic, ethnographic case materials, to some extent, parallel actual experience, feeding into the most fundamental processes of awareness and understanding.7 (Stake 1994, p. 240)
Assumptions such as this make case study as a genre of research easily confused with case-focused reasoning and consequently it is easy to suggest that it offers superior results for situational judgment. While a case can be made that situational judgment is in fact an example of case-focused reasoning, there is ample evidence that the mental and social construction of cases is not dependent on the format (genre specifics) of a case study and can be (and frequently is) just as easily built on sample-based quantitative data. Indeed it can be argued that case-focused reasoning is central to intelligent judgment as typified by the political ‘man of action.’
Let us step back from viewing case study as a research genre and instead view it as a form of deliberative reasoning (phronesis) about a situation in which there is a political imperative to act. Paradoxically case-focused reasoning is oriented towards both socially and politically committed action, or praxis, and the clarification of universal conceptions of the good. Such reasoning has been depicted as practical philosophy (See Carr 2006, pp.421-436, Elliott 2007, pp. 99-112). Gramsci, the Italian political revolutionary and intellectual, depicted such a process as lying at the heart of all authentic political discourse (See ‘The Modern Prince’ in Hoare and Nowell Smith {Eds} Selections from the Prison Notebooks of Antonio Gramsci 1971, pp 313-441 in the 1999 Electric Books edition). The ‘man of action’ he argued is also a true philosopher; one who clarifies universal political ends – equality, justice, and freedom - by deliberating about what constitutes an appropriate expression of them in a particular concrete situation. John Lloyd has recently pointed out in Prospect (July 2007, pp.24-28), that Gramsci’s political ‘man of action’, cast in the mould of the practical philosopher, is not the professional intellectual or specialist philosopher that one finds in universities. Nor is he or she a French-style polymath public intellectual, like Derrida or Foucault, or the kind of bookish and cultured politician that was typified in Margaret Thatcher’s UK government by Sir Keith Joseph. Lloyd’s article portrays the UK Prime Minister Gordon Brown as a practical philosopher in Gramscian terms (Brown being a youthful admirer of Gramsci). He cites (p.24) Gramsci’s portrait of the philosopher as ‘man of action’ in The Modern Prince as someone who:
“can affect his own development and that of his surroundings only so far as he has a clear view of what the possibilities of action open to him are. To do this he has to understand the historical situation in which he finds himself and once he does this, then he can play an active part in modifying that situation. The man of action is the true philosopher: and the philosopher must of necessity be the man of action”.
From Gramsci’s point of view case-focused reasoning or practical philosophy has a strong historical dimension. The discernment of concrete ways of expressing in action conceptions of the universal good (situational understanding) depends upon the political agent’s grasp of the historical dimensions of his/her situation.
In the light of this we can see case-focused reasoning as a process that stems from the experience of the moral or political imperative to act in a social situation, and which unifies universal and situational understanding. And we can also see descriptive/holistic portrayals of cases – normally referred to as case studies - as artefacts that may provide resources for such reasoning inasmuch as they help the political ‘man of action’ to discern new possibilities of action that are open to him or her in his/her situation. This essential independence of case-focused reasoning from the case study was, we believe, well understood by Stenhouse (See 1979b). Not only does he regard the case study, conceived as a specific format, as simply one complementary resource to support case-focused reasoning, he also points to “an acute need for attention to be paid to quantitative aspects of case study.” (Stenhouse, 1979b, p.8 {1980}). He cautions that “descriptive case studies should not confine themselves to words.” Rather they should ask questions along the lines of “What indices might best be gathered to describe a school and to locate it within the population of schools?” (Stenhouse, 1979b, p.8 {1980}). The same claim was made by ethnographers and sociologists, for instance recently by Pole and Morrison (2003) who maintain that their concept of ‘inclusive ethnography’ also “permits the use of quantitative methods.” (p. 9)
Stenhouse attempted to represent case study as a serious methodological option for educational research as well as an object of reasoning. From the standpoint of members of the portrayal school, such as Stake, “Case study is not a methodological choice, but a choice of object to be studied.” (1994, p. 236). For this school the case study is not a research genre that complements other research genres. It serves a quite unique and distinctive, if paradoxical, purpose in relation to the object of reasoning, ‘the case’.
All this leaves us with three distinct views of case-study research operating in the discourse of educational research: contemporary history, ethnography and grounded theory, and those holistic narratives of experience called ‘portrayals’. Each shapes what a case study looks like rather differently, establishing a subgenre, and implies somewhat different notions of case-focused reasoning.
Research by Kennedy (1999; see below) suggests that practitioners place greater emphasis on the direct relevance of research results over the procedural parameters of a particular research subgenre. Such a finding matches Joseph Schwab’s (1970) seminal work “The Practical: a Language for Curriculum.” He argued (pp. 1-5) that practical reasoning was case-focused and eclectic in its orientation, utilizing a range and variety of intellectual and cultural resources in its search for better understanding of the practical situation. From this point of view different kinds of case studies and other kinds of research may all have a role as resources for the policy-maker who creates and finds space in the policy context to be a practical philosopher.
Furthermore, we argue that, quantitative data provide intrinsically no better source of generalization because case-study-like stories need to be constructed about how samples are conceived and how they apply to a given situation. In fact, the same genre-led confusion is present: because large-scale (usually based on randomized sampling) quantitative studies appear to bear on situations where particulars of individuals need to be abstracted and generalized, it is assumed that they are better suited to providing generalizations such as may be needed by policy-makers. However, upon closer investigation, large-scale quantitative research is inexorably interconnected with case-focused reasoning at a number of levels. The construction of scenarios and cases is essential in the design of quantitative experiments and research projects both in the form of discursive assumptions and actual case-based pilot studies. But it is no more absent in the policy implementation of the resulting data. Case-like stories have to be told before a policy based on quantitative “evidence” can be translated into prescriptive documentation, but they are also essential in the political justification of a policy. Thus, when a politician policy advocate says that “X in Y children cannot read” she is not referring to the properties of a population as exemplified by a sampled cohort but rather enumerating the frequency of an occurrence of a particular schematised story.8 However, this knowledge has higher social acceptability in the ethnic structures of the groups interacting at the intersection of policy and research. This is an argument similar to that made by Hammersley’s (2001) discussion of Bassey’s (1999) notion of “fuzzy generalizations”. Generalizations in the “hard” sciences, Hammersley claims, are only approximate in their applications to the real world. And therefore fuzzy generalizations (such as those Bassey suggests should be drawn from quantitative research) are “not a distinct type of generalisation but a mode of formulation that ought to be employed in all predictions for practical use derived from scientific generalisations” (Hammersley 2001, p. 219). Hammersley takes his case further in suggesting that “survey research cannot produce strong evidence for probabilistic laws unless the population conforms to the terms and conditions of the theory.” (p. 221)
CASE STUDY AS A COGNITIVE AND SOCIAL ACTIVITY
Research by Kennedy (1999) into the way teachers evaluate different kinds of research evidence appears to back up this distinction between case-focused reasoning and case study as a research genre. She found that research genre was not as significant as the extent to which teachers could interpret the research findings in ways that enabled them to make sense of the complexities of teaching and learning in their own classrooms. She claims that her study casts doubt “on virtually every argument for the superiority of any particular research genre, whether the criterion is persuasiveness, relevance, or ability to influence practitioner’s thinking.” (p. 536) For example, when she asked teachers about what persuaded them about the utility of a particular piece research, the reasons similar to “descriptions of teaching or of student work were sufficiently detailed that I could envision them and see how this would be so” (p. 530) were given more often for an experimental study than a case study. Both of these left narrative studies far behind. The teachers’ judgments of relevance were their own and were made along the lines of their immediate need rather than simply following the cognitive rails supposedly laid out by the genre. Kennedy concludes that…
…teachers forged analogies between the studies they read and their own situations or practices and that they were able to do that for the experiment as well as for other genres of research. That is they did not need ethnographic studies or narratives to make these connections themselves (Kennedy 1999, p. 537).
Hence, we can argue that policy-makers like teachers may employ case-focused reasoning to interpret research findings regardless of the research genre. We need to know what they need to make sense of in their particular policy context. There is nothing about case study as a research genre that disqualifies it a source of useful insights for policy makers, just as there is nothing about experimental or random controlled trials as a genre that would disqualify them also.
Becker illustrates this by describing an imaginary look at some local census data as a basis for creating an image about the people behind the numbers, concluding that “[i]maginative, well-read social scientists can go a long way with a little fact”:
I can work up, in my mind, a complete if provisional, picture of the neighbourhood. [...] My picture is more than a compilation of statistics. It includes details that are not in the books and tables I consulted, details I have invented on the basis of what those books told me. I ‘know’, for instance, what kinds of houses these people live in – I can practically see the flamingos, the furniture ‘suites’ from the credit furniture store and whatever else my stereotype of the population produces (Becker 1992, p. 211).
Abbot (1992) makes a similar case when he observes that quantitative social scientists tend to start referring to ‘real people’ when their numerical data fails to account for some aspect of what they are interested in.
How this process of causal imagination plays itself out in the process of policy building is addressed in more detail by Schön and Rein (1994) in their theory of frame reflection. They suggest that all policy disputes can be understood in terms of frames:
Frames are not free-floating but are grounded in the institutions that sponsor them, and policy controversies are disputes among institutional actors who sponsor conflicting frames. (p. 29)
These frames contain both rich and schematic images (Lakoff, 1987, 1996) and scenarios that are more than remotely reminiscent of the sorts of images social scientists construct out of qualitative as well as quantitative data. The notion of frame has a long pedigree and has appeared as ‘horizon’ in Gadamer (1975), ‘perspective’ in Becker et al. (1977 [1961]) and finally introduced as frame by Goffman (1975 [1974]). Fisher (1997) provides a detailed overview of the different ways the concept has found its place in the social sciences. Gamson’s (1992) work is of particular interest in this context showing how frame construction can become independent of ‘the message’ in social movements. Although we do not make use of the concept here, an instrumental epistemological account taking into consideration both cognition and socialization may have to proceed in the direction of frame analysis (see Lukeš 2007).
That’s why when we ask whether case studies are generalizable, what we are really asking is, whether we can tell a sufficiently socially acceptable story about case-study based research as a basis of policy making. Because epistemologically there is no doubt about the generalizability of case study. Not only is it generalizable, it is in every instance the basis of generalizations. Of course, this requires a non-normative and non-instrumental view of generalization. In refusing to doubt the generalizability of the case study we are not assuming that all or not even most generalizations derived from case study will be “good” generalizations just as most generalizations derived from quantitative studies turn out to apply to a much narrower scope of “reality” than their patina of scientism seems to imply. This underscores the central point of our paper that we introduced from its beginning, viz that the epistemology of case study is not transcendent of the complexity of the situation in which the researcher, practitioner and policy makers find themselves. Indeed, it is possible to construct a set of criteria that will lead us to conclude that a case study is not generalizable but such a set will perhaps serve the purposes of a Platonic world of ideals but not the complex world in which researchers, practitioners and policy makers are located. However, as we were reminded was Stenhouse’s position, that also does not mean that rigour and objectivity are absent from the case study and should not be sought. Kennedy (1979) attempted to outline how qualitative research can draw on principles of generalization established in qualitative research to guide case study design citing medicine and law as examples of disciplines where such a combination is possible. Kennedy contrasts the situation in social sciences where “researchers and evaluators are not accustomed to the notion of leaving generalization up to the practitioner” (p. 672) with that in other fields. Judges and juries in the legal system and doctors in medicine commonly have to do just that as Kennedy observes: “generalizations are frequently necessary from single cases, but it is also clear that these generalizations are done by the user of the case data rather than by the person who originated the case data.” (p. 676) Robinson and Norris (2001) assume the cognitive inevitability of generalization but they see the role of method only as part of the practice of research:
The warrant we have for making judgements on the basis of research can take a number of forms: political, legal, moral, methodological. For researchers, the first concern is often the methodological reasons we have for the judgements we make. This should be the case since other justifications only become an issue when we are clear what can be said on the basis of the research although, of course, when we generalise we have particular purposes and utilities in mind and these cannot be separated from moral or political considerations. (p. 308)
Kemmis (1980) points out that methodological issues are not sufficient to address questions of warrant and generality. We must look to the situation of the researcher:
In general, there are two kinds of justification: those concerned with justification of the truth-status of findings, and those concerned with the accountability of researchers for the conduct of the research. In both cases, public access to the processes and products of research should provide the grounds for deciding whether a given research study is justified. (p. 97)
This makes methods of case study not particularly distinctive due to the pragmatic need to respond to the nature of the case. The person of the researcher is paramount.
since the methods employed in the study will depend upon the hypothesized nature of the case, a ‘methodological’ definition will always seem inadequate to the experienced case study worker - it will not grasp the problem of case study as the case study worker experiences it. (p. 108)
It is particularly telling that when faced with accusations of ‘positivism’, proponents (e.g. Hargreaves, 1997) of the evidence-based policy paradigm (allowing for controlled quantitative evidence only) tend to fall back on the moniker of ‘evidence informed’, thus substantially diluting their claims to have discovered general rules to govern practice. They implicitly afford case-focused reasoning a role in generating universal practical insights while in the same breath appearing to deny such a role. This applies as much to the researched practitioner as well as the researcher practitioner as was apparent even in the early days of educational evaluation:
“Many of the quite legitimate questions that are put to evaluators, especially by teachers, cannot be answered by the experimental methods and numerical analyses that constitute the instrumental repertoire of conventional educational research.” (MacDonald and Walker, 1977, p. 189)
CASE STUDY AS METHOD
The initial question was: “What kind of confidence can we have in the ethnographic study and construction of a single case (or cross site analysis of a limited number of cases) as a basis for policy and practice?” The answer seems to be that an ethnographic case study unreservedly can be, and frequently is, used as a basis for policy decisions. The more interesting question should what are the sources of confidence we place in a particular piece of research, a researcher and a policy maker. Here the picture becomes much murkier. Stenhouse speaks of a need “to establish conventions for the conduct and reporting of fieldwork in case studies, which attempt to secure a basis for verification and for cumulation.” (1979a, p. 9)
The key issue should not be whether and how a collection of case studies can be of use to the policy maker but rather how a single case study can and does enter into the world of knowledge and policy. What is doubly important is that a case can be made for the claim that at the moment of policy/practice decision-making any research data is epistemologically a kind of a case study. The only difference between different sources of data is then their social acceptability for using it as a basis for its intended purpose, the social reference frame being a combination of the views of the society at large, codes of the particular discipline, and the codified practices of a specific policy-making outfit. Stake sees a rather limited role for case study:
Case study is part of scientific method, but its purpose is not limited to the advance of science. Whereas single or a few cases are a poor representation of a population of cases and poor grounds for advancing grand generalization, a single case as negative example can establish limits to grand generalization. For example, we lose confidence in the generalization that a child of separated parents is better off placed with the mother when we find a single instance of resultant injury. Case studies are of value in refining theory and suggesting complexities for further investigation, as well as helping to establish the limits of generalizability. (Stake, 1994, p. 245)
Ethnographers, on the other hand, have rather more confidence. Lacey (1970) enlists the anthropologist Frankenberg to make the point that far from being complementary, the study of an individual case can be made to illuminate the general concerns of an entire population:
Though I am presenting a case study of one school, its significance is not confined to the particularistic concerns of this one school. It extends to general problems in sociology and education. I agree with Frankenberg when he argues that an essential ingredient of the social sciences is ‘...a methodology in which the discussion of small segments of society in great detail is used to throw light on the general.’ He continues, ‘It is my firm view that only the particularistic can illuminate the universalistic.’ (Lacey 1970 p. xi)9
The question, again, is not whether we can be confident in the case study as a genre of research, but rather what sources of confidence can we identify in all the individuals involved. This is where most of Stenhouse’s interest in illuminative research has its roots.10 The utility of case study research (be it ethnographic or historical or holistic portrayal) can be illustrated on the subject of the student experience of university life. While surveys and other census-like data abound and are frequently used in denouncing the quality of today’s tertiary education, deeper understanding will remain elusive until we have a careful look at the four major ethnographic case studies (Becker et al. 1961, Moffatt, 1989, Holland and Eisenhart 1990, Nathan, 2005) dealing with the university life. Unlike many of the quantitative surveys, these case studies provide sufficient information to make better generalizations about the subject at hand, particularly when it comes to situated judgment and practitioner action Both Moffatt and Nathan employ a wide range of approaches to gathering data including participant observation, guided interviews, collection of artefacts, self-administered surveys, diaries, etc. Their work shows the great potential of case study research and provides a cautionary tale to large-scale probabilistically sampled studies whose ultimate result is the telling of case-like stories that are substantiated by not much more than the researcher’s imagination. In particular, Nathan’s revelations about the relegation of knowledge acquisition in the social lives of students to a ritualistic incantation (‘I have to study’, ‘This is a difficult course’ vs. ‘I found out that…’, ‘That course discusses…’) cannot be revealed by a survey and yet it demonstrates universal features of the student experience through its depth and thickness of description. Research by Moffatt, Becker and others from previous decades can serve as replication and a basis for predictive generalization that does not rely on statistical sampling. Interestingly, Nathan (2005) and Becker et al. (1961) offer suggestions for policy decisions based on their research that are remarkably similar despite the gulf of nearly half a century. By coincidence, Nathan’s work allows us to compare case study with journalistic accounts. In the same year, another treatment of student culture on campus appeared entitled ‘Binge’ (Seaman 2005). It was the result of investigative reporting done across US campuses by a retired journalist and on the surface provided descriptions of student activities similar to Nathan’s (albeit generally with an unfavourable tinge). It could even be argued that since Seaman covered multiple campuses where Nathan only focused on one, it would be his account that provides the most generality. However, a close reading of both works makes it clear that it was Nathan’s painstaking observation that provides insight into the situational complexity of student life whereas Seaman’s volume only serves to establish stereotypes. The question remains, of course, on what criteria we can base this conclusion. Should there not be an objective measure of which piece of research is insightful and which superficial? The answer is no. Conclusions such as this one can only be based on the practice of reading and the situated practice of inquiry. In other words, they need to be made by a particular practitioner in the context of his or her work. An epistemological feature analysis of both books would leave the difference obscured.
DEFENDING THE CASE STUDY
Despite all this, case-study based research has been on the defensive as a basis for a politically acceptable procedure for making policy decisions. Most recently this has taken on the guise of “evidence-based practice” as the only acceptable source of knowledge for policy makers. While there is little to be contested about the sentiment that policy makers should base their decisions on the available evidence, the domain of medicine, upon which this implicit metaphor is based, often serves as an inadequate analogue to the problems faced by education and other social domains. This is further compounded by the fact that the realities of evidence-based practice in medicine are often poorly understood by the most vocal exponents of the ‘evidence-based’ mantras. This was illustrated by Lerner’s (2006) description of the role celebrity illnesses play in shaping how funding for research is allocated and how patients’ knowledge of ailments of the famous interacts with the decisions made by medical practitioners. Škrabánek’s and McCormick’s (1989) Follies and fallacies in medicine can hardly inspire more confidence, demonstrating that much of what counts as evidence in medicine is transmitted through tradition rather than the disembodied scientific method of hypothesis falsification the field presents as its mainstay. The medical metaphor of research-based policy is also problematic because on closer inspection many of the assumed correspondences don’t hold up when applied to the lived experiences of practitioners.
The idea that teachers and educational policy makers can behave like physicians in basing their decisions on the knowledge of latest available evidence is doomed to failure from the start. Apart from what we know about the differences in the stability and individual transmissibility of sociological versus medical knowledge, both these types of practitioners operate in markedly different contexts. Evidence-based reasoning in medicine is based partly on the fact that the very training of medical practitioners is from the very start focused on teaching future doctors how to use research results (knowing the transitory nature of much medical knowledge). Furthermore, physicians are expected to keep abreast of the latest developments in their field either through self-study, conference attendance, and, not insignificantly, advertising from the pharmaceutical industry. Not one of these elements is consistently present in the context of education.
The ‘educational practitioner is medical practitioner’ metaphor brings up other undesirable inferences. One of them is the idea of the researcher as a provider of information, to whom the practitioners relate as clients. However, given what we know about the kinds of reasoning really available to policy makers, it is probably necessary to stop thinking of them as clients of the research profession but rather to start thinking about them as researchers themselves. This may not be possible without first unpicking the industrial production metaphor based on the image of individual lines supplying parts that are then completed into a product. In line with this metaphor, policy-making bodies tend to structure themselves as modular production units and appointing individuals or departments to be “responsible for research”. But what this metaphor does not provide as readily, is a mechanism for ensuring that each and every policy-maker is engaged with research on a level much deeper than the occasional enquiry as to the well-being of the “research guy(s)”. To achieve this, it is necessary to convince policy-makers that they can be researchers in the same way that teachers or students can be researchers of their own practice. This may be problematic because of the imagery (frames) associated with the concept of research, many of which are built around the structural necessities of publication and prestige building necessary for survival in the academic world.
However, teachers and policy-makers can be researchers without engaging in activities that appear research-shaped on the surface (reflective practitioners in the original sense; not as people who think about their practice but those who construct situational knowledge equivalent to academic knowledge). It can be said that the way they do research in fact constitutes practical philosophy (a disciplined inquiry into knowledge) that should, in fact, be the sole basis for assessing the reliability of any kind of research results about to be put into the service of justifying policy decisions.
POLICY MAKING AS PRACTICE
One of the features that has characterized our joint inquiry is an essential commitment to the continual questioning of and challenge to what we refer to as instrumental rationality. Streamlined rational judgment is often, and almost always in the context of policy-making, a convenient fiction, a ritual of justification, that is trying to ignore, or worse, disguise the complexities involved in situational judgment. We now realize that the design of the project, the merits of which we perhaps insufficiently questioned at the outset but which in many respects we are still prepared to defend, tends to reinforce the idea of an independent rational agent, the philosopher, who can swoop in onto the policy-making process in some consultative role and point out “irrationalities” of this or that policy. It is not, however, the philosopher’s role to function as a vizier to the policy-maker’s sultan. If philosophy has any role in the process of policy-making, it is through setting an example of a disciplined inquiry – in which a policy-maker as researcher11 can find direct relevance to their practice. It is important, we feel, to look at the process of policy making in its own terms, doing justice to its complexities, rather than just assuming an impartial “rational” robotic process. It appears that the following words are as true today as they were a quarter of a century ago.
“Despite the considerable volume of work on the problematics of policy making and implementation, educational researchers are prone to underrate the constraints on policy makers which limit their ability to respond to research findings.” (Nisbet and Broadfoot, 1980, p. 56)
If Nisbet and Broadfoot are correct in assuming that “[t]here is as much of a gap between policy and practice as there is between research and policy or research and practice.” (ibid., p. 57), we must focus our inquiry on policy-making as a practice in its own right that, as all practice does, generates its own knowledge and makes its own generalizations irrespective of the data research presents to it. And as such it can be subject to the same dynamic interrelation between action and reflection as any other professional activity. Nutley et al. (2007) offer an overview of research utilization and reflections on the complexity of this relationship are also available in Pole and Morrison (2003) and Hammersley (2002). It is clear that policy making needs to be treated as an area of practice with its own situated system of ethical action that is distinct from the practice of research and the practice of educators.
This is important given that each practice generates its own generalizations but only some of these provide us with universal insights such as those suggested above by Simons, Dunne, Lacey, and Nussbaum. This distinction is crucial whenever a practitioner asks the philosopher of the epistemological warrant of his or her generalizations. Generalizations are generated within a particular discipline (area of practice) and apply to data as construed by that discipline. Universal insights can be inferred from outside that particular discipline. However, they are always made by individuals situated in their own context of need, desire and requirement. Whereas the general can be subject to instrumental disciplinary rules, the universal is a human act for which no analogous normative ideal can be constructed. One way of viewing the interface between research and practice is then as a continuing conversation between the general and the universal.
SUMMARY AND CONCLUSIONS
Overall, we have informed our inquiry with three perspectives on case study. One rooted in ethnography and built around the metaphor of the understanding of cultures one is not familiar with (these are presented by the work of scholars like Becker, Willis, Lacey, Wallace, Ball, and many others). Another strand revolves around the tradition of responsive and democratic evaluation and portrayal represented by the work of scholars such as MacDonald, Simons, Stake, Parlett, and others. This tradition (itself not presenting a unified front) aims to break down the barriers between the researcher, the researched and the audience. It recognizes the situated nature of all actors in the process (cf. Kemmis 1980) and is particularly relevant to the concerns of the policy maker. Finally, we need to add Stenhouse’s approach of contemporary history into the mix. Stenhouse provides a perspective that allows the researcher to marry the responsibility to data with the responsibility to his or her environment in a way that escapes the stereotypes associated with the ‘pure forms’ of both quantitative and qualitative research. A comprehensive historian-like approach to data collection, retention and dissemination that allows multiple interpretations by multiple actors accounts both for the complexity of the data and the situation in the context of which it is collected and interpreted.
However, we can ask ourselves whether a policy-maker faced with examples of one of these traditions could tell these perspectives apart? Should it not perhaps be the lessons from the investigation of case study that matter rather than an ability to straightforwardly classify it as an example of one or the other? In many ways, in the policy context, it is the act of choice, such as choosing on which case study a policy should be based, that is of real importance.
In that case, instead of transcendental arbitration we can provide an alternative test: Does the case study change the prejudices of the reader? Does it provide a challenge? Perhaps our notion of comprehensiveness can include the question: Is the case study opening the mind of the reader to factors that they would have otherwise ignored? This reminds us of Gadamer’s “fusion of horizons” (“Understanding [...] is always the fusion of […] horizons [present and past] which we imagine to exist by themselves.” Gadamer 1975, p. 273).
However, this could be seen as suggesting that case study automatically leads to a state of illumination. In fact, the interpretation of case study in this way requires a purposeful and active approach on the part of the reader. What, then, is the role of the philosopher? Should each researcher have a philosopher by his or her side? Or is it necessary, as John Elliott has long argued, to locate the philosopher in the practitioner. Should we expect teachers and/or policy-makers to go to the philosopher for advice or would a better solution be to strive for philosopher teachers and philosopher practitioners? Being a philosopher is of course determined not by speaking of Plato and Rousseau but by the constant challenge to personal and collective prejudices.
In that case, we can conclude that case study would have a great appeal to the politician and policy maker as a practical philosopher but that it would be a mistake to elevate it above other ways of doing practical philosophy. In this, following Gadamer, we advocate an antimethodological approach. The idea of the policy maker as philosopher and policy maker as researcher (i.e. underscoring the individual ethical agency of the policy maker) should be the proper focus of discussion of reliability and generalization. Since the policy maker is the one making the judgement, the type of research and study is then not as important as a primary focus.
And, in a way, the truthfulness of the case arises out of the practitioners’ use of the study. The judgment of warrant as well as the universalizing and revelatory nature of a particular study should become apparent to anybody familiar with the complexities of the environment. An abstract standard of quality reminiscent of statistical methods (number of interviewees, questions asked, sampling) is ultimately not a workable basis for decision making and action although that does not exclude the process of seeking a shared metaphorical perspective both on the process of data gathering and interpretation (cf. Kennedy 1979, Fox 1982). Gadamer’s words seem particularly relevant in this context: “[t]he understanding and interpretation of texts is not merely a concern of science, but is obviously part of the total human experience of the world.” (Gadamer 1975, p. xi)
We cannot discount the situation of the researcher no more than we can discount the situation of the researched. One constitutive element of the situation is the academic tribe: “[w]e are pursuing a ‘scholarly’ identity through our case studies rather than an intrinsic fascination with the phenomena under investigation” warns Fox (1982, p.32). To avoid any such accusations of impropriety social science cultivates a ‘prejudice against prejudice’, a distancing from experience and valuing in order to achieve objectivity whereas the condition of our understanding is that we have prejudices and any inquiry undertaken by ‘us’ needs to be approached in the spirit of a conversation with others; the conversation alerts participants to their prejudices. In a sense, the point of conversation is to reconstruct prejudices, which is an alternative view of understanding itself.
It should be stressed however, that this conversation does not automatically lead to a “neater picture” of the situation nor does it necessarily produce a “social good”. There is the danger of viewing ‘disciplined conversation’ as an elevated version of the folk theory on ideal policy: ‘if only everyone talked to one another, the world would be a nicer place’. Academic conversation (just like any democratic dialectic process) is often contentious and not quite the genteel affair it tries to present itself as. Equally, any given method of inquiry, including and perhaps headed by case study, can be both constitutive and disruptive of our prejudices.
Currently the culture of politeness aimed at avoiding others’ and one’s own discomfort at any cost contributes to the problem. Can one structure research that enables people to the reflect about prejudices that they inevitably bring to the situation and reconstruct their biases to open up the possibility of action and not cause discomfort to themselves or others? We could say that concern with generalization and method is a consequence of academic discourse and culture and one of the ways in which questions of personal responsibility are argued away. Abstraction, the business of academia, is seen as antithetical to the process of particularization, the business of policy implementation. But given some of the questions raised in this paper, we should perhaps be asking whether abstraction and particularization are parallel processes to which we ascribe polar directionality only ex post facto. In this sense we can further Nussbaum’s (1990) distinction between generalization and universalization by rephrasing the dichotomy in the following terms: generalization is assumed to be internal to the data whereas universalization is a situated human cognitive and affective act.
A universalizable case study is of such quality that the philosopher policy maker can discern its relevance for the process of policy making (similarly to Stake’s naturalistic generalization). This is a different way of saying the same thing as Kemmis (1980, p. 136): “Case study cannot claim authority, it must demonstrate it.” The power of case study in this context can be illustrated by anecdotes from the field where practitioners had been convinced that a particular case study describes their situation and berating colleagues or staff for revealing intimate details of their situation, whereas the case study had been based on research of an entirely unrelated entity. In cases like these, the universal nature of the case study is revealed to the practitioner. Its public aspects often engender action where idle rumination and discontent would be otherwise prevalent. Even when this kind of research tells people what they already “know”, it can inject accountability by rendering heretofore private knowledge public. The notion of case study as method with transcendental epistemology therefore cannot be rescued even by attempts like Bassey’s (1999) to offer ‘fuzzy generalizations’ since while cognition and categorization is fuzzy, action involves a commitment to boundaries. This focus on situated judgement over transcendental rationality in no way denies the need for rigour and instrumentalism. We agree with Stenhouse that there needs to be a space in which the quality of a particular case study can be assessed. But such judgments will be different when made by case study practitioners and when made by policy-makers or teachers. The epistemological philosopher will apply yet another set of criteria. All these agents would do well to familiarize themselves with the criteria applied by the others but they would be unwise to assume that they can ever fully transcend the situated parameters of their community of practice and make all boundaries (such as those described by Kushner 1993) disappear.
Philosophy often likes to position itself in the role of an independent arbiter but it must not forget that it too is an embedded practice with its own community rules. That does not mean there’s no space for it in this debate. If nothing else, it can provide a space (not unlike the liminal space of Turner’s rituals) in which the normal assumptions are suspended and transformation of the prejudice can occur. In this context, we should perhaps investigate the notion of therapeutic reading of philosophy put forth by the ‘New Wittgensteinians’ (see Crary and Read 2000).
This makes the questions of ethics alluded to earlier even more prominent. We propose that given their situated nature, questions of generalization are questions of ethics rather than inquiries into some disembodied transcendent rationality. Participants in the complex interaction between practitioners of education, educational researchers and educational policy makers are constantly faced with ethical decisions asking themselves: how do I act in ways that are consonant with my values and goals? Questions of warrant are internal to them and their situation rather than being easily resolvable by external expert arbitration. This does not exclude instrumental expertise in research design and evaluation of results but the role of such expertise is limited to the particular. MacDonald and Walker (1977) point out the importance of apprenticeship in the training of case study practitioners and we should bear in mind that this experience cannot be distilled into general rules for research training as we find them laid out in a statistics textbook.
Herein can lay the contribution of philosophy: An inquiry into the warrant and generalization of case study should be an inquiry into the ethics surrounding the creation and use of research, not an attempt to provide an epistemologically transcendent account of the representativeness of sampled data.
Correspondence:
John Elliott, School of Education and Lifelong Learning, University of East Anglia, Norwich NR4 7TJ, UK
Dominik Lukeš, School of Education and Lifelong Learning, University of East Anglia, Norwich NR4 7TJ, UK
NOTES
1 It should be noted that words like ‘confidence’ and ‘warrant’ have their instrumental counterparts in lexemes like ‘scientific’ and ‘verification’. However, although they are intended to provide a hedge, the underlying assumption frequently is that “things” will eventually be clarified in a properly instrumental manner.
2 It would appear that Stenhouse’s vision of illuminative research has survived most prominently in the arena of programme evaluation.
3 Also, it is to a large extent an empirical question. How people acquire and retrieve knowledge is a question that can be asked at a cognitive level. Unfortunately, at the moment the cognitive and neural sciences can only provide a very sketchy image of how knowledge (indeed questions as to the nature of knowledge itself abound) is represented. However, there are at least some indications that the human brain is not reminiscent of a database in which knowledge is stored and neither is the “retrieval” and application of knowledge in a given situation driven primarily and exclusively by algorithmic processes of feature matching underwritten by the formal laws of logic (see e.g. Lakoff, G. (1987) Women, fire, and dangerous things: What categories reveal about the mind, (Chicago, University of Chicago Press).).
Kuhn (Kuhn, T. S. (1996 [1962]) The structure of scientific revolutions, (3rd ed. edn) (Chicago; London, University of Chicago Press).) reminds us that the forgetting of important knowledge is an important part of paradigm shifts. Scientific knowledge, far from being straightforwardly cumulative, is embedded in its present situation of theory, method and social structure. This is equally true of the social sciences and humanities.
See for example a proposal to the ESRC by Carmichael and Elliott (2007) entitled Learning from and about Case study: Developing a Teaching Archive for Educational Research to be based on an initial archive of case study-based evaluation studies developed as part of the ESRC QUADS Initiative and entitled Representing Context in a Research Archive of Educational Evaluation Studies, Re-346-25-3003.
There is a further disturbing element here. Not only do sample-based studies fail to account for much of the particular knowledge that could be used in practice, they often inform the policy-maker of properties that can be ascribed to populations alone (i.e. as specific configurational features of a given cohort) but that may not be “true” in any meaningful sense of any individual in a single situation. This will diminish their utility for policy-making even further.
Note that Stake is making an essentially empirical claim about the nature of memory and the kinds of “narrative” experiences that can trigger memories and connect them to other structures of knowledge “awareness and understanding”. However, not only does Stake not provide evidence that that is the case, he leaves it up to his readers’ “imaginations” to conclude that non-case study research is not as good at providing opportunities for “vicarious experience”.
This is intensified by an English discursive convention to translate statistical data into the “X in Y” formula rather than reporting percentages as is more common for instance in Czech.
R. J. Frankenberg quoted from 1963 paper presented to the British Association titled “Taking the blame or passing the buck”. Reference to published work not provided by Lacey.
Stenhouse’s attempt at analogy of case study with history as opposed to ethnography has its justification but a careful study of historiography shows that just as often as not the historical record itself needed to be reexamined and/or deconstructed. On the other hand, the lack of detailed notes did not prevent Mead’s or Whorf’s hypotheses about other cultures to be questioned.
Similarly to the notion of ‘teacher as researcher’.
REFERENCES
Adelman C, Jenkins D, & Kemmis S (1976), Rethinking Case Study: Notes From The Second Cambridge Conference, Cambridge Journal of Education, Vol. 6, No.3. Reprinted in Simons, H (1980) pp. 47-61 (See Below).
Adelman, C. (Ed) (1984) The politics and Ethics of evaluation (London, Croom Helm).
Ball, S.J. (1981) Beachside Comprehensive: A case study of secondary schooling (Cambridge & New York, Cambridge University Press).
Bassey, M. (1999) Case study research in educational settings (Buckingham, England; Philadelphia, Open University Press).
Becker, H. S. (1992) Cases, causes, conjunctures, stories and imagery, in C. C. Ragin & H. S. Becker (Eds) What is a case?: Exploring the foundations of social inquiry ( Cambridge & New York, Cambridge University Press), pp. 205-227.
Becker, H. S., Geer, B., Hughes, E. C. & Strauss, A. L. (1977 [1961]) Boys in white: Student culture in medical school (New Brunswick {N.J},Transaction Books).
Carr, W. (2006) Philosophy, methodology and action research. Journal of Philosophy of Education, 40(4), 421–435.
Crary, A. M. & Read, R. J. (Eds.) (2000) The New Wittgenstein (New York, Routledge).
Cronbach, L (1963) Course improvement through evaluation Teachers’ College Record, 64, pp.672-683.
Elliott, J. Bridges, D. Ebbutt, D. Gibson, R. Nias, J. Case Studies in School Accountability, Vols. 1-3 (Cambridge: Cambridge Institute of Education) (some are now accessible through a Digital Archive of Contextualised Case Studies, http://www.caret.cam.ac.uk/tel).
Elliott, J. (1984) Methodology and Ethics, in C. Adelman (Ed) The politics and ethics of evaluation (Beckenham {UK} Croom Helm).
Elliott, J (1990) Validating Case Studies, Westminster Studies in Education, Vol.13.
Elliott, J. & Doherty, P. (2001) Restructuring educational research for the ‘third way’, in: M. Fielding (Ed.) Taking education really seriously, four years’ hard labour (London & New York, RoutledgeFalmer), pp. 209 – 222.
Elliott, J. (2007) Educational theory, practical philosophy and action research, in Reflecting where the action is: The selected works of John Elliott (Oxford and New York, Routledge), Ch. 6.
Elliott, J & Kushner, S. (2007) The need for a manifesto for educational evaluation, Cambridge Journal of Education, pp. 321-336.
Evers, C.W. & Wu, E.H. ( 2006) On Generalising from Single Case Studies: Epistemological Reflections, Journal of Philosophy of Education. Vol. 40, Issue 4.
Fisher, K. (1997) Locating frames in the discursive universe. Sociological Research Online.
Fox, T. G. (1982) Applying mathematics to the study of educational cases, Mimeo (Madison, University of Wisconsin).
Gadamer, H. G. (1975) Truth and method (London, Sheed and Ward)
Gamson, W. A. (1992) Talking politics, (Cambridge & New York, Cambridge University Press).
Goffman, E. (1975 [1974]) Frame analysis: An essay on the organization of experience, Harmondsworth, UK: Penguin).
Hamilton, D., Jenkins, D., King, C., MacDonald, B. & Parlett, M. (Eds.) (1977) Beyond the Numbers Game (Basingstoke & London, Macmillan Education).
Hammersley, M. (2001) On Michael Bassey's concept of the fuzzy generalisation. Oxford Review of Education, 27(2), pp. 219-225.
Hammersley, M. (2002) Educational research, policymaking and practice (London, PCP).
Hargreaves, D (1997) In defence of research for evidence-based teaching: a rejoinder to Martyn Hammersley, British Educational Research Journal, 23,pp.405-419.
Hoare, Q & Nowell Smith, G (Eds and Transl’s.) (1971) The Modern Prince, Selections from
the Prison Notebooks of Antonio Gramsci (London, Lawrence & Wishart).Holland, D. C. & Eisenhart, M. A. (1990) Educated in romance: Women, achievement, and college culture (Chicago, University of Chicago Press)
Kemmis, S. (1980) The imagination of the case and the invention of the study, in: H. Simons (Ed.) Towards a science of the singular: Essays about case study in educational research and evaluation *(*Norwich, Centre for Applied Research in Education, University of East Anglia), pp. 96-142.
Kennedy, M. M. (1979) Generalizing from single case studies. Evaluation Review, 3(4), pp. 661-678.
Kennedy, M. M. (1999) A test of some common contentions about educational research. American Educational Research Journal, 36(3), pp. 511-541.
Kuhn, T. S. (1996 [1962]) The structure of scientific revolutions (3rd ed. edn) (Chicago & London, University of Chicago Press).
Kushner, S. (1993) Naturalistic evaluation: Practical knowledge for policy development, Research Evaluation, 3(2), pp. 83-94.
Lacey, C. (1970) Hightown Grammar: the school as a social system (Manchester, Manchester University Press).
Lakoff, G. (1987) Women, fire, and dangerous things: What categories reveal about the mind (Chicago, University of Chicago Press).
Lakoff, G. (1996) Moral politics: What conservatives know that liberals don't Chicago, University of Chicago Press).
Lerner, B.H. (2006) When illness goes public: celebrity patients and how we look at medicine (Baltimore, Johns Hopkins University Press).
Lloyd, J. (2007) An intellectual in power, Prospect, 2 Bloomsbury Place, London wc1a 2QA. pp.24-28.
Lukeš, D. (2007) What does it mean when texts ‘really’ mean something?: Types of evidence for conceptual patterns in discourse, in C. Hart & D. Lukeš (Eds) Cognitive linguistics in critical discourse studies: Application and theory (Newcastle, Cambridge Scholars Publishing).
MacDonald, B & Parlett, M. (1973) ‘Rethinking evaluation: notes from the Cambridge Conference’, Cambridge Journal of Education, vol.2, pp.74-81
Macdonald, B. (1976) Evaluation and the control of education, in: D. Tawney (Ed.) Curriculum evaluation today: Trends and implications (London, Schools Council Research Studies, Macmillan Educational).
Macdonald, B. & Walker, R. (1975) Case-study and the social philosophy of educational research, in the Cambridge Journal of Education (5) pp.2-11; Reprinted in Hamilton, D. Jenkins, D. King, C. Macdonald,B & Parlett, M. (Eds) (1977) Beyond the numbers game: A reader in educational evaluation (London, Macmillan Education), pp. 181-189.
May, R (1994) The Courage to Create (New York:W.W. Norton & Co).
Nathan, R. (2005) My freshman year: what a professor learned by becoming a student, (Ithaca, Cornell University Press).
Nisbet, J.D. & Broadfoot, P (1980) The impact of research on policy and practice in education, (Aberdeen, Aberdeen University Press).
Nussbaum, M. C. (1990) Love's knowledge: Essays on philosophy and literature (Oxford, Oxford University Press).
Nutley, S. M., Walter, I. & Davies, H. T. O. (2007) Using evidence: How research can inform public services (Bristol, The Policy Press).
Parlett, M & Hamilton, D (1972) Evaluation as Illumination: a new approach to the study of innovatory programmes, Occasional Paper 9, Centre for Research in the Educational Sciences, University of Edinburgh, Reprinted in Hamilton, D. Jenkins, D. King, C. Macdonald, B & Parlett, M. (Eds) (1977) Beyond the Numbers Game: A Reader in Educational Evaluation (London, Macmillan), pp*.*6-22.
Parlett, M. & Dearden, G. (Eds.) (1977) Introduction to illuminative evaluation: Studies in higher education (Cardiff-by-the-Sea {Calif} Pacific Soundings Press).
Pole, C. & Morrison, M. (2003) Ethnography for education (Buckingham, Open University Press).
Robinson, J. E. & Norris, N. F. (2001) Generalisation: The linchpin of evidence-based practice? Educational Action Research, 9(2),pp. 303-310.
Schön, D.A. & Rein, M, (1994) Frame Reflection: toward the resolution of intractable policy controversies (New York, Basic Books).
Schwab, J. J. (1970) The practical: A language for curriculum (Washington {D.C.},National Educational Association, Centre for the Study of Instruction).
Schwandt, T. A. (2005) A diagnostic reading of scientifically based research for education. Educational Theory, 55(3), pp. 285-305.
Seaman, B. (2005) Binge: What your college student won't tell you: Campus life in an age of disconnection and excess (Hoboken {N.J.}, John Wiley & Sons).
Simons, H. (Ed) (1980) Toward a Science of the Singular: Essays about Case Study in Educational Research and Evaluation, CARE Occasional Publications No.10, (Norwich, CARE/ University of East Anglia).
Moffatt, M. (1989) Coming of age in New Jersey: college and American culture (New Brunswick, Rutgers University Press).
Simons, H. (1987) Getting to Know Schools in a Democracy: the politics and process of evaluation (London, New York & Philadelphia:,The Falmer Press), Ch.3.
Simons, H. (1996) The paradox of case study. Cambridge Journal of Education, 26(2), pp. 225-240.
Stake, R.E. (1978) The case study method in social inquiry, Educational Researcher, 7, pp.5-8. Reprinted in Simons, H (Ed) Towards a Science of the Singular, CARE Occasional Publications 10, pp. 64-67 (Norwich: Centre for Applied Research in Education, University of East Anglia).
Stake, R.E. (1994) Case Studies, in Denzin, N. & Lincoln Y. (Eds) Handbook of Qualitative Research (Newbury Park {CA,} Publications).
Stake, R.E (1995) The Art of Case Study Research (London & Thousand Oaks {California}& New Delhi, Sage Publications).
Stenhouse, Lawrence. (1978) Case study and case records: towards a contemporary history of education, British Educational Research Journal, vol. 4 (2) pp21-39
Stenhouse, Lawrence. (1979a) The problem of standards in illuminative research. A lecture given to the Scottish Educational Research Association at its Annual General Meeting, Stenhouse Archive, Norwich: University of East Anglia, http://research.edu.uea.ac.uk/stenhousearchive.
Stenhouse, Lawrence.(1979b) The study of samples and the study of cases. Presidential Address to the British Educational Research Association, Stenhouse Archive, Norwich: University of East Anglia, http://research.edu.uea.ac.uk/stenhousearchive. Published version (1980) in British Educational Research Journal, Vol.6, No. 1. pp. 1 - 6.
Stenhouse, Lawrence (1979 c) Case Study in Comparative Education: particularity and generalization, Comparative Education, Vol.15, No.1. pp.5-10.
Stenhouse, Lawrence (1985) A Note on Case Study and Educational Practice, in R. Burgess (Ed) Field Methods in the Study of Education, London & Philadelphia: The Falmer Press.
Škrabánek, P. & McCormick, J. (1989) Follies and fallacies in medicine (Glasgow, Taragon).
Willis, P.E. (1977) Learning to labour: How working class kids get working class jobs, (Farnborough {Eng}, Saxon House).
Wallace, W.L. (1966) Student Culture: Social structure and continuity in a liberal arts college (Chicago, Aldine Publishing Company).