Reflections on Dr. Kelly’s
Guiding Principles Document
EDUC 805, Leadership Seminar, December 5, 2000

Unlike previous papers provided by guest speakers, Guiding Principles for Mathematics and Science Education Research Methods is a report of a two-day workshop conducted in 1998 by the Division of Research, Evaluation and Communication (REC) of the National Science Foundation. The purpose of the workshop was to stimulate discussion among researchers who engage in funded projects and produce a set of guiding principles for designing research studies and evaluating research proposals in the area of math and science education. The report describes the kinds of research that have obtained funding, reviews existing guidelines from selected research experiences, presents a set of guiding principles that expands upon those guidelines, and presents a vision of what is meant by high quality research in math and science education.

Although the area under discussion is math and science education, all education and social science researchers should welcome this report with a great Amen! For years, education and social science researchers have suffered from what is popularly called “physics envy” – the compulsion to conduct experimental and quasi-experimental research as conducted by researchers in the natural sciences. The report cites some recent criticisms of education research as failing to achieve high standards of scientific merit, with new technologies for data capture and analysis further muddying the research waters. However, the problem is not with the new technologies, but rather with the (mis)perception among education and social science researchers that good research – particularly funded research – must follow the “scientific” model of hypothesis-variable definition-hypothesis testing-examination of outcomes-rejection/acceptance of the original hypothesis. Robson (1993) notes that this kind of model with its orderliness and separation into a clear linear sequence is the stuff of methods textbooks and journal articles that of the scientific research process in practice. The report supports this view in the summary table describing the shift from positivism to interpretivism in math and science research.

The review of proposals funded between 1992 and 1997 reveals that nearly half of all the projects use both quantitative and qualitative methods, reflecting the multidisciplinary character of the research teams. Nevertheless, a review of existing standards shows distinct sets of standards for quantitative versus qualitative research. Further, the standards for qualitative research described in the report focus on ethnographic education research and address a very narrowly focused study in a specific environment with a specific set of subjects. This does little to change the perception that the only “true” research is the experiment.

The report does a solid job of putting to rest the myth of the “education researcher in lab coat” when it notes that random assignment of participants to “treatment” groups is nearly impossible in an educational setting because researchers do not have sufficient power to convince school administrators to maintain consistency in experimental and control groups. This makes pure experimentation all but impossible in education research.

In presenting the guiding principles for research proposals, the report states that what defines a quality proposal is a clear description of the issues to be addressed and the match between the approach taken and the ideas, outcomes, or models that the research is trying to explore. This is the same idea expressed by Locke, Silverman and Spirduso (1998) when they state that a particular strategy for inquiry is neither good nor bad in an absolute sense. A type of research is good or bad to the extent that it fits well or poorly with the question at hand.
What is especially valuable in the report are the guidelines for clarifying the nature of the research problem. By articulating the relevance to important socio-political research issues of the day, the contribution to the understanding of education, the value system underlying the research and linkage to future studies, the researcher is forced to be clear as to why he/she is doing the study in the first place. As noted by Maxwell (1996), without a clear sense of purpose, the researcher is bound to get lost or spend time and effort doing things that will not contribute to intended goals. As a researcher with strong positivist tendencies, my greatest fear is in quantifying the obvious. That is why the workshop report resonated with me and should resonate with all researchers who seek to address specific questions through research, not demonstrate their methodology skills.

References

Locke, Lawrence F., Silverman, Stephen J. & Spirduso, Waneen Wyrick (1998). Reading and Understanding Research. Thousand Oaks, California: Sage Publications.

Maxwell, Joseph A. (1996). Qualitative Research Design: An Interactive Approach. Thousand Oaks, California: Sage Publications.

Robson, Colin. (1993). Real World Research. Oxford, UK: Blackwell Publishers, Inc.