Chapter 3 Draft
4-Phase EMPNOS
2-21-07
Erin E. Peters
Research
Questions
Student understanding of the nature of science helps students to learn that science is more than a collection of facts. Learning and applying the aspects of the nature of science helps students to see and think about their world using a scientific way of knowing. Developing metacognitive skills should also be an explicit activity in the science classroom. Students using metacognitive skills can evaluate their thinking to determine if it aligns with the rigorous requirements of science. This study uses an intervention (4-Phase EMPNOS) to find out if students can be taught to think scientifically on a metacognitive level and seeks to answer the following questions: (a) What is the effect of 4-Phase EMPNOS on science students’ content knowledge, knowledge about the nature of science, metacognition, and self-regulatory efficacy? It is hypothesized that students exposed to the intervention would report a higher level of content and nature of science knowledge, metacognition and self-regulatory efficacy. (b) How are the specific constructs of science content knowledge, knowledge about the nature of science, metacognition, and self-regulatory efficacy related to each other when students complete activities with embedded metacognitive prompts? It is hypothesized that science content knowledge and knowledge about the nature of science are positively correlated and that knowledge about the nature of science, metacognition and self-regulatory efficacy are positively correlated. (c) What characterizes the shared experiences of students who use 4-Phase EMPNOS and students who do not use 4-Phase EMPNOS? and (d) In what ways do students approach activities with embedded metacognitive prompts and activities without metacognitive prompts?
Methods
Sample
Three
hundred and eight eighth-grade science students from an urban middle school in
the mid-Atlantic region of the
Four classes were used for this quasi-experimental study. Two classes were given an intervention that has embedded metacognitive prompts based on the nature of science and were called the experimental group. Two classes were given an intervention that does not include the metacognitive prompts and were called the control group. The students are already formed into classes, so the members of the groups were not randomly selected. However, the classes will be randomly selected as either experimental or control.
Measures
Quantitative
Measures
Metacognitive Orientiation Scale (MOLES-S).
The
Metacognitive Orientation Scale (Thomas, 2002b) is designed with a social
constructivist view in mind and considers that knowledge is not constructed in
a vacuum, but is developed through interactions with the learning environment.
Thomas (2002a) argues that most measures in the science classroom regarding
metacognition involved lengthy interviews and observations and that the
development of a large-scale measure of metacognition in the classroom would be
useful. Eight aspects of metacognition which were supported by the research
literature were measured on the MOLES-S: (1) metacognitive demands, (2) teacher
modeling and explanation, (3) student-student discourse, (4) student-teacher
discourse, (5) student voice, (6) distributed control, (7) teacher
encouragement and support, and (8) emotional support. The MOLES-S is a 67-item
instrument that includes the eight aforementioned dimensions based on a Likert-scale. The initial instrument was administered to
1026 students within the 14-17 year old age group. At the time the instrument
was administered,
Metacognition of Nature
of Science Scale (MONOS).
The MONOS (Peters, in press) 16-item survey was designed to test five different student perceptions: a) attitude about the subject of science, b) use of metacognition in observation, c) use of metacognition in data collection, d) use of metacognition in measurement, e) ability to explain reasoning in making conclusions. Each of the topics was chosen because they exemplify skills that are valuable in teaching science as a way of knowing.
Students were asked to choose a number between 1 and 5 to show whether they agreed with the statement (5) or disagreed with the statement (1). Multiple questions were designed to test the same variable so that instrument subscale reliability could be verified. Questions 1, 3 and 8 tested student attitudes toward science. Questions 2, 4 and 11 tested student perception of ability to have metacognition about observations. Questions 7 and 16 tested student perception of metacognitive ability in measurement. Questions 5, 6, 9 and 15 measured student perception of metacognitive ability in data collection. Questions 10, 12, 13 and 14 measured student perceived ability to reason when making conclusions. A copy of the survey can be found in Appendix 2.
Field tests of the survey were conducted with three high achieving, three average achieving and three low achieving readers from the eighth grade. Feedback regarding comprehension and meaning of the questions provided during the field test interviews after the survey guided the revisions of the instrument. Changes in the statements were made based on the interviews of the students after the draft survey was administered. The students involved in the field test did not take the survey, since they had prior knowledge of the intention of the survey.
Reliability as measured by alpha test for the entire instrument is .89. Subscales were also tested for reliability using the alpha test. The subscale for observation items is .43. The subscale for measurement items is .60. Items that measure metacognition for data collection had a reliability of .62. Items that measure metacognition for attitude had a reliability of .62. The items that tested the ability to explain reasoning in concluding had an alpha test of .71
Self-efficacy for Learning Form (SELF).
The SELF scale (Zimmerman & Kitsantis, 2005) is a 19-item survey designed to test student self-efficacy for learning. The items ask students to determine their ability to complete self-regulated learning strategies on a percentage scale divided into increments of ten percent. It is designed to have students self-report on a variety of situations that require academic self-regulatory efficacy such as reading, note taking, test taking, writing, and studying. High scores on this scale represent a high ability to be self-regulatory in academic strategies. This scale has a reliability coefficient of .97 and was highly correlated to teacher reports on students.
Qualitative
Measures
Test of Electricity-Magnetism Knowledge (TEMK)
The science content taught during the intervention includes magnetism, static electricity, current electricity, and electromagnetism. The TEMK (Peters, unpublished) assesses each students’ attainment content goals at an eighth grade level: (a) behavior of static electrical charges, (b) behavior of electrical current, (c) behavior and internal mechanisms of magnets, and (d) behavior of electromagnetic interactions. The questions on this test are open-ended and assess each of the content goals using visual, logical and analytical forms of communication. Each test will be analyzed for strengths and weaknesses in particular content areas, themes in the way students answer questions, and themes in the way students design scientific products such as data tables or observations. The content test will be administered before and after the intervention.
The Views of the Nature of
Science- Form B (VNOS –B).
The
VNOS-B (Lederman, Abd-El-Khalick,
Bell & Schwartz, 2002) assesses student understanding of science as a way
of knowing and consists of seven open-ended questions corresponding to the
seven identified aspects of the nature of science: (a) scientific knowledge is
durable, yet tentative, b) empirical evidence is used to support ideas in
science, c) social and historical factors play a role in the construction of
scientific knowledge, d) laws and theories play a central role in developing
scientific knowledge, yet they have different functions, e) accurate record
keeping, peer review and replication of experiments help to validate scientific
ideas, f) science is a creative endeavor, and g) science and technology are not
the same, but they impact each other (McComas,
2004).
Lederman, Abd-El-Khalick,
Student products from
inquiry units.
The control group and the experimental groups will have identical inquiry units that contain identical science content and science process skills to master. Within the inquiry unit, students will answer science content questions and use science process skills to make conclusions about the phenomena. The student products will be analyzed using the same protocol as the content test: strengths and weaknesses in particular content areas, themes in the way students answer questions, and themes in the way students design scientific products such as data tables or observations.
Teacher memos.
Memos are a versatile tool used to in many ways such as helping researchers reflect on events that are occurring during the research study or documenting confusing events for later analysis (Maxwell, 2005). Complications could arise during the interpretation phase of data analysis due to the dual role of teacher and researcher. Memos could help to reduce the confusion in interpretation because they will discuss implicit events during the research study. Memos will be written throughout the research study and then coded for emergent themes.
Think Aloud Protocol.
After the interventions six students will be randomly chosen from the control group and six students will be randomly chosen from the experimental group and videotaped separately while they perform an investigation from the intervention. Students will be asked to think aloud during the videotape in order to elicit their thinking processes during a scientific investigation. Since eighth grade students have little experience in expressing their “inner voices”, an established protocol to encourage three levels of verbal reports will be used, verbalization of covert encodings, explication of thought content, and explanations of thought processes (Ericsson & Simon, 1993). Students will be instructed to talk aloud about what they are thinking, and not to explain the answer to the problem. Students will be prompted at key points throughout the think aloud to continue their explanation of what they are thinking. The frequency of each level of verbal report will be reported as well as the themes that emerge from each level.
Focus Group Interviews.
After the intervention, six members will be randomly chosen from the experimental group and six members will be chosen from the control group to participate in a focus group. A focus group was chosen as a method of data collection rather than individual interviews because eighth grade students tend to minimize interactions with adults. A focus group will elicit more rich verbal data from the students because they will interact with each other and expand each others’ ideas. The questions are semi-structured because they focus the conversation without giving up the freedom that may be needed to explore phenomena that emerges. Sample questions are (a) What was the topic of your last science class? (b) How did you think like a scientist in that lesson? (c) How did you act like a scientist in that lesson? (d) How do you think science class is different from English, history or math class? (e) How can you think about your thinking? (f) What does it mean to you to think like a scientist? (g) Are there other ways of thinking? (h) Do scientists behave differently than other people? Focus group conversations will be audio-taped and transcribed using the software, Transana.
Intervention
The intervention, 4-Phase Embedded Metacognitive Prompts based on the Nature of Science (4-Phase EMPNOS), consists of four modules that cover the content of electricity and magnetism at an eighth grade level. Each module is based on inquiry methods (NRC, 1996) and asks students to make observations and inferences about phenomena. Module one investigates behaviors of permanent, ceramic magnets. Module two investigates phenomena involved with static electricity. Module three investigates models that explain current electricity. Module four investigates series and parallel circuits. Module five investigates electric and magnetic interactions. Module six investigates the historical context of the discovery of motors. Module seven investigates the social implications of motors, generators and transformers. Each of the experimental modules includes nature of science metacognitive prompts from one of the seven aspects of the nature of science: a) scientific knowledge is durable, yet tentative, b) empirical evidence is used to support ideas in science, c) social and historical factors play a role in the construction of scientific knowledge, d) laws and theories play a central role in developing scientific knowledge, yet they have different functions, e) accurate record keeping, peer review and replication of experiments help to validate scientific ideas, f) science is a creative endeavor, and g) science and technology are not the same, but they impact each other (McComas, 2004). Metacognition is developed throughout the units based on Zimmerman’s (2000) model of the 4-phases of self regulation: observation, emulation, self-control and self-regulation. The observation phase prompts give examples of how a scientist would answer the question and the rationale behind it. The emulation phase gives a checklist to the student about the aspects of nature of science that should be considered for the task. The self-control phase gives students a short checklist of major aspects of the nature of science as well as some simple questions about “thinking about their thinking.” The self-regulation phase gives students more advanced questions that ask students to check to see if their thinking is aligned with the nature of science aspect. Each module has each of the four phases for one aspect of the nature of science. For example Module 1 for the experimental group is based on magnetism content and has four sections of embedded developmental phases of metacognitive prompts on the use of empirical evidence to support claims throughout the unit.
Procedures
Students from all four classes will be given the MOLES-S, the MONOS, the SELF, the VNOS-B, and the content test before the intervention begins. Classes will then be chosen randomly to be in the control group or the experimental group. The inquiry unit on electricity and magnetism without the embedded metacognitive prompts will be given to the control group and the inquiry unit on electricity with the embedded metacognitive prompts (4-phase EMPNOS) will be given to the experimental group. Students will proceed with the four modules of the intervention (the inquiry unit) with the guidance of the teacher/researcher. Student groups of four will use the intervention packets to investigate the learning goals in electricity and magnetism. The teacher/researcher will act as a facilitator in their learning process and will write reflective researcher memos throughout the intervention. After the second module is complete, all students will take the SELF survey. After all four modules are complete, students will take the MOLES-S, the MONOS, the SELF survey, the VNOS-B, and the content test. When students are finished the modules, all of their work products will be collected. Six students from the control group and six students from the experimental group will be randomly selected to participate in a think aloud by performing one investigation from the intervention while being coached to think aloud. The control group will perform the think aloud separately from the experimental group. Six students from the control group and six students from the experimental group will be randomly selected to participate in a focus group which is designed to elicit their shared experiences in the two different types of inquiry units.
Design
This quasi-experimental study is designed to show differences in content knowledge, knowledge of the nature of science, metacognition and self-regulatory efficacy between the control and experimental group. The MOLES-S, the MONOS, the SELF survey, the VNOS-B and the content test will be given as a pre- and post-test so that variances between the control and experimental can be analyzed. The SELF survey will also be given at the midpoint of the intervention to determine the pattern of the level of self-regulatory efficacy the students experience before, during and after the intervention. Researcher memos that were written throughout the intervention and student work products will be used to back up any inferences made with the pre- and post-test analysis. Focus group results, think aloud results, researcher memos, and student work will be used to determine the processes students used to achieve the measured outcomes.
Proposed
Data Analysis
Quantitative data will be gathered using the MOLES-S, the MONOS, and the SELF survey. The MOLES-S and the MONOS are Likert-scales and the SELF is a percentage scale. Combined, the scales will measure the constructs of metacognition, knowledge of the nature of science, and self-regulatory efficacy. These data will be first analyzed using MANOVA and then compared to the results of a MANCOVA analysis so that any covariates can be eliminated from the analysis. The VNOS-B will be analyzed using a rubric that determines the frequency of knowledge of the nature of science as well as the comprehensiveness of the knowledge of the nature of science. The content test will be analyzed for student comprehensiveness of the content goals as well as their knowledge of the nature of science. The focus group results will be analyzed for common experiences within the groups using a phenomenological stance and the processes that emerge from the common experiences will be reported. The think aloud results will be analyzed for the frequency of each of the three levels of verbal report discussed in the instrument section of this paper as well as for the processes that students use to achieve metacognition related to the nature of science. The researcher memos and student work products will be analyzed for common themes and for processes that students use to achieve metacognition related to the nature of science. All data sources will be catalogued in a matrix so that all data can be triangulated.