Erin E. Peters
Research
Questions
Student understanding of the nature of science helps students to learn that science is more than a collection of facts. Learning and applying the aspects of the nature of science helps students to see and think about their world using a scientific way of knowing. Developing metacognitive skills should also be an explicit activity in the science classroom. Students using metacognitive skills can evaluate their thinking to determine if it aligns with the rigorous requirements of science. This study uses an intervention (4-Phase EMPNOS) to find out if students can be taught to think scientifically on a metacognitive level and seeks to answer the following questions: (a) What is the effect of 4-Phase EMPNOS on science students’ content knowledge, knowledge about the nature of science, metacognition, and self-regulatory efficacy? It is hypothesized that students exposed to the intervention would report a higher level of content and nature of science knowledge, metacognition and self-regulatory efficacy. (b) How are the specific constructs of science content knowledge, knowledge about the nature of science, metacognition, and self-regulatory efficacy related to each other when students complete activities with embedded metacognitive prompts? It is hypothesized that science content knowledge and knowledge about the nature of science are positively correlated and that knowledge about the nature of science, metacognition and self-regulatory efficacy are positively correlated. (c) What characterizes the shared experiences of students who use 4-Phase EMPNOS and students who do not use 4-Phase EMPNOS? and (d) In what ways do students approach activities with embedded metacognitive prompts and activities without metacognitive prompts?
Methods
Sample
Three
hundred and eight eighth-grade science students from an urban middle
school in
the mid-Atlantic region of the
Four classes were used for this quasi-experimental study. Two classes were given an intervention that has embedded metacognitive prompts based on the nature of science and were called the experimental group. Two classes were given an intervention that does not include the metacognitive prompts and were called the control group. The students are already formed into classes, so the members of the groups were not randomly selected. However, the classes will be randomly selected as either experimental or control.
Measures
Quantitative Measures
Metacognitive Orientiation Scale (MOLES-S).
The
Metacognitive Orientation Scale (Thomas, 2002b) is designed with a
social
constructivist view in mind and considers that knowledge is not
constructed in
a vacuum, but is developed through interactions with the learning
environment.
Thomas (2002a) argues that most measures in the science classroom
regarding
metacognition involved lengthy interviews and observations and that the
development of a large-scale measure of metacognition in the classroom
would be
useful. Eight aspects of metacognition which were supported by the
research
literature were measured on the MOLES-S: (1) metacognitive demands, (2)
teacher
modeling and explanation, (3) student-student discourse, (4)
student-teacher
discourse, (5) student voice, (6) distributed control, (7) teacher
encouragement and support, and (8) emotional support. The MOLES-S is a
67-item
instrument that includes the eight aforementioned dimensions based on a
Likert-scale. The initial instrument was administered to 1026 students
within
the 14-17 year old age group. At the time the instrument was
administered,
Metacognition of Nature of
Science Scale
(MONOS).
The MONOS (Peters, in press) 16-item survey was designed to test five different student perceptions: a) attitude about the subject of science, b) use of metacognition in observation, c) use of metacognition in data collection, d) use of metacognition in measurement, e) ability to explain reasoning in making conclusions. Each of the topics was chosen because they exemplify skills that are valuable in teaching science as a way of knowing.
Students
were asked to choose a number between 1 and 5 to show whether they
agreed with
the statement (5) or disagreed with the statement (1). Multiple
questions were
designed to test the same variable so that instrument subscale
reliability
could be verified. Questions 1, 3 and 8 tested student attitudes toward
science. Questions 2, 4 and 11 tested student perception of ability to
have
metacognition about observations. Questions 7 and 16 tested student
perception
of metacognitive ability in measurement. Questions 5, 6, 9 and 15
measured
student perception of metacognitive ability in data collection.
Questions 10,
12, 13 and 14 measured student perceived ability to reason when making
conclusions.
Field tests of the survey were conducted with three high achieving, three average achieving and three low achieving readers from the eighth grade. Feedback regarding comprehension and meaning of the questions provided during the field test interviews after the survey guided the revisions of the instrument. Changes in the statements were made based on the interviews of the students after the draft survey was administered. The students involved in the field test did not take the survey, since they had prior knowledge of the intention of the survey.
Reliability as measured by alpha test for the entire instrument is .89. Subscales were also tested for reliability using the alpha test. The subscale for observation items is .43. The subscale for measurement items is .60. Items that measure metacognition for data collection had a reliability of .62. Items that measure metacognition for attitude had a reliability of .62. The items that tested the ability to explain reasoning in concluding had an alpha test of .71
Self-efficacy for Learning Form (SELF).
The SELF scale (Zimmerman & Kitsantis, 2005) is a 19-item survey designed to test student self-efficacy for learning. The items ask students to determine their ability to complete self-regulated learning strategies on a percentage scale divided into increments of ten percent. It is designed to have students self-report on a variety of situations that require academic self-regulatory efficacy such as reading, note taking, test taking, writing, and studying. High scores on this scale represent a high ability to be self-regulatory in academic strategies. This scale has a reliability coefficient of .97 and was highly correlated to teacher reports on students.
Qualitative
Measures
Test of Electricity-Magnetism Knowledge (TEMK)
The science content taught during the intervention includes magnetism, static electricity, current electricity, and electromagnetism. The TEMK (Peters, unpublished) assesses each students’ attainment content goals at an eighth grade level: (a) behavior of static electrical charges, (b) behavior of electrical current, (c) behavior and internal mechanisms of magnets, and (d) behavior of electromagnetic interactions. The questions on this test are open-ended and assess each of the content goals using visual, logical and analytical forms of communication. Each test will be analyzed for strengths and weaknesses in particular content areas, themes in the way students answer questions, and themes in the way students design scientific products such as data tables or observations. The content test will be administered before and after the intervention.
The
Views of the Nature of Science- Form B (VNOS –B).
The VNOS-B (Lederman,
Abd-El-Khalick, Bell & Schwartz, 2002) assesses student
understanding of
science as a way of knowing and consists of seven open-ended questions
corresponding to the seven identified aspects of the nature of science:
(a)
scientific knowledge is durable, yet tentative, b) empirical evidence
is used
to support ideas in science, c) social and historical factors play a
role in
the construction of scientific knowledge, d) laws and theories play a
central
role in developing scientific knowledge, yet they have different
functions, e)
accurate record keeping, peer review and replication of experiments
help to
validate scientific ideas, f) science is a creative endeavor, and g)
science
and technology are not the same, but they impact each other
(McComas, 2004).
Lederman, Abd-El-Khalick,
Student products from
inquiry units.
The control group and the experimental groups will have identical inquiry units that contain identical science content and science process skills to master. Within the inquiry unit, students will answer science content questions and use science process skills to make conclusions about the phenomena. The student products will be analyzed using the same protocol as the content test: strengths and weaknesses in particular content areas, themes in the way students answer questions, and themes in the way students design scientific products such as data tables or observations.
Teacher memos.
Memos are a versatile tool used to in many ways such as helping researchers reflect on events that are occurring during the research study or documenting confusing events for later analysis (Maxwell, 2005). Complications could arise during the interpretation phase of data analysis due to the dual role of teacher and researcher. Memos could help to reduce the confusion in interpretation because they will discuss implicit events during the research study. Memos will be written throughout the research study and then coded for emergent themes.
Think Aloud Protocol.
After the interventions six students will be randomly chosen from the control group and six students will be randomly chosen from the experimental group and videotaped separately while they perform an investigation from the intervention. Students will be asked to think aloud during the videotape in order to elicit their thinking processes during a scientific investigation. Since eighth grade students have little experience in expressing their “inner voices”, an established protocol to encourage three levels of verbal reports will be used, verbalization of covert encodings, explication of thought content, and explanations of thought processes (Ericsson & Simon, 1993). Students will be instructed to talk aloud about what they are thinking, and not to explain the answer to the problem. Students will be prompted at key points throughout the think aloud to continue their explanation of what they are thinking. The frequency of each level of verbal report will be reported as well as the themes that emerge from each level.
Focus Group Interviews.
After the intervention, six members will be randomly chosen from the experimental group and six members will be chosen from the control group to participate in a focus group. A focus group was chosen as a method of data collection rather than individual interviews because eighth grade students tend to minimize interactions with adults. A focus group will elicit more rich verbal data from the students because they will interact with each other and expand each others’ ideas. The questions are semi-structured because they focus the conversation without giving up the freedom that may be needed to explore phenomena that emerges. Sample questions are (a) What was the topic of your last science class? (b) How did you think like a scientist in that lesson? (c) How did you act like a scientist in that lesson? (d) How do you think science class is different from English, history or math class? (e) How can you think about your thinking? (f) What does it mean to you to think like a scientist? (g) Are there other ways of thinking? (h) Do scientists behave differently than other people? Focus group conversations will be audio-taped and transcribed using the software, Transana.
Intervention
The intervention, 4-Phase Embedded Metacognitive Prompts based on the Nature of Science (4-Phase EMPNOS), consists of seven modules that cover the content of electricity and magnetism at an eighth grade level. Each module is based on inquiry methods (NRC, 1996) and asks students to make observations and inferences about phenomena. Module one investigates behaviors of permanent, ceramic magnets. Module two investigates phenomena involved with static electricity. Module three investigates models that explain current electricity. Module four investigates series and parallel circuits. Module five investigates electric and magnetic interactions. Module six investigates the historical context of the discovery of motors. Module seven investigates the social implications of motors, generators and transformers. Each of the experimental modules includes nature of science metacognitive prompts for each of the seven aspects of the nature of science: a) scientific knowledge is durable, yet tentative, b) empirical evidence is used to support ideas in science, c) social and historical factors play a role in the construction of scientific knowledge, d) laws and theories play a central role in developing scientific knowledge, yet they have different functions, e) accurate record keeping, peer review and replication of experiments help to validate scientific ideas, f) science is a creative endeavor, and g) science and technology are not the same, but they impact each other (McComas, 2004). Metacognition is developed throughout the units based on Zimmerman’s (2000) model of the 4-phases of self regulation: observation, emulation, self-control and self-regulation. For example Module 1 for the experimental group would be based on magnetism content and have four sections of embedded developmental phases of metacognitive prompts throughout the unit.
Procedures
Students from all four classes will be given the MOLES-S, the MONOS, the SELF, the VNOS-B, and the content test before the intervention begins. Classes will then be chosen randomly to be in the control group or the experimental group. The inquiry unit on electricity and magnetism without the embedded metacognitive prompts will be given to the control group and the inquiry unit on electricity with the embedded metacognitive prompts (4-phase EMPNOS) will be given to the experimental group. Students will proceed with the four modules of the intervention (the inquiry unit) with the guidance of the teacher/researcher. Student groups of four will use the intervention packets to investigate the learning goals in electricity and magnetism. The teacher/researcher will act as a facilitator in their learning process and will write reflective researcher memos throughout the intervention. After the second module is complete, all students will take the SELF survey. After all four modules are complete, students will take the MOLES-S, the MONOS, the SELF survey, the VNOS-B, and the content test. When students are finished the modules, all of their work products will be collected. Six students from the control group and six students from the experimental group will be randomly selected to participate in a think aloud by performing one investigation from the intervention while being coached to think aloud. The control group will perform the think aloud separately from the experimental group. Six students from the control group and six students from the experimental group will be randomly selected to participate in a focus group which is designed to elicit their shared experiences in the two different types of inquiry units.
Design
This quasi-experimental study is designed to show differences in content knowledge, knowledge of the nature of science, metacognition and self-regulatory efficacy between the control and experimental group. The MOLES-S, the MONOS, the SELF survey, the VNOS-B and the content test will be given as a pre- and post-test so that variances between the control and experimental can be analyzed. The SELF survey will also be given at the midpoint of the intervention to determine the pattern of the level of self-regulatory efficacy the students experience before, during and after the intervention. Researcher memos that were written throughout the intervention and student work products will be used to back up any inferences made with the pre- and post-test analysis. Focus group results, think aloud results, researcher memos, and student work will be used to determine the processes students used to achieve the measured outcomes.
Proposed
Data Analysis
Quantitative data will be gathered using the MOLES-S, the MONOS, and the SELF survey. The MOLES-S and the MONOS are Likert-scales and the SELF is a percentage scale. Combined, the scales will measure the constructs of metacognition, knowledge of the nature of science, and self-regulatory efficacy. These data will be first analyzed using MANOVA and then compared to the results of a MANCOVA analysis so that any covariates can be eliminated from the analysis. The VNOS-B will be analyzed using a rubric that determines the frequency of knowledge of the nature of science as well as the comprehensiveness of the knowledge of the nature of science. The content test will be analyzed for student comprehensiveness of the content goals as well as their knowledge of the nature of science. The focus group results will be analyzed for common experiences within the groups using a phenomenological stance and the processes that emerge from the common experiences will be reported. The think aloud results will be analyzed for the frequency of each of the three levels of verbal report discussed in the instrument section of this paper as well as for the processes that students use to achieve metacognition related to the nature of science. The researcher memos and student work products will be analyzed for common themes and for processes that students use to achieve metacognition related to the nature of science. All data sources will be catalogued in a matrix so that all data can be triangulated.
Expected
Results
Since the 4-phase EMPNOS intervention utilizes a developmental self-regulatory strategy, I suspect that students’ self-regulatory efficacy will rise throughout the intervention compared to the control group. I also suspect that the student knowledge of the nature of science will increase for the experimental group due to student interaction with explicit questioning and checklists based on the nature of science. Based on prior research, often knowledge of the nature of science does not correlate with content knowledge, so I do not expect a rise in content knowledge. I am especially interested in the comparison of the experimental and control outcomes of the think aloud protocols because I think this protocol will be very helpful in explaining students’ cognition during the process of the scientific investigation.
To date, I have drafted a problem statement (Chapter 1), a literature review (Chapter 2) and a methodology section (Chapter 3) for my dissertation proposal. I have also run a pilot study with approximately 90 eighth grade students. Much of the research process went according to the original plan. The students who completed the required forms completed the pretests in two days: MOLES, MONOS, VNOS, TEMK, and the self-efficacy measure. Four modules on electricity and magnetism were taught throughout three weeks. Two classes, the experimental group, received modules with metacognitive prompts and two classes, the control group, received modules without metacognitive prompts. After two of the four modules were complete the students took the self-efficacy measure. After all four of the modules were complete the students took the post-tests. Student volunteers were chosen randomly to participate in focus groups and think-aloud sessions one week after the post-tests. None of the students withdrew from the study and there were no complaints about the study. A portion of the data has been analyzed and a proposal was accepted for presentation at the National Association for Research in Science Teaching. I plan on adding two questions from the NAEP exam that are at an appropriate grade level and content area to my Test of Electricity and Magnetism Knowledge (TEMK) instrument so that I can compare my students to a national sample. I will be able to add to my data base when I run my dissertation experiment, resulting in about 200 participants over two years. After my dissertation I plan to expand my experiment to include both elementary and high school students in order to better understand in interactions of nature of science, self-regulation, metacognition and content knowledge.
During the summer of 2006, I set up the SPSS data base from my pilot dissertation study and I was able to analyze the metacognition results from my dissertation pilot study. I submitted a proposal to the NARST 2007 Conference and was accepted as a full paper.
During spring 2007, I took an independent study class with Dr. Kitsantas where I did a descriptive and inferential analysis of the data.
On a larger scale, I am interested in developing research into the academic interactions in the classroom and their role in helping or hindering students from pursuing an interest in science. I feel that students often get turned off by science because they do not identify with the teacher or style of teaching that often occurs in science. Often science is taught as a body of knowledge and when students acquire the body of knowledge, they are "in the club" and succeed. I feel that if science were taught in a way that allowed students to understand how the knowledge was generated, they would identify more with the subject area and be more successful. This research agends is so broad, it could result in many years of research work.
Second Portfolio Review
Using my prelimary literature review as a foundation, I have incorporated relevant research articles to the document as I read. I will continue to add to my literature review throughout the next year. Click here to read what I have so far.
Since I have taken Qualitative Research Methods and Quantitative Research Methods, as well as being involved in several research projects, I am feeling more informed about how to design an independent research project. I plan on conducting a field test for a possible dissertation study, "Using the Nature of Science as a Metacognitive Resource." This study will be an experimental set up with 2 classes as the control and 2 classes with the intervention. The intervention is detailed in the lesson plan below. I plan to pre- and post-test the students with all of the measures. In this study I hope to answer the following questions: a) Will metacognitive prompts aid in student understanding of science content? b) Do students become more aware of the nature of science through metacognitive prompting? c) Will metacognition increase with the use of prompts based on the nature of science?
- Timeline for Field Test
- Assessment Matrix for Field Test
- Connecting Inquiry to Nature of Science to Metacognition
- Phase 1 - Observation
- Phase 2 - Emulation
- Phase 3 - Self-Control
- Phase 4 - Self-Regulation
- Lesson Plan without metacognitive prompts
- Lesson Plan with metacognitive prompts (4-phase EMPNOS)
- MOLES-S (Metacognitive Orientation)
- MONOS (Metacognition of the Nature of Science)
- SELF (Self-regulation and Self-Efficacy)
- VNOS-B (Views of the Nature of Science)
- Think Aloud Protocol for One Inquiry Module
- Focus Groups
- Student Work Products
First Portfolio Review
As I am taking Quantitative Research Methods this fall, I will try to attach my research project for this class to an idea that I have for my dissertation. Since I am teaching 8th grade science full-time, perhaps I will look at my students' beliefs about science as a discipline. I want to find out if they had experiences where science was treated as factual information only and no mention of the epistemology of science was discussed.
Click here for some of my ideas for the initial stages of my dissertation planning.
Home
