10/16/01
Critique #1
Learning Connections
Presented to the Baptist Community Ministries.
Evaluation Report for Year Three
Gorrell, J. (2000)
As I read Professor
Gorrell’s article about evaluating an educational program in five
Louisiana parishes I wondered where it was leading. The article
started out describing the program in which teachers were taught techniques
to help them become better teachers. It detailed the
goals of the Learning Connections Project (LCP) and the methodology used.
The LCP goal in this three year study was to improve student performance
by getting teachers to use techniques which improved the quality of teaching,
student-teacher interactions and the school and classroom climate.
From Dr. Gorrell’s lecture we found out that in the third year of the project,
the students’ standardized test scores, rather than the teachers’
performance were evaluated. This change of focus in the project
caused confusion in trying to figure what was being evaluated and
why.
During the three years of
the project, teachers in the selected schools were trained
and observed to see how well they implemented the techniques which were
supposed to improve instruction and these results were measured against
those teachers who were not implementing the goals of the LCP. Teachers
who implemented the techniques taught were called ‘high implementers’.
The evaluation team described the success of the program by discussing
the changes in the classroom of the high implementers. The low implementers,
who were also known as the ‘non-high implementers’ in order not to cast
a negative spin on the identifying terms, were not the focus of the study.
The project had merit.
It dealt with the improvement of teaching in order to eventually improve
student scores on standardized assessments. As I understood
it, the focus of the Learning Connection Project was on improving
teaching and dynamics in the classroom. These student-centered learning
strategies would carry over to the students learning more in class and
therefore performing better on the standardized tests. For
me early in the paper confusing sentences started to crop up such as: “...only
five of the original list of high implementers were still considered by
the CDL staff to be high implementers at the end of the project” (p5.).
Also on the same page, the author notes that some of the teachers who were
high implementers in the first year “can actually be considered to be low
implementers” by the third year. That started the red lights flashing.
If improvement in teaching was being measured, how could the study be accurate
if the teachers whom they were following were no longer considered as high-implementers?
And, more importantly, WHY were they no longer implementing the good teaching
techniques? I wondered how the research team was going to measure results.
In the conclusion section,
the differences between the two groups of teachers were not very impressive.
According to the evaluation team, very little difference appeared
between the high-implementers and the non-high implementers in grades which
the students received. In terms of whether the classes were more
student centered, another of the objectives of the Learning Connection,
the results were not considerably different between the high implementers
and the non-high implementers. The only difference between the two
groups appears to be that the high implementers used more activities which
engaged the students’ high order thinking skills (p25).
The article left me wondering
why such research was done in the first place because it seems to have
proven that the program has not had much of an effect on the teaching in
the target schools.
As I approached the end of
the article my reaction continued to be: what is the point? What
is being evaluated? What can we learn from this study? As Dr.
Evelyn Jacob would say, “Where is the ‘so what’ factor?” As I finished
the article, I realized that what we can learn is that it is difficult
to evaluate a study which does not have a clear focus. From Dr. Gorrell’s
presentation, we found out that the study changed focus from evaluating
improvement in teaching skills to improvement in student test scores.
Since test scores were not part of the original focus of the study, the
evaluation team had to switch gears and find a way to make the scores be
the evaluating factor.
This change in focus prevented
a very promising study, about changing teacher attitudes to improve classroom
instruction, from becoming a valuable research tool. It is important
in an evaluation study for the goals to be stated and agreed upon by all
the parties involved before the research begins. In that way,
from the beginning, the focus of the research and therefore the data will
match what is being measured.
From the beginning I questioned
why the focus of the study was not on teaching low achieving teachers how
to become high implementers and on the strategies used to encourage positive
change in teacher attitude. With that focus, the increase of high
implementers could be compared from the first year to the third year of
the study. This would have given the funding organization quantitative,
measurable results which would have been more relevant than the student
scores. If the emphasis of the LCP had remained on improving student
achievement through improving motivation and engagement in the classroom,
improving the school climate, improving teacher strategies, and fostering
positive student-teacher interaction, then test scores would eventually
improve and the research would have provided a valuable learning tool.
Without the correlation between what is being measured and why, the report
loses focus and causes confusion not only with those on the evaluation
team but also for the readers of the report.
Reference
Gorrell, J. (2000) Learning Connections: Evaluation Report for
Year Three. Presented to the Baptist Community Ministries
Mark comments that I should look indent paragraphs in my next
critique. I corrected that in this version of the paper.
He said that the questions I brought up in the paper were excellent. |