As I read Professor Gorrell’s  article about evaluating an educational program in five Louisiana parishes I wondered where it was leading.  The article  started out describing the program in which teachers were taught techniques to help them become better teachers.   It detailed the  goals of the Learning Connections Project (LCP) and the methodology used.  The LCP goal in this three  year study was to improve student performance by getting teachers to use techniques which improved the quality of teaching,  student-teacher interactions and the school and classroom climate.    From Dr. Gorrell’s lecture we found out that in the third year of the project, the students’ standardized test scores,  rather than the teachers’ performance were evaluated.  This  change of focus in the project caused  confusion in trying to figure what was being evaluated and why.

        During the three years of the project, teachers in the selected schools were  trained  and observed to see how well they implemented the techniques which were supposed to improve instruction and these results were measured against those teachers who were not implementing the goals of the LCP.  Teachers who implemented the techniques taught were called ‘high implementers’.  The  evaluation team described the success of the program by discussing the changes in the classroom of the high implementers. The low implementers, who were also known as the ‘non-high implementers’ in order not to cast a negative spin on the identifying terms, were not the focus of the study.

        The project had merit.  It dealt with the improvement of teaching in order to eventually improve student  scores on standardized assessments.  As I understood it, the  focus of the Learning Connection  Project was on improving teaching and dynamics in the classroom.  These student-centered learning strategies would carry over to the students learning more in class and therefore performing better on the standardized tests.   For me early in the paper confusing sentences started to crop up such as: “...only five of the original list of high implementers were still considered by the CDL staff to be high implementers at the end of the project” (p5.).  Also on the same page, the author notes that some of the teachers who were high implementers in the first year “can actually be considered to be low implementers”  by the third year. That started the red lights flashing.  If improvement in teaching was being measured, how could the study be accurate if the teachers whom they were following were no longer considered as high-implementers?  And, more importantly, WHY were they no longer implementing the good teaching techniques? I wondered how the research team was going to measure results.

        In the conclusion section, the differences between the two groups of teachers were not very impressive.  According to the evaluation team,  very little difference appeared between the high-implementers and the non-high implementers in grades which the students received.  In terms of whether the classes were more student centered,  another of the objectives of the Learning Connection, the results were not  considerably different between the high implementers and the non-high implementers.  The only difference between the two groups appears to be that the high implementers used more activities which engaged the students’ high order thinking skills (p25).

        The article left me wondering why such research was done in the first place because it seems to have proven that the program has not had much of an effect on the teaching in the target schools.

        As I approached the end of the article my reaction continued to be: what is the point?  What is being evaluated?  What can we learn from this study?  As Dr. Evelyn Jacob would say, “Where is the ‘so what’ factor?”  As I finished the article, I realized that what we can learn is that it is difficult to evaluate a study which does not have a clear focus.  From Dr. Gorrell’s presentation, we found out that the study changed focus from evaluating improvement in teaching skills to improvement in student test scores.  Since test scores were not part of the original focus of the study, the evaluation team had to switch gears and find a way to make the scores be the evaluating factor.

        This change in focus prevented a very promising study, about changing teacher attitudes to improve classroom instruction, from becoming a valuable research tool.  It is important in an evaluation study for the goals to be stated and agreed upon by all the parties involved before the research begins.  In that way,  from the beginning, the focus of the research and therefore the data will match what is being measured.

        From the beginning I questioned why the focus of the study was not on teaching low achieving teachers how to become high implementers and on the strategies used to encourage positive change in teacher attitude.  With that focus, the increase of high implementers could be compared from the first year to the third year of the study.  This would have given the funding organization quantitative, measurable results which would have been more relevant than the student scores.  If the emphasis of the LCP had remained on improving student achievement through improving motivation and engagement in the classroom, improving the school climate, improving teacher strategies, and fostering positive student-teacher interaction, then test scores would eventually improve and the research would have provided a valuable learning tool.  Without the correlation between what is being measured and why, the report loses focus and causes confusion not only with those on the evaluation team but also for the readers of the report.

Reference

Gorrell, J. (2000) Learning Connections:  Evaluation Report for Year Three.  Presented to the   Baptist Community Ministries