| EDRS 811 Quantitative Methods in Educational Research: Final Paper |
The Effects of
Gender, Age, Ethnicity, and Tutoring on the PALS Performance of Virginia
Kindergarten Students
Abstract
The
effects of early literacy skills on later school success have been a topic
widely discussed in education literature. Research has shown that phonological
awareness skills in kindergarten are highly related to a child’s later school
success. However, as Lee and Burkham
(2002) noted, not all students enter the kindergarten setting with the same
early literacy experiences. Individuals
who have studied the achievement gap between white and minority students
contend that difference in early reading skills may be related in part to the
continuation of the gap. The No Child
Left Behind Act (NCLB) was developed with the idea of providing all students
with the same access to educational skills.
Reading First, a portion of the Act, focuses on strengthening the
pre-reading skills of young children.
In response to NCLB and Reading First, states have begun to develop
measures that will provide data regarding the early literacy skills of students
in their public schools so that appropriate early reading interventions can be
utilized.
In
1995, the Virginia Department of Education designed the Early Intervention
Reading Initiative (EIRI) to tackle the problem of reading failure in young
children. The state funded programs to
develop screening measures to assess the pre-reading skills of children
entering kindergarten, and it was through this funding that the Phonological
Awareness Literacy Screening:
Kindergarten (PALS-K) was started in 2003 by Invernizzi, Meier, Swank,
and Juel. The PALS-K program was
designed to serve as a screener for the three main goals of EIRI: “to identify
children requiring early literacy intervention, to guide teachers’ development
of assessment-driven, classroom-based early literacy enhancement strategies,
and to document the effectiveness of early literacy instruction for children
identified as needing intervention (Invernizzi, Cook, & Geller, 2003).
The
PALS-K examines the following skills:
rhyme awareness, beginning sound awareness, alphabet knowledge, letter
sounds, spelling, concept of word, and an optional task to look at word
recognition skills. These topics are
closely aligned with the skills targeted by the Reading First initiative. Screening measures are presented
individually to students in both the fall and spring of the school year and a
summed score is calculated for their responses. If a child’s summed score does not meet the benchmark for the
test, they are identified as a student who needs additional instruction in
pre-reading skills. The additional
instruction may come in the form of in-school tutoring programs that are funded
by the state. Each school has the
ability to determine the type of additional instruction used, however the most
common form of remediation consists of 30 minutes of additional practice each
day targeting the specific skills that were weak on the PALS-K. The tutoring is conducted by trained
personnel such as retired teachers, parent volunteers, or substitute teachers.
Researchers are
just beginning to analyze the effectiveness of PALS-K in predicting reading
skills in classrooms around the state.
Primary questions that have arisen from the screening process include
differences between scores of children from non-minority/minority background,
gender differences in testing, and the effectiveness of tutoring on spring
PALS-K performance. This study will
attempt to provide data related to these questions based on information
collected from a sample of 60 kindergarten students in a suburban area of
Virginia. Specifically, the following
research questions will be addressed:
RQ1- Is there a
change in fall to spring PALS-K test scores for students regardless of age,
gender, ethnicity, or participation in tutoring?
RQ2-Are there
gender differences on the gain scores from fall and spring testing and do they
depend on age?
RQ3- Are spring
PALS-K scores predicted by performance on the fall testing or participation in
tutoring and what is the unique contribution of each factor to spring
performance?
RQ4-Are there
gender differences on spring PALS-K scores when the fall test differences are
controlled?
RQ5-Is there an
association between gender and tutoring on spring PALS-K performance?
Information gleaned from the data
analysis of these questions will be added to a state-wide database established
by the author that will examine both longitudinal data tracking the students in
later grade levels as well as yearly kindergarten data. The information will be collected for a
period of three years which coincides with the original funding cycle.
Method
Participants
Sixty
kindergarten students (29 boys and 30 girls) aged 5 (n=30) and 6 (n=30)
were chosen to participate in the study.
The students were primarily from Caucasian, African-American, and
Hispanic ethnic backgrounds; however students from other ethnicities were
accepted to participate in the study and were coded under an “other”
category. The students were chosen
randomly from a suburban elementary school in Virginia that was randomly
selected by the researchers. Permission
to participate in the study was obtained by the student’s parents, the
classroom teacher, and the students.
The students participating in the
study were administered the PALS-K instrument which examines knowledge of
several fundamental literacy principles.
The screening instrument was broken down into seven sections. Rhyming awareness consisted of 10 items
asking students to identify two pictures that had rhyming words. Beginning sound awareness had 10 items that
required students to find a picture that began with the same sound as a target
picture. The alphabet knowledge test
asked children to name the letters of the alphabet, while the letter sound task
required the students to name 23 letter sounds and 3 diagraphs. The spelling test looked at children’s
ability to spell five consonant-vowel-consonant words and concept of words
asked students to identify words in and out of text. The optional word recognition task was given to students who
shows some reading ability, however completion of this task was for information
only and did not affect the benchmark scores of participants.
Design and Procedures
Kindergarten participants were
administered the PALS-K by their classroom teacher, who had been trained with
their grade level at their school building in PALS administration using the
PALS administration guidelines.
Students were tested during the assessment windows described by the
program, meaning the assessment was required to be given during the month of
October for the fall test and the month of May for the spring test. Students were screened individually during
sessions that lasted between 5 and 15 minutes depending on the reading skill
level of the student. Benchmark scores
were tabulated and entered into the PALS-K on-line database.
Statistical Methods
In
order to examine the research questions posed at the beginning of the study, a
number of statistical methods were employed.
Paired t-tests were used to analyze the amount of
change between fall and spring PALS-K test scores regardless of gender, age,
minority, and participation in tutoring.
The differences between age and gender and their interaction on gain
scores were determined using 2-way ANOVA.
Multiple regression techniques were used to examine the prediction of
spring PALS-K scores based on fall scores and tutoring participation. Differences between genders on spring test
scores, controlling for fall test differences, were analyzed using ANCOVA. Finally, the association between gender and
tutoring on spring performance was examined using the Chi-Square technique.
Results
Research Question 1
The results from the paired t-test are presented in Table 1. On average, there is a statistically significant difference between student scores on the fall PALS (M=26.35, SE = .348) and the spring PALS (M=77.68, SE=.667) regardless of age, gender, ethnicity, or if the student attended tutoring. Therefore, the null hypothesis that the fall and spring test scores are the same is rejected at α=.05 level of significance. Results of the paired t-test show that at the 95% confidence interval for the mean difference, spring PALS scores are higher than fall PALS scores by at least 50.57 points but not more than 52.09 points.
Research Question 2
The effects of the interaction between age and gender on gain score are graphically represented in Figure 1. The results from the 2-way ANOVA show there was no statistically significant main effect for gender, F(1,56) =0.442, p=.509, p or for age, F(1,56)=0.021, p=.885, on student’s gain scores. Additionally, there was no significant interaction between age and gender on gain scores F(1,56) = 0.027, p=.870. Table 2 provides the 2-way ANOVA results.
Research Question 3
Table 3 shows the results of the multiple
regression examining the use of fall scores and tutoring as predictors of
spring performance on PALS-K. When
adjusted for one outlier, the R2 value demonstrates that .889% of the variance in spring PALS-K scores
is accounted for by fall scores and tutoring.
Fall scores and tutoring provide a statistically significant explanation
of the spring PALS-K scores F(2,56)=223.15, p=.000. Individually, fall
scores on the PALS-K have a statistically significant unique contribution to
predicting spring scores B=1.728, p=.000. Tutoring does not have a statistically
significant unique contribution to spring score prediction B=.168,
p=.705. The regression equation for this calculation is as follows:
SPALS=.168tutoring +1.728FPALS – 32.256. The regression slope for fall scores,
1.728, shows that any time fall scores increase by 1 unit, spring scores
increase by 1.728 when all other variables are held constant.
Table 4 presents information from the ANCOVA analysis. There is no statistically significant difference between genders on spring PALS-K performance when controlling for the differences in fall test scores F(1,57)=0.000, p=.982.
The association between gender and tutoring on spring test scores was unable to be calculated in SPSS due to some of the cells having less than the expected count of 14.02, therefore no results from the chi-square analysis are reported at this time.
Discussion
The results of this study provide some important preliminary data regarding the information gathered from the fall and spring PALS-K testing. Pairwise t-tests showed a significant gain in reading skills between the fall and spring testing administrations regardless of age, gender, ethnicity, and tutoring. While no effects were found in additional analyses for age, ethnicity, and gender, fall test scores and tutoring were determined to be significant predictors of spring test scores, with fall test scores having a statistically significant unique contribution.
These results directly relate to the differences in skill levels of students starting kindergarten that were described by Lee and Burkham in 2002. Students who had stronger skills for the fall PALS-K testing had higher scores on the spring PALS-K testing. Since fall testing is conducted in October with only brief time for classroom instruction, much of what is assessed using this measure are skills that were present in the student before he or she entered the kindergarten class. Therefore, the data from this study supports the idea that educational experiences prior to school-age directly impact performance in school not only at the beginning of the year, but also at the end of the school year. Achievement gap research has argued for the importance of quality universal preschool programs to expose all children to basic pre-reading skills. Based on this study, it would seem that Virginia would be making a sound educational decision to consider funding preschool programs for all children.
It should be noted however, that this study, although random, was conducted on only a sample of students from a suburban Virginia elementary school. Because of the limitations in numbers of participants, it is highly suggested that further research be examined before fiscal decisions are made by the district school board regarding the implementation of universal preschool. Further studies could repeat this research in school systems in Virginia with more diversity in ethnicity and age to continue to look for effects of preschool in raising the reading scores of children in grades K-5. In addition, the important question of the benefit of tutoring on spring scores was only briefly addressed in this study. Later studies could use both a qualitative and quantitative approach to gathering data regarding the both implementation of PALS tutoring in school buildings and the generalizability of information based on intervention effect. Repetition of the study with a larger, more diverse population could lead to more insights into the need for more focused reading interventions for young children. Having more structured interventions could greatly improve the students reading skills and lead to more in-depth discussions of the limitations of families on their child’s reading performance.
Invernizzi,M., Cook, A., &
Geller, K. (2003) PALS-PreK: Phonological Awareness
Literacy Screening for Preschool (Tech.Rep. for Virginia Department of
Education). University of Virginia, Curry School of Education.
differences in achievement as children begin school. Washington, DC:
Economic Policy Institute.
PALS Pre-K Instruments. (2007). Retrieved April 30, 2007, from
http://pals.virginia.edu/PALS-Instruments/PALS-K.asp.
Publication Manual of the American Psychological Association (5th ed.) (2001).
Washington, DC: American Psychological Association.
Paired Samples
Statistics
________________________________________________________________________
Mean N Std. Std. Error
Deviation Mean
________________________________________________________________________
Pair Fall PALS 26.35 60 2.692 .348
Spring PALS 77.68 60 5.167 .667
________________________________________________________________________
Analysis of Variance
for Gain Scores
________________________________________________________________________
Variable df F pη2 p
________________________________________________________________________
Between subjects
Age 1 .021 .000 .885
Gender 1 .442 .008 .509
Age x Gender 1 .027 .000 .870
________________________________________________________________________
Table 3Variables Predicting Spring PALS-K Scores (n=56)
________________________________________________________________________
Variable B SE B β
________________________________________________________________________
(Constant)
32.256 2.182
Tutoring .168 .441 .071
FPALS 1.728 .082 .943*
________________________________________________________________________
*p<0.05
Table 4
Scores Controlling for
Fall PALS-K Scores
________________________________________________________________________
Variable df F pη2 p
________________________________________________________________________
Between subjects
Fall PALS 1 253.425 .816 .000
Gender 1 .000 .000 .982
________________________________________________________________________