How can I get it right? (Wolcott)
How might I be wrong? (Maxwell)
To me, the two questions above
boil down the essence of validity. At every step of the research
process, I have to be aware of validity issues so that when I reach the
conclusions I can feel confident that I have gotten it right. I will try
to acknowledge and deal with validity threats from the beginning of the
literature review through to the writing of the project.
I think that my concept of
validity is a combination of Wolcott’s and Maxwell’s. To me they go together
nicely. One aims to get it right and the other aims not to get it wrong.
Wolcott talks about being involved in the setting and not separating his
personal and professional view. That seems a bit extreme to me, especially
after knowing the extent of his involvement. (But, I digress.) Wolcott
mentions recording first impressions and using them as a baseline. I like
that idea as a way to gauge how my impressions change as I gather more
knowledge from observation and reflection. So far, after four years
of observing the dual language program, my ideas have changed considerably
as I understand more about that specific program and about bilingual instruction
in general.
My topic has to do with dual
language programs and the two language/ethnic groups who select the dual
language program for their children. When starting my research to
investigate what has been written about DL programs, I feel that I should
cast a wide net. According to Jim Crawford, in 2003 most dual language
programs were found in schools with a high economic group that benefits
from the DL program and a low economic group that the program is designed
to serve. If I go by this information, I would limit my background
information to only that formula. So instead, I think that I should investigate
dual language programs not found in those demographics. For example, I
would look into the Oyster Elementary program and Arlington dual language
programs, as well as Texas, Arizona and California border programs.
A broader knowledge base might bring up issues that I hadn’t thought of
because they are not obvious in the school program I’m studying.
What validity threats am I most concerned about?
How can I be wrong and what can I do to acknowledge
it or guard against it?
When
|
Activity |
Threat |
Strategy to guard against it |
At the beginning |
Research for background information |
Narrow background information—just similar cases |
Wide literature review |
Designing the study |
Interview questions |
Bias in choice of questions |
One set of questions to form a basic core. Follow-up
questions same to everyone. |
During the study |
Informal conversations with parents/teachers |
'one of us' attitude''
Is she evaluating me? |
Record what was said and describe the situation/circumstances
rather than or as well as analyze content |
During the study |
Observations |
Bias in interpretation |
‘rich data’
try to explain rather than interpret
|
During the study |
Interview:
English
Spanish |
Bias
Accurate translation
'one of us' attitude |
Trial interviews for question bias
Feedback on accuracy
Feedback on
|
After gathering information |
Writing up the study |
Reflecting bias |
"word check" for value laden words or expressions
|
How can I consider validity
while designing the study? In planning the interviews, I have to
be careful not to ask questions in a biased way. I can have others check
the questions/field-try the questions/make sure the questions are open-ended
enough so that the participant can answer freely. I can design a
core of questions that dig at what I want to know and ask the same questions
to everyone. I should be aware of whether the questions are guiding the
data or the data guiding the questions.
Can I find negative evidence
or negative cases? I’m sure this will come up as I am interviewing
parents from two seemingly different groups. Instead of looking for
the differences from the beginning, I will try to ask the same questions
to everyone and consider all the families rather than discard those who
offer differing information. I will have to remain open to all cases
(families) and not disregard any ‘case’ until I have looked at all the
data. It would be very easy to find stereotypical families of the
two groups. I must consciously make the effort to include everyone
until my categories are set and I decide what to focus on. Then the other
cases or the ‘outliers’ will be mentioned as part of the data but not necessarily
as part of the two case studies—the two groups of language speakers.
We’ll see. At this point, I’m not sure what I will find.
If I write profiles or case studies
of the families, do the families agree with what I have written?
Here, definitely ‘member checks’ will help. Each section will be
read by a selected person or group of people who supplied the information,
to make sure that I have characterized the group properly. Since
it will be a composite of information from a group, I want to make sure
that what I wrote is a description of the group as a whole rather than
just of a set of verbally active parents.
Are the participants giving
me the information that they believe I want to hear? This could be a real
validity threat. With the Spanish-speaking parents, I would try to overcome
this threat by asking another person, the family liaison (a Mexican woman
who deals mostly with the Hispanic parents) to sit in with me during the
interviews. Having her with me will: 1) be a check on my Spanish
when asking the questions to make sure that I phrase them in a way that
the parents can understand and respond to; 2) also help me to verify the
translation to make sure that I understood the meaning implied in the answers
as well as the words.
When writing up the research,
I have to be clear about whether I am explaining, describing or interpreting.
I think that all of these enter at some time in looking at the research.
For example in interviews, what am I looking for? A description of what
parents want for their children. The danger I see here is that I have to
balance description with analysis. I think that my tendency is to
interpret what I hear through my own cultural filters. When dealing with
another ethnic group, I have to be very careful not to interpret when I
should describe.
When explaining the living
situations of the two groups of families, it would be easy to offer explanations
that boarder on interpretations. My researcher bias on where the Hispanic
families come from and what their life was like before coming to this country,
has to be checked by others to make sure that my bias doesn’t enter into
the description. Since I have lived in Central America and Mexico,
I understand the living situation that the families have come from.
As part of my job in Mexico, I had to interview families who wanted to
come to the US and therefore I know the economic situation they have left
behind.
My background helps me gain
entry into both groups. For one group, I speak the language and therefore
I can communicate with the Spanish-speakers. In many cases, I know their
children and have been their translator for the past 4 years for parent/teacher
conferences. For the English-speaking group, I am ‘one of them’ as a resident
of the neighborhood and have been active in the parent/teacher group composed
mostly of Caucasian mothers. I am known by both groups but am not
seen as part of either.
In general my main bias is
the topic itself. I started studying the DL program because I thought it
was poorly run. Then my ideas evolved to finding out more about it.
I did this by studying the teachers who taught in the program. I have also
studied the evolution of the program and how it started and how it has
evolved over the past 4 years. Now I am studying the families who
send their children to the program. Am I still circling the wagons
on a program I feel is poorly run? No, but I think that my original
attitude forms the baseline from where I started. Now, I think that this
study will provide valuable insight into the two groups of people represented
in this dual language program.
In the end, I am aiming
for a descriptive account of why two diverse groups of families support
the same program. My hope is that other schools in similar demographic
areas will look at the study and question whether my findings could inform
their decisions. I aim to get it right by being aware of and considering,
at every step of the way, how I could be wrong.
|