Help with RP2: Designing Experiments

This is the ITE 11 problem from class:
Design a Factorial Experiment with at least 3 levels of one variable (draw a Table)
Answer this question:
Which is more persuasive on a web site: plain text (black letters on a white background, no links) or text supplemented in some way?
1) Write out 2 Hypotheses (H1, H2):
· One Predicting Effects of IV 1, the other the Effects of IV 2
2) Declare the DV (You Choose - It is in Your H1, H2)
3) Label the 2 IVs and Label Their Levels
4) List 2 ("People") Variables you Should "Control for"

As we discussed in class, to make this concrete: imagine that you already have a plain text web site (black text, white background), with some text that makes a persuasive argument (e.g., that we should not cut off immigration into the USA despite terrorist threats).

How to think about this type of problem:

1. Ask yourself - what are the most important variables here? That is, what variables might influence the persuasiveness of your text? If you were really doing research in this area, you would know some of the factors that make an argument more persuasive (supporting with evidence, quoting experts in the area, etc.).
2. Assuming you don't know the literature on persuasion, start by considering: how could I make meaningful changes to a web site to enhance it? Answer: add a background color, graphics, colored text, change the font or font size, add links, etc.
3. Now you have a basis for comparison: you could create multiple web sites and expose different people to them, and measure how persuaded they are by your argument regarding immigration.
4. H1: Arguments presented with quotes from recognized experts are more persuasive than arguments presented without such quotes.
5. H2: Arguments presented with graphics are more persuasive than arguments presented without graphics.
6. Note that while color, graphics, links, and "bells and whistles" on a web site may not actually have a large effect on persuasion, this exercise is simply practice in designing experiments. You should choose meaningful variables, but you were allowed to use ANY relevant variables in your hypotheses.
7. Just be sure that the variables in your hypotheses match those in your experimental design. Each hypothesis proposes that the IV influences the DV in some way.
8. But for RP2, your answers will be judged (in part) on your ability to choose variables that really matter - variables that are likely to affect the outcome (including the variables for which one should control),
9. While you do not have to draw a chart for the RP2 assignment, consider what one would look like.

10. This is a 3x2 design
11. IV1 is Quotes (3 levels: no quote, quote from an unknown, quote from the President)
12. IV2 is Graphics (2 levels: no graphics, graphics)
13. DV = perusadability (level of persuasion after reading the argument, to be measured by a pretest-posttest)
14. Two variables that should be controlled: 1) initial opinion regarding immigration (a pretest will be used), and 2) level of knowledge of current immigration policy (the same pretest will ask about this)
15. Additional variables one might wish to control for include: 3) level of Internet experience (the pretest can ask about this too), 4) gender (while random assignment can be used, a researcher can go further: assign half men and half women to each condition; within genders, subjects would be randomly assigned to conditions - to take care of other variables you never assign subjects systematically, or let the subjects choose which condition (cell) they will go in.

Things to note about this experiment (and other like it):

1. The researcher must create 6 different web sites - one for each of the 6 conditions (1-1, 1-2, 1-3, 2-1, 2-2, & 2-3)
2. Each subject goes in one and only one cell - that is, each subject will read only one web site
3. The procedure would be: subject shows up at the lab, logged in as a man or a woman, and is assigned randomly to a condition (treatment level)
4. The subject then fills out the pretest instrument (questions), is asked to read the web site (the persuasive argument), and then is asked again about his/her position on immigration.
5. Each subject is given a difference score. Then the data is analyzed to see if there are differences among the groups (in persuadability) across either IV. That is, did quotes make a difference in persuadability? Did graphics make a difference in persuadability?
6. Note that there are other limitations to a study such as this (qualifications the researcher should mention in the write-up). For example, even if an effect is found for one or both IVs, the persuasion effect may be short lived. Much like students often forget material they knew at the time of a test, any persuasion effect may be temporary. Or perhaps there's a sleeper effect - subjects aren't persuaded immediately, but the scores might be different if measured again a week later (after they've had time to think about the issue some more). These are the kind of issues I want you to address under #7 on RP2.

Good luck!