Chapter 7
Experimental Research

I. Introduction

A. The attribution of causation pervades everyday, common-sense explanations of behavior.

B. Many researchers are interested in finding the cause(s) of events and behavior.

1. Just as criminal detective looks for the culprits who committed a crime, researchers interested in attributing causation look for the variable(s) that is responsible for an outcome.

a. Recall that the variable thought to produce the outcome is called the independent variable; and the outcome variable is called the dependent variable.

2. By conducting experiments, researchers assess causal effects of independent variables on dependent variables.

a. These systematic investigations are performed under tightly controlled conditions where people are exposed to only the independent variable and the effects of that exposure on the dependent variable are observed.

II. Establishing Causation

A. It should be pointed out, right at the beginning, that many scholars question the application of principles of causality to explain human behavior, including communication behavior.

1. One distinction that is helpful to the preceding debate is between universal laws and statistical laws.

a. Universal laws: Advance explanations that some event or outcome is always preceded by a particular event or “trigger.”

b. Statistical laws: Suggest that some event or outcome will be followed a some of the time (along a continuum of high-to-low probability) by a particular event or “trigger.”

2. Based in part on Mill’s reasoning, there are at least three requirements necessary for attributing a causal relationship between an independent and a dependent variable.

a. The independent variable must precede the dependent variable.

i. Mistakenly believing something that is chronologically impossible is called an anachronism.

ii. In experimental research, making sure that the independent variable precedes the dependent variable is accomplished by first exposing research participants to the independent variable and then seeing how that exposure affects the dependent variable.

iii. A placebo group believes they are receiving the independent variable being studied, but they are not; placebo means “I shall please” and any change that occurs is then called a placebo effect.

b. The independent and dependent variables must covary, that is they must go together in a meaningful way.

i. There are many behaviors and events that go together statistically, but are not related in any meaningful manner. These are called spurious relationships or nonsense correlations.

c. The changes observed in the dependent variable must be the result of changes in the independent variable an not some other variable.

i. If some other variable causes the observed changes, there is an alternate causality argument (sometimes called alternate hypothesis).

II. Exercising Control in Experimental Research

A. To establish a causal relationship between an independent and dependent variable, experimental researchers must exercise a good deal of control.

1. Control means that a researcher attempts to exclude other possible causes of the effects that he or she is studying.

a. Control exists on a continuum, ranging from loosely controlled experiments to those that are very tightly controlled.

b. Three factors shape the amount of control.

B. Exposing Research Participants to the Independent Variable

1. The first factor, level of exposure, includes researchers who regulate or manipulate how participants are exposed to an independent variable (high control), and researchers who only observe without manipulation (low control).

2. In highly controlled experiments, when researchers control participants’ exposure to a variable that variable is called a manipulated (active or controlled) variable (see Figure 7.1).

a. The examples in Figure 7.1 help illustrate how experiments typically involve various conditions or groups that receive differential exposure to the independent variable.

b. The most basic procedure is to divide research participants into two conditions, one that receives a manipulation (called a treatment or experimental group) and one that does not (the control group).

c. A more common and complex way in which researchers manipulate an independent variable is by exposing research participants to different levels or types.

i. The term, level, is generally used to indicate a condition of an independent variable in an experiment, both for nominal and ordered variables.

ii. The term, comparison group, is used in the broad sense to imply any group against which another is compared, such as two treatment groups.

3. It is not always possible, or even desirable to manipulate an independent variable; some variables, called attribute variables, cannot be manipulated (e.g., people’s age or gender).

a. Other variables can be theoretically manipulated, but researchers would never do so; as in the case of a person living with AIDS and his or her social support network.

i. No Institutional Review Board (See Chapter 6) would ever approve the manipulation of the HIV virus or one’s social support network when it has already been decimated.

ii. Natural experiments allow investigators to observe the independent variable in a naturally occurring context.

iii. Whenever exposure to an independent variable is not manipulated by researchers themselves, the variable is called an observed variable.

C. To further move towards establishing causation, researchers engage in ruling out initial differences between the conditions.

1. One of the most important things researchers need to know is that people in the different conditions were equivalent at the start of the experiment.

2. If equivalence cannot be assured, any changes in the dependent variable might be the result of selection, in which an unrepresentative sample is used (one that lacks equivalent participants).

a. Experiments that rule out initial differences demonstrate high control, whereas those low in control cannot rule these differences out as an explanation for the observed effects.

b. Equivalence often can be achieved through random assignment of individuals to treatment, control, and/or placebo groups.

c. In quasi-equivalent conditions, researchers use pretests in an attempt to establish a higher degree of equivalence among the research participants.

d. In nonequivalent conditions, no procedures are used and researchers cannot be sure that an change in the dependent variable is the result of the independent variable manipulation.

3. In random assignment, each research participant has an equal chance of being assigned to any particular condition of an experiment.

a. This procedure attempts to rule out the effects of any initial differences among the prospective research participants.

b. Random assignment gives the best possible assurance that the conditions of an experiment start off equivalent because all the initial differences between people should be distributed evenly across the conditions.

c. Random assignment is not always the perfect solution to ruling out a selection bias; especially if mortality occurs at high levels among the treatment groups, or if only one point in time is being studied.

d. Random assignments is not the same as random sampling. The former term is concerned with how research participants are placed into the different conditions of an experiment.

e. Random assignment increases the internal validity of a study while random sampling increases the external validity of a study.

4. When researchers cannot use random assignment, pretests are often used.

a. A pretest measures research participants on relevant variables that need to be accounted for before exposing the treatment group(s) to the manipulation of the independent variable.

b. Pretests are used in two primary ways to rule out initial differences between conditions.

i. The first is when researchers suspect that some important variable(s) that is not the focus of the study might potentially influence people; pretests can then be used to exclude those individuals who might jeopardize the findings in that they would possess or demonstrate the undesired variable(s).

ii. The second use of pretests is to measure research participants on the dependent variable before the independent variable manipulation occurs.

iii. To assess the impact of a manipulated independent variable, difference scores (also called gain scores) are computed by subtracting the pretest from the posttest.

c. A posttest is a measurement of relevant variables that occurs after the manipulation of the independent variable.

d. Using pretests, see “a” and “b” above, is not as effective as random assignment for creating equivalent groups because there may be important differences between the conditions that can’t be accounted for by pretests.

e. In addition, pretests of a dependent variable may also crate a validity threat due to sensitization (See Chapter 5).

5. Controlling for the effects of extraneous influences: The preceding procedures don’t control for all the problems that potentially undermine the findings of an experiment.

a. With regard to the way experimental research is conducted, one potential threat is a lack of procedural reliability, which makes it unclear whether any observed difference between conditions is due to the independent variable or the different treatments.

b. Researchers must also be wary of threshold effect, where changes in a dependent variable may not occur until the independent variable reaches a certain level.

c. Experimenter effects occur when different experimenters consistently administer particular manipulations of the independent variable.

i. If one experimenter always administers treatment A and another always administers treatment B, it won’t be clear whether participants’ responses are due only to the differences between the treatments or perhaps the way the experimenters administered them (researcher personal attribute effect).

ii. To control for the researcher personal attribute effect, researchers commonly use a variety of assistants and double-blind procedures that ensure those who administer the different independent variable manipulations and those who receive it do not know (are “blind” to) which participants are getting what manipulation.

iii. Confederates, can also be used, and these are assistants who pretend to be research participants.

d. Researchers also attempt to control for such participant threats as the Hawthorne effect, mortality, and interparticipant bias.

i. Researchers control for the Hawthorne effect by not letting participants know they are being observed or diverting participants’ attention away from the purpose of the experiment.

ii. Blank experiments introduce an irrelevant treatment to keep participants from guessing the true purpose of the experiment or becoming automatic in their responses.

iii. Researchers also want to control for the John Henry effect, a type of Hawthorne effect that occurs when research participants in a control group take the experiment as a challenge and exert more effort than any otherwise would.

iv. Researchers also employ many participants to minimize the threat of mortality.

v. Researchers also stress during debriefings the importance of not discussing an experiment with future research participants.

vi. The bottom-line is that these and other threats to internal and external validity must be controlled for if researchers are to have confidence in the findings from their experiments.

e. There are a number of other variables that might potentially influence the dependent variable (besides the independent variable being studied).

i. An Intervening, intermediary, or mediating variable, intervenes between the independent and dependent variable to explain the relation between them or provides the causal link.

(a) The effects of an intervening variable can be diagramed as: Independent variable--->Intervening variable--->Dependent variable

ii. Confounding variables obscure the effects of another variable and there are two types.

(a) A suppressor variable conceals or reduces a relationship between an independent and dependent variable (see text diagram).

(b) A reinforcer variable increases a causal relationship between variables (see text diagram).

© Lurking variables are present, often in subtle ways, and can confound the interpretation of the independent to dependent variable relationship (see text diagram).

(d) Extraneous variables are an umbrella-like term and could include all of the above variables.

(e) Control or concomitant variables are variables that researchers try to control in order to protect the validity levels of an experiment.

iii. One way researchers control for such variables is to make them another independent variable in the experiment.

iv. Another procedure is to use a matched-pairs design (also called participant or subject matching) in which participants are mated in pairs on some important characteristic.

v. There are also statistical procedures that can be employed to parcel out the effects of extraneous variables (See Chapter 14), and a variable controlled for statistically is called a covariate.

vi. Step-by-step procedures used in an experiment are called the protocol.

III. Experimental Research Designs

A. Campbell and Stanley

(1963) identify three types of designs including full experiments, quasi-experiments, and preexperiments.

1. These designs differ according to whether an independent variable is manipulated or observed, whether random assignment and/or pretests are used to rule out the validity threat due to selection, and what form of equivalence between conditions is created.

a. These three types of experiments range from highly controlled (full experiments) to loosely controlled (preexperiments) (see Figure 7.2).

2. Full experiments demonstrate the highest level of control because the independent variable is manipulated by the researcher and random assignment occurs among two or more equivalent conditions; note that full experiments demand that there be two or more conditions.

3. Quasi-experiments either manipulate or observe the independent variable and may have one or more conditions. There is no random assignment and pretests are used to assess whether there are some important initial differences between the conditions.

4. Preexperiments demonstrate the least amount of control of the three experiments. preexperiments, like quasi-experiments, manipulate or observe the independent variable and may have one or more conditions. There is also no random assignment; however, unlike quasi-experiments there are also no pretests and equivalence is not assumed.

5. The three general experimental designs are not hard and fast categories as what one authority might call a full experiment, another would might label it as quasi-experimental.

B. Preexperimental Designs

1. The simplest preexperimental design is the one-group posttest-only design in which a single treatment group is exposed to the independent variable and then assessed on a posttest (see text diagram).

a. This is a very problematical design and isn’t really used in experimental research unless under unusual circumstances, such as the Challenger disaster.

2. The one-group only pretest-posttest design is similar to the previous design except that it adds a pretest (see text diagram).

a. Adding the pretest allows researchers to compute a difference score between the pretest (before) and posttest (after) scores (sometimes called before-after designs).

b. Problems with this design can include sensitization, history, maturation, and/or statistical regression.

c. Many of the problems with this and the one-group posttest-only design stem from there being only one condition in the experiment.

3. The posttest-only nonequivalent groups design (sometimes called the static group comparison design) nonrandomly assigns research participants to a treatment or a control group and then measures them on a posttest (see text diagram).

a. Static group comparisons pose problems in that we can’t be sure whether the treatment really did or did not have an effect, and even if it did, the extent of that effect.

b. Confidence is usually lacking in the validity of the supposed causal findings obtained from this design.

C. Quasi-experimental Designs

1. Quasi-experiments, like preexperiments, either manipulate or observe an independent variable, and have one or more conditions. However, unlike preexperiments, in both single-and multiple condition cases, pretests are used in a fundamentally different way.

a. Quasi-experiments are most often carried out in the field rather than in the laboratory.

b. Quasi-experiments are often used to try and establish at least partial cause-effect relationships by maximizing the real-world transferability of research findings.

2. The single-group interrupted time series design (sometimes called time series design) involves giving a series of pretests to a single group prior to an experimental treatment, followed by a series of posttests (see text diagram).

a. The multiple pretests in this design help establish an intragroup baseline comparison, a way of comparing the same group over time prior to the experimental manipulation.

b. At least three problems are posed by this design: sensitization, the sleeper effect, and the lack of a comparison group.

3. The pretest-posttest quasi-equivalent groups design nonrandomly assigns research participants to a treatment to control condition, measures them on a pretest, exposes one group but not the other to the treatment, and then measures both groups again on a posttest (see text diagram).

a. The potential problems with this design include selection, sensitization, history, maturation, and/or the various selection-interaction effects.

4. The interrupted time series quasi-equivalent groups design (sometimes called the multiple time series design) combines the previous two quasi-experimental designs by nonrandomly assigning participants to a treatment of control group and measuring them on a series of pretests and posttests (see text diagram).

a. This design does not solve all the validity threats previously discussed since selection sensitization remain a concern, and random assignment is not used to help promote equivalence.

D. Full Experimental Designs

1. Full experiments are the highest in terms of control because the independent variable is manipulated by researchers and there are two or more conditions to which research participants are randomly assigned.

2. A traditional full experiment is the pretest-posttest equivalent groups design (sometimes called the pretest-posttest control group design) that randomly assigns research participants to a treatment or a control group and administers a pretest and posttest (see text diagram).

a. This design provides a high degree of confidence that the findings are due to the treatment and not to initial differences between the conditions; however, sensitization can still be a threat to validity levels.

3. The posttest-only equivalent design (sometimes called the post-test only control group design) is the same as the previous design except, that a pretest is not used; participants are randomly assigned to a treatment or control group and given a posttest (see text diagram).

a. This is a very powerful design, but it is still possible that important initial differences could be missed if random assignment does not work.

4. The Solomon four-group design literally combines the posttest equivalent groups and the posttest-only equivalent designs (see text diagram).

a. Three primary benefits include: Showing whether random assignment worked; showing whether the pretest worked; and assuming sensitization did not occur revealing whether the pretest combined with the treatment to produce a unique interaction effect that id different form the experimental treatment alone.

b. This design also requires twice as many research participants as the others and tends not to be used too frequently in experimental research.

5. A factorial design occurs when there is more than one independent variable studied, and the multiple independent variables are called factors.

a. Depending on how they are conducted, factorial experimental designs can be full experiments, quasi-experiments, or preexperiments.

b. As the full experimental posttest-only equivalent groups factorial design example reveals (see text diagram), the effects of each independent variable are assessed and the experimenter can ascertain whether the combination of conditions is more effective than one condition alone.

c. In theory, researchers can investigate the main and interaction effects of as many independent variables as they deem important.

d. Interaction effects among three or more variables are called second-order or higher order interaction effects; an interaction effect between two variables when three or more variables are studied is called a first-order interaction effect.

i. Because of the large number of possible interaction effects involved, researchers typically choose only those independent variables they consider most crucial and can examined reasonably within a single study.

ii. When an interaction effect is found, it typically is considered to be more important than any main effect.

E. Factorial Design Statements and Diagrams

1. Studies with more than one independent variable are summarized with a design statement: A series of numbers, one number for each independent variable in the study, separated by a multiplication sign (x).

a. The simplest factorial design statement has two independent variables and two levels for each variable, so there are two numbers (because there are two variables); in this case, both numbers are 2s: a 2 x 2 design.

b. More complex factorial designs have more than two independent variables (sometimes called N-by-M designs as opposed to the preceding two-by-two design).

2. Both the designs just discussed are crossed factor designs, because they involve having every level of one factor appear with every level of the other factor.

a. This is the most common factorial design, but in some experiments, the levels of one factor only appear (are “nested”) within a single level of another factor; this is called a nested factor design.

i. When the number of levels for each factor aren’t equal, this is known as a mixed design.

3. To understand factorial designs and design statements more fully, it’s often helpful to depict them visually in a design diagram.

a. In a simple, two-factor design, a design diagram is a box with one independent variable represented on the abscissa, and the horizontal (or x) axis (a line used to construct the graph), and the other on the ordinate, the vertical (or y) axis (see Figure 7.3).

i. A design diagram shows researchers all the possible combinations of the independent variables; each possible combination is called a cell.

ii. The experiment must include research participants for each cell usually equal numbers (where there are an unequal number, it is called an unbalanced design); with a generally accepted rule being that at five participants are needed for each cell in a factorial design.

4. A between-group design (between-subjects design) are those in which one group of research participants receive one level of an independent variable (such as a treatment) and are compared to another group that receives another level (such as no treatment).

5. A within-group design (within-subject design) is one in which a single group of people is tested two or more times.

6. In a repeated-measures design, the same participants are given multiple treatments of the independent variable and measured after each exposure.

a. This design can pose a problem if a treatment order effect occurs in which the order of the treatments presented makes a difference; earlier treatments might sensitize participants to later treatments.

b. For this reason, researchers randomize the treatment order or counterbalance it by making sure that all possible orders are used (see Figure 7.9).

c. Researchers using the repeated-measures design must also be sure that there are no treatment carryover effects, that is, the effects of each treatment have passed before exposing participants to subsequent treatments.

F. Laboratory Versus Field Experiments

1. Laboratory experiments take place a setting created by researchers, whereas others are field experiments conducted in participants’ natural setting.

2. Full experiments, quasi-experiments, and preexperiments can all be conducted in the laboratory or field.

a. Experimental research typically is conducted in a laboratory.

b. A laboratory is actually any research environment that is set up by researchers, including ones created in participants’ natural setting.

c. Laboratories allow researchers to exercise high control, but often they can minimize external validity.

d. Sometimes field researchers can conduct full experiments.

e. At other times, field researchers cannot randomly assign research participants to conditions or manipulate the independent variable, so they conduct quasi-experiments.

IV. Conclusion

A. Designing a high-quality experiment is no easy task.

B. The beauty of the required choreography pirouettes on researchers exercising a high degree of control so that alternate causality arguments for the changes in the dependent variable are ruled out.

C. By bringing their passion to bear to discover the details—the causes and effect of communication behavior—to steal a phrase, communication research takes one small step for the research participants studied, one giant step in helping people.