Transcription
Hi class. So, so far, with the exception of case studies, we've talked as if using groups of subjects is the only way to do research. Now, although most psychological research does involve groups of subjects, this approach is not the only way to do research. And so today's lecture is going to deal with strategies for achieving control in experiments using single-subject designs.
So, as we did at the beginning of the class, it's helpful to have everything in a historical context. So, looking back to the early underpinnings of single-subject research design, it was common and has a pretty long tradition. In fact, scientists have used single participants in research for longer than they've used groups. Gustav Fechner, who some historians say is the founder of experimental psychology, worked extensively on individual participants himself and his brother-in-law beginning in 1860. Fechner invented the basics psychophysical methods that are still used to measure sensory thresholds and discovered principles of psychophysics that are still taken seriously today.
Twenty-five years later, inspired by Fechner's work, Hermann Ebbinghaus did his experimental work on memory. Following Fechner's example, Ebbinghaus used himself as his own participant. Now, Wilhelm Wundt is a name that should sound familiar because we discussed him at the beginning of the class, and he is credited with founding the first psychological laboratory in 1879. He also conducted experiments measuring various psychological and behavioral responses in particular individual participants, so he was doing single-subject research as well. One of his famous students, Edward Titchener, espoused the use of introspection, which is the careful reporting of one's own experience, and we talked a lot about that at the beginning of the course as well. Because this procedure required a great deal of training, much of his work was done using one or a few individuals. And finally, Pavlov did his pioneering work on conditioning with individual dogs as subjects. And the list of psychologists who relied on individual participants is long and includes most of those working before 1930, when modern statistical methods were developed.
So, these early researchers used single subjects in the time-honored scientific tradition. In any case, modern statistical methods did not exist. And the solutions these researchers, the solutions of these researchers to the problems of reliability and validity were extensive observations and frequent replication of results. A traditional assumption of researchers doing single-subject experiments has been that individual participants are essentially equivalent and that one should study additional participants only to make sure the original participant was not grossly abnormal.
Most modern statistical methods that have become an integral part of present-day research grew out of a very different tradition. A Belgian astronomer, Adolphe Quetelet, discovered that human traits followed the normal curve. So that's that standard bell curve that you're used to seeing in statistics, plug in your statistics class. From this, he concluded that nature strove to produce the average man, and the variability around the mean that is always found was considered to be a result of nature's failure to achieve the ideal average person in every case. The individual differences tradition of Sir Francis Galton and Karl Pearson grew out of this thinking. According to the individual differences tradition, variability between participants is inevitable. The task then becomes how to separate the effects of the experimental manipulation from this inherent variability.
Now, during the 1930s, statistician R.A. Fisher and mathematician working on problems of genetics invented many of the statistical methods that we use today, such as analysis of variance, that have really become standard in psychological research. And these techniques dominated psychological research to such an extent that the single-participant tradition almost disappeared for several decades. Nevertheless, certain notable psychologists continued to work in the single-subject tradition during this period, notably B.F. Skinner. Skinner disdained the use of statistics, claiming that he would rather study one animal for a thousand hours rather than study a thousand animals each for an hour each.
So, Skinner's philosophy of research is described in the classic book by Mary Sidman. And Sidman makes clear the difference in attitude between the single-participant approach and the group's approach to research. The single-participant tradition assumes that most variability in the participant's behavior is imposed by the situation, so the power of the situation, and therefore can be removed by careful attention to experimental control. The individual differences group research tradition assumes that much of the variability is actually intrinsic and should be statistically controlled for and analyzed.
Now, in this class, we really can't settle the debate between these two positions. But psychologists began using statistical methods to evaluate the results of experiments in which removing all sources of variability was not feasible. So, a set of data may look so regular that it's hard to believe that they could be the result of chance, especially when one has a large personal investment in getting a certain result. The use of statistics is one way to avoid being fooled into thinking that data are more reliable than they really are. Then again, using statistical methods does not guarantee that you'll draw the right conclusions about the data. That's why in our previous lectures, we talked about Type 1 and Type 2 errors. So, if you need a reminder for some of these statistical errors that we can account for or encounter, I would encourage you to look back to your various threats to validity lectures and some of the errors that we can encounter there.
In addition, we should probably also note that employing single-participant methods is not completely incompatible with statistical analysis. This is really as much as statistical methods are being developed to handle data from individual participants as well. So, although we acknowledge that the group comparison approach has a rightful place in psychology, as a psychological research community, we're going to talk today about several advantages of the single-participant approach. We should keep these advantages in mind whenever we're designing a research project because there are some situations, albeit them somewhat limited, where single-subject research would really be beneficial.
So, people or animals in a single-participant design act as their own controls. Though, so, reminder, control is really your comparison group. So, it's where you're looking to see whether an effect actually exists. And so this is similar to within-subjects designs in statistics. So, think back to your statistics lectures. Within-subjects designs are the longitudinal studies where you're seeing the same individuals again and again and measuring those same individuals in the study, rather than having two disparate groups that are being studied at different time points. So, the benefit of a single-participant design where an individual is acting as their own control is that it avoids the possibility that the average picture is a distortion of the behavior of the individual participants, which is a potential problem whenever data are averaged over many participants.
So, let's consider the first graph that you see on your screen, the number of responses by trials. The one, this one right here. So, suppose this represents a learning curve of a group of participants on some task. Because the curve is a smooth, S-shaped curve, we might conclude from the group data that learning was a gradual, continuous process. But now, let's look at the graph right below it. This graph shows individual data for the five participants who make up the group in this top graph right here. This S-shaped curve graph here, we get a different picture. And each participant learns suddenly, going from no Ts on a single trial. So, participants learn on different trials. However, so participant one goes from no on trial six to complete mastery on trial seven. Participant two goes from a no on trial nine to complete mastery on trial ten, and so on and so forth. So, when the data of the whole group are averaged, though the learning appears, though the learning appears gradual. So, this example is extreme when you're comparing the two. It occurs fairly often in laboratory situations. So, what looks like a gradual progression when averaging across respondents, when looking at individual respondents, it's more of a rapid jump.
So, an experiment that employs large groups of participants will be likely to discover that an independent variable has an effect, even if the effect is a minor one. For example, given enough participants, it might be possible to show that a clinical treatment produced improvement in 55% of the participants, whereas 50% of the control participants improved spontaneously. So, the therapist is not likely to adopt a treatment that shows such a marginal difference in success rate. When we're talking about 5%, the experiment would have what's called little clinical significance, even if it had plenty of statistical significance. Right? So, 5% could still be a significant effect, but it's a really small one. So, the therapist learns a whole new type of intervention if it's only going to improve the effectiveness of the outcome or improve the positivity of an outcome by 5%. Probably not. They're going to want to have a treatment that's a lot more effective than just random variation or random improvement. Some researchers in non-clinical situations feel the same reluctance. They'd probably rather not spend time investigating the effects of variables that produce really small effects, but would rather find the powerful variables that produce really large effects. So, in a single-participant experiment, the effect of a very minor variable is less likely to be discovered. So, the experimenter will not be distracted by it. So, in addition, the researcher can spend time reducing variability so that the effect of a given variable will be maximized, instead of spending time testing more participants.
Statisticians use the term that you've probably familiar with from your stats classes, power, to refer to the probability that a statistical test will find a significant difference when there actually is a difference in the population from which the data are drawn. So, the power of the test depends on the size of the difference that exists in the population and the size of the sample drawn from the population. Therefore, a researcher has two ways of increasing the probability of finding a significant result of an experiment. They can increase the size of the effect or increase the size of the sample. The other tactic, and one favored by single-participant researchers, is to increase the size of the effect.
So, suppose that you're interested in whether the students at Alma Mater College, let's just give a fake name for a college, are smarter than those at Rival College. So, their rival college. The larger number of students sampled from each college, the greater the likelihood of finding a difference in intelligence between the groups. Eventually, if you include every student from both colleges, any difference you find is statistically significant because it's not based on the sample at all. You've measured the whole population. So, you're not performing an inferential statistic, you're not necessarily using parametric statistics, as we discussed in our previous lectures, but measuring the population value itself. You're actually, you don't need to approximate, you've got the whole thing. So, this statement is true even if the difference between the students of the two colleges is barely measurable, because it exists, it's significant.
So, suppose two researchers work on the same problem. Each measures the correlation between the same two variables. Researcher A uses ten participants and finds a correlation of .765. And Researcher B uses fifty participants and finds a correlation of .361. Well, both researchers find their correlations are significant at the .01 significance level. That is, there's a one chance, there's a one chance at a hundred that the correlation either researcher obtained does not reflect a true correlation between the two variables in the population studied. So, the question is, in which researcher's findings should you have greater trust? Should you put more confidence in Researcher B's results because more participants were used? Well, the answer is that you should feel more confident with Researcher A's results because the same level of significance was obtained with fewer participants. Remember that each had the same probability that the results were spurious, were random, one in a hundred. So, to get the same level of significance with fewer participants, Researcher A had to obtain a larger effect.
Now, this fact is shown by Researcher A having found a correlation that's larger than Researcher B's. And the square of the correlation coefficient gives us the percentage of the variance in the data that's accounted for by the independent variable. So, Researcher A's correlation of .765 accounts for 58.5% of the variance, whereas Researcher B's correlation of .361 accounts for only 13% of the variance. So, Researcher A obtained a larger correlation, and the independent variable accounts for a greater percentage of the variance, even though fewer participants were used. So, Researcher A must have had better control over the sources of variability in the study. This is why accounting for confounding variables and accounting for noise in the data is so important.
So, whenever research involves testing the efficacy of a treatment that's expected to benefit the participant, an ethical question arises over placing some participants into a control group, so that comparison group that will not receive treatment, or that will receive an inferior treatment that you're comparing your treatment of interest to. So, in clinical psychology, this area is particularly touchy when the client situation can be life-threatening, as with patients with active suicidal, kind of suicidal tendencies, suicidality. So, one solution is to treat all participants, but to evaluate them from a single-participant standpoint. So, this is actually where, if you're taking an ethical approach on something like this, it would be beneficial to use a single, single-subject research design.
So, another situation that calls for a single-participant experiment is when the researcher cannot locate enough participants to constitute a group to study. So, perhaps the researcher is testing the efficacy of a clinical treatment. There may not be enough people suffering from the same condition that are willing to participate in your study. Some participants will have to be studied on a single-subject basis. In case another, I, another point to consider is that an experiment on a group of subjects must be designed so that all subjects receive the same experience and can be compared. This necessity can result in...