To the research!

Researchers from a sleep laboratory were interested in examining the effects of sleep deprivation on short-term memory. Short-term memory holds a limited amount of information for seconds to minutes.

100 adults were randomly assigned to either the study group (n=50) or the control group (n=50). In the study group, participants were not permitted to sleep for 24 hours, whereas in the control group, participants slept normally (8 hours of sleep from 11:00 pm - 7:00 am).

Individuals from both groups spent the 24 hours prior to testing (at 8:00 am) in the laboratory, where they were observed.

The critical test and other measures

Participants were given a short-term memory task, for which they had to…

  1. memorize a list of 20 words in 2 minutes,

  2. do math problems (a filler task) for 2 minutes, and then

  3. recall as many as they could in 2 minutes.

The number of words they could recall accurately was the response variable.

Additional information was collected about the participants. We’ll consider participant age and sleep habits.

The data

x <- read.csv('slpdata.csv')
str(x)
## 'data.frame':    100 obs. of  5 variables:
##  $ ppt      : int  1 2 3 4 5 6 7 8 9 10 ...
##  $ condition: chr  "control" "control" "control" "control" ...
##  $ memory   : int  13 11 15 13 10 11 11 15 11 11 ...
##  $ wk.slp   : int  6 8 9 5 5 5 7 5 4 7 ...
##  $ age      : int  31 22 44 33 24 66 55 25 20 20 ...

“memory” is the number of recalled words (out of 20). “wk.slp” is the average number of hours slept during the week before the experimental session.

2.1 Group differences in memory performance

Visualize the effects of sleep deprivation on short-term memory using boxplots. What do you observe?

Compare the mean number of words recalled in each group. What do you observe?

What interim conclusion might you draw?

2.1 Group differences in memory performance

boxplot(memory ~ condition,
        data = x,
        ylab = "memory (number of items recalled)",
        main = "Effects of sleep deprivation on memory")

2.1 Group differences in memory performance

aggregate(memory ~ condition, data = x, FUN = mean)
##   condition memory
## 1   control  11.98
## 2    depriv   7.60

Sleep deprivation impairs short-term memory. In this experiment, participants in the study group remembered 4.4 fewer words than those in the control group. That’s a reduction of about one third.

As we saw on the boxplots, there is a fairly large spread of scores in both groups. But the difference between the group medians (or means) is still clear.

2.2 Potential confounds

Recall that a confounding variable is one that causes or prevents the outcome of interest (for example, memory problems), other than the variable of interest (for example, sleep deprivation).

In Activity 1, you discussed potential confounds in an RCT of sleep deprivation’s effect on memory. Which of those confounds apply here? Which are guarded against? How?

2.2 Potential confounds

Some confounds are hard to remove entirely. For example:

  • Participants will know whether they have been deprived of sleep.

  • Sleep-deprived participants will do things (read, play games, etc.) instead of sleeping.

  • Control participants may have trouble sleeping in the lab.

But for many potential confounds (such as caffeine intake or baseline memory ability), random assignment with a sufficient sample size reduces the problem. Potential confounding variables are spread randomly across our groups.

Potential confounds: Checking

Random doesn’t guarantee equivalence! So let’s check where we can.

2.3 Potential confounds: Checking sleep habits

Recall that we have data for each participant’s average hours of sleep during the week before the experiment.

  • Examine if there are any differences in wk.slp across the groups.

  • What do you observe?

  • Consider a hypothetical situation. What if participants in the control group had longer hours of wk.slp compared to those in the study group? How might that situation affect our conclusions about the effect of sleep deprivation?

2.3 Potential confounds: Checking sleep habits

boxplot(wk.slp ~ condition, data = x,
        ylab = "Average hours of sleep (wk.slp)",
        main = "Sleep habits by group")

2.4 Potential confounds: Checking age

Each participant’s age was also measured.

  • Examine if there are any differences in age across the groups.

  • What do you observe?

  • Consider a hypothetical situation. What if participants in the control group were older compared to those in the study group? How might that situation affect our conclusions about the effect of sleep deprivation?

2.4 Potential confounds: Checking age

boxplot(age ~ condition, data = x,
        ylab = "Participant age (years)",
        main = "Age by group")

Exploring further

It’s good that age didn’t differ by group. Short-term memory performance declines (gracefully) with age.

Can we see such effects in our sample?

And does sleep deprivation affect the relationship?

2.5 Relationship of age and memory by group

Separately for each group, perform the following:

  • Create a scatterplot of memory as a function of age.

  • Perform a linear regression analysis on the same. What do the slopes mean?

  • Add the regression line to the plot.

  • Compute the correlation and R-squared. What do they mean?

2.5 Relationship of age and memory by group

control <- x[(x$condition == "control"), ]

reg.control <- lm(memory ~ age, data = control)
reg.control
## 
## Call:
## lm(formula = memory ~ age, data = control)
## 
## Coefficients:
## (Intercept)          age  
##    14.24206     -0.06838

\[\text{memory} = 14.24 - 0.068\,\text{age}\]

2.5 Relationship of age and memory by group

plot(memory ~ age, data = control,
     xlab = "age",
     ylab = "memory",
     main = "relationship between age and memory for control group")

abline(reg.control, col = "blue", lwd = 3)

2.5 Relationship of age and memory by group

cor(control$memory, control$age)    
## [1] -0.3691382
cor(control$memory, control$age)^2 #Same as R-squared from lm() 
## [1] 0.136263

2.5 Relationship of age and memory by group

depriv <- x[(x$condition == "depriv"), ]

reg.depriv <- lm(memory ~ age, data = depriv)
reg.depriv
## 
## Call:
## lm(formula = memory ~ age, data = depriv)
## 
## Coefficients:
## (Intercept)          age  
##     12.0019      -0.1286

\[\text{memory} = 12.00 - 0.13\,\text{age}\]

2.5 Relationship of age and memory by group

plot(memory ~ age, data = depriv,
     xlab = "age",
     ylab = "memory",
     main = "relationship between age and memory for sleep deprivation group")

abline(reg.depriv, col = "red", lwd = 3)

2.5 Relationship of age and memory by group

cor(depriv$memory, depriv$age)    
## [1] -0.5394467
cor(depriv$memory, depriv$age)^2  
## [1] 0.2910027

2.6 Informally comparing relationships across groups

To better illustrate the group differences for the relationship between age and short-term memory, create a single plot with the data points and regression lines for each group.

What conclusions do you draw? How certain are you about your conclusions? What factors affect your certainty?

2.6 Informally comparing relationships across groups

plot(memory ~ age, data = control,
     col = "blue",
     xlab = "age",
     ylab = "memory",
     main = "relationship between age and memory",
     ylim = c(0, 17))

points(memory ~ age,data = depriv,
       col = "red")

abline(reg.control, col = "blue", lwd = 3)

abline(reg.depriv, col = "red", lwd = 3)

2.6 Informally comparing relationships across groups

Drawing conclusions:

  • 24 hours of sleep deprivation caused poorer short-term memory performance compared to a night of normal sleep.

  • Age is associated with declining short-term memory performance. This conclusion was possible because correlation analyses can often be performed within experiments.

  • The negative slope of short-term memory performance as a function of age appeared to be steeper during sleep deprivation. That is, sleep deprivation may have made age-related decline more apparent and more severe.

  • Age and sleep deprivation may interact. That is, their combination would be different from either effect alone. Most effects have multiple causes! Subsequent courses explore this idea in detail.

  • Keep in mind that these conclusions all have a degree of uncertainty. How much? That’s (largely) for another day…

Recap of learning goals

Today, we sought to:

  • Identify strengths of experiments for drawing causal inferences. We focused specifically on a certain type of experiemnt, the randomized controlled trial (RCT).

  • Understand the elements of a good experiment.

  • Be mindful of the difficulties and limitations in using experiments, including practice and ethical considerations.

  • Develop an intuition for when results meaningfully differ across groups.

  • Review implementing and drawing inferences from boxplots, scatterplots, correlations, and linear regressions.