Counterbalancing noise, randomization, and power analysis for a reading study

[Lily68]

I'm designing a psychology experiment for my master's thesis to test the impact of different types of background noise on concentration during a reading comprehension task. I'm struggling with the experimental design, specifically how to properly counterbalance the order of noise conditions (silence, white noise, cafe chatter) across participants to avoid order effects, and whether I should use a within-subjects or between-subjects design given my relatively small potential sample size from the undergraduate pool. What are the best practices for randomization and power analysis in this scenario?

[LukeST]

Within-subjects with three conditions is doable and efficient. Use a counterbalancing scheme like a Latin square (or Williams design) so each noise type appears in every position and carryover is evenly distributed. Keep sessions close enough to feel cohesive but with a short break between blocks to avoid fatigue. Do a quick pilot to estimate effect sizes before locking in the plan.

[Scott22]

Power/randomization tip: for a within-subjects ANOVA with 3 conditions, a medium effect size (f ≈ 0.25) at alpha 0.05 and power 0.8 typically needs roughly 28–34 participants, assuming moderate correlation among measures. If you expect a smaller effect or more noise, plan 40+. Use G*Power or the pwr package in R to run exact numbers and input your estimated correlation and nonsphericity epsilon.

[AveryQA]

Here are design-options: (a) complete counterbalancing with the 3! = 6 orders across three sessions; (b) a Williams design to balance order and potential carryover more evenly; © if fatigue is a real concern, split into two shorter sessions and randomize condition order across sessions. Also include a baseline reading task to covary individual differences.

[LoganWC]

Decide on how you’ll measure concentration: reading comprehension accuracy, speed, or a validated attention task. Use a reliable, standardized test if possible. Collect a simple baseline (e.g., a short reading task under quiet) to use as a covariate. Ensure randomization is automated so each participant sees conditions in a random order.

[EvelynC]

Consider a between-subjects option only if your pool is tiny and you can't recruit enough for a robust within-subject design. In that case, three groups (one per noise condition) with around 40–60 participants total per group is more common, but you’ll need much larger N to achieve similar power due to inter-subject variance.

[JonathanG]

Practical logistics: pre-register your design, specify the randomization scheme, and use an experiment platform or simple scripts to randomize orders. Keep testing sessions consistent (same room, same time of day, same noise setup). Plan for potential dropouts and have a plan for handling missing data.