Abstract
Scientific education provides a set of tools to make sense of a complex world by teasing out complicated cause-and-effect relationships, such as the elimination of effects of confounding factors in controlled experiments. There is evidence that depth of understanding of controlled experiments is lacking among undergraduate science students despite exposure to controlled experiments in courses. To examine the understanding of controlled experiments, we developed a two-tiered assessment that includes closed-ended and open-ended questions, with three types of questions, i.e., (i) a scientific scenario about a flawed drug study, (ii) an everyday-life scenario about flawed thinking regarding product effectiveness, and (iii) a direct question about explaining controlled experiments. Consistent with previous findings, we demonstrated that large percentages of students in introductory biology courses at both a research-intensive institution and a primarily undergraduate, minority-serving institution failed to recognize the need to account for confounds. Based on these findings, we tested the hypothesis that scientific literacy could be improved through a course-based intervention using an active learning framework focused on science as a process of evaluating alternative hypotheses. We found start-to-end-of-semester improvement in students' identification of unaccounted confounds with a scientific scenario in an intervention course but not in the control course. Interestingly, students in both the control and intervention courses showed improvement when tested with a scenario based on everyday life. The study findings suggest that a concerning number of college students may not account sufficiently for uncontrolled variables in real-life situations, and we present a widely applicable instructional strategy that improves on this broadly relevant scientific reasoning skill.