Immediate effects of 35 minutes of online content, delivered in a single class session, on Year 9 students’ STEM motivation, measured as expectancy, intrinsic value, utility value and (low) cost, each relative to the student’s own baseline.
A brief, structured goal-setting activity raised students’ confidence in and interest in STEM right after the session. Watching videos of women STEM role models did not produce a clear benefit, and combining the two in one session added nothing extra.
Students were randomized to an active control, goal setting only, role modeling only, or a combined condition. Cost is reverse-scored as “low cost”, so higher is better on all four outcomes. See the full design →
Compared with the active control, students in the goal-setting-only condition reported higher expectancy (+0.17 SD) and intrinsic value (+0.18 SD) immediately after the session. Averaged across all conditions, goal setting had the same effect. The effects are small, and they held after correction for multiple testing. Exploratory subject analyses showed similar gains: higher expectancy in biology and chemistry, and higher intrinsic value in biology and physics.
Conditions with goal setting were linked to higher perceived cost (effort, stress, sacrifice). In the planned contrasts, only the combined condition had significantly higher perceived cost than the active control (−0.18 SD on low cost, q = .027). Its shortened sequence left out strategy selection and the if–then plan, the steps aimed at managing difficulty, so students may have registered the demands of STEM without a plan for meeting them. For goal setting alone the estimate was imprecise (−0.14, q = .45).
In the planned contrasts, role modeling alone differed from control by no more than 0.04 SD on any outcome (all q ≥ .89). Similarity, identification and attainability with the models were not measured, so the study cannot tell whether the format, the dose or the mechanism was the limiting factor.
Against control, the combined condition raised expectancy modestly (+0.08 SD, q = .027) but not interest or usefulness. It was no better than the average of the two single activities on any outcome. Fitting both into one session meant cutting the goal review and if–then plan, so the result may reflect a diluted goal-setting dose rather than interference.
Contrary to our prediction, neither goal setting nor the women role models benefited girls more than boys on overall STEM motivation; no gender interaction survived correction. In exploratory arm-level contrasts, goal setting alone raised girls’, but not boys’, intrinsic and utility value for chemistry.
In the three rural schools, but not the three urban schools, goal setting alone raised how useful students found STEM (+0.46 SD), a pattern repeated in biology, chemistry and physics, and role modeling alone raised expectancy (+0.11 SD). Location is confounded with school, so these are leads for replication rather than established rural–urban effects.
The preregistered planned contrasts compare each condition, as delivered, directly with the active control on overall STEM motivation. Goal setting alone produced the largest gains, with significant improvements in expectancy and intrinsic value.
Figure 2. Planned contrasts vs the active control · STEM overall
Table 1. Planned contrast estimates · Δ [95% CI], STEM overall
| Outcome | Goal setting onlyvs active control | Role modeling onlyvs active control | Combinedvs active control | Combinedvs average of single arms · exploratory |
|---|---|---|---|---|
| Expectancy | +0.17*[+0.03, +0.32]q = .034 | +0.04[−0.04, +0.12]q = .893 | +0.08*[+0.02, +0.14]q = .027 | −0.03[−0.14, +0.08]p = .601 |
| Intrinsic value | +0.18*[+0.04, +0.33]q = .034 | +0.01[−0.04, +0.07]q = .893 | +0.06[−0.02, +0.14]q = .177 | −0.04[−0.15, +0.07]p = .501 |
| Utility value | +0.17[−0.09, +0.43]q = .271 | +0.04[−0.14, +0.23]q = .893 | +0.08[−0.04, +0.19]q = .179 | −0.03[−0.15, +0.10]p = .658 |
| Low cost | −0.14[−0.49, +0.21]q = .447 | −0.01[−0.27, +0.25]q = .946 | −0.18*[−0.33, −0.04]q = .027 | −0.11[−0.46, +0.23]p = .525 |
We tested whether the effects on overall STEM motivation depended on student gender (confirmatory, with a predicted advantage for girls) or school location (exploratory, no prediction). Measurement invariance across gender held for all four beliefs, so boys’ and girls’ scores can be compared directly.
Table 2. Gender moderation · confirmatory
| Outcome | Goal setting × gender | Role modeling × gender |
|---|---|---|
| Expectancy | −0.001p = .991q = .991 | +0.15p = .052q = .070 |
| Intrinsic value | −0.12p = .141q = .514 | +0.12p = .296q = .296 |
| Utility value | −0.07p = .564q = .753 | +0.15p = .047q = .070 |
| Low cost | +0.13p = .257q = .514 | +0.32p = .030q = .070 |
Figure 3. School-location moderation · exploratory