Time for Practice: Three Astronomy Summers for K–12 Teachers

Pick a summer. In 2014, teachers spent most of each day in planned astronomy activities. In 2015, the team flipped the day: teachers spent most of it on research projects of their own, with faculty nearby. In 2016, RAMPED offered six topics (computer science and modeling, virtual reality, Arduino, Raspberry Pi, robotics, and space), and each teacher chose four. Each year the teachers said what they still needed, and each answer shaped the next design.

Watch the “What teachers still needed” row change: room to fail in 2014, more structure in 2015, and fewer topics in 2016. The audience for teachers’ final presentations also grew, from peers only, to about five visitors, to more than 15. The “Who came” bars show how different the groups were: the 2015 cohort was mostly elementary teachers (16 of 22), and middle school teachers made up the largest share of the 2016 cohort (14 of 30).

LASSI served 30 teachers across 25 days of an astronomy research project over the 2014 and 2015 summers, with a university telescope and a campus observatory. RAMPED served 30 teachers across 16 days: ten summer days in which each teacher took four of six sessions, plus six days during the school year, with an emphasis on web-based communities and Jupyter notebooks. Sessions ran in two-day blocks (a day and a half of content and expert instruction, half a day of lesson planning), and the same principal investigator designed all three with university faculty and school districts. Teachers self-selected through a district listserv and direct email, and some attended more than one summer.

In the 2016 space session, teachers worked with the Sloan Digital Sky Survey (SDSS), a public database of millions of objects photographed in five colors. Stars and quasars (the bright centers of distant galaxies) both look like single points of light, so astronomers tell them apart by color. Slide the cut, press Reveal, and try to catch every quasar without grabbing any stars. This demo stands in for that exercise; the points are simulated.

A quasar gives off more ultraviolet light than a star of the same green brightness, so its u − g color (ultraviolet magnitude minus green magnitude) is small. Astronomical magnitudes run backward: a smaller number means brighter. Move the cut right and you catch more quasars but also more stars; move it left and you miss quasars. Every cut is a tradeoff, and finding a good one takes practice.

The plot is a stand-in for the 2016 work the paper describes (finding changing-look quasars in SDSS and, in teachers’ words, telling quasars from stars by their point-source attributes). The simulated colors follow the shape of real SDSS data, with stars along a curved locus and quasars at small u − g, the ultraviolet excess that early SDSS quasar selection used; the paper does not describe the exact exercise. Completeness and purity trade off as the cut moves, which makes the demo a quick lesson in classification error for any STEM classroom.

Each summer measured learning a little differently. In 2015, teachers took short tests before and after each topic, and scores rose on five of seven topics. In 2014, a standard test of common misconceptions showed that most teachers did well, while five of eight still misunderstood the shape of Earth’s orbit. In 2016, the space session’s clearest gain was practical: before, 15 percent of teachers knew where astronomers share their data; afterward, 90 percent did.

The 2015 tests had three to five questions per topic. The two topics without significant gains (Galileoscopes, and the Doppler effect with angular size) were among the most technical, with four technical questions for every conceptual one. The paper notes that its 2015 questions leaned technical and were the same for elementary and secondary teachers.

MOSART (the Misconceptions-Oriented Standards-Based Assessment Resources for Teachers) is a 20-item Harvard-developed test covering astronomy, chemistry, earth science, and physics. In 2016, the pooled MOSART items rose from pre to post (t = 2.38, p = 0.0244), and half of the teachers scored above 60 percent. The paper treats p above 0.001 as not significant in its methods, yet calls this change statistically significant and, given its effect size, trivial.

Matched pairs reproduces both printed 2016 statistics. With t = 2.38 and 29 teachers (Table 3’s 29 of 30), the two-sided p is 0.02436, which rounds to the printed 0.0244; with r = 0.94, the change divided by the average of the pre and post standard deviations is d_av = 0.153, the printed 0.15. The standardized mean change that the paired test uses is d_z = t ÷ √n = 0.44. With r = 0.94, individual changes vary far less than the scores themselves, so d_z runs about three times d_av, and the two readings (“trivial” against “small to medium”) describe the same data. On 2016 space session, the shaded block shows that a paired test on 20 people cannot reach the printed p < 0.00001.

Teachers liked all three summers, with about three positive comments for every negative one. Where the paper reports both weeks, ratings rose from the first to the second, most sharply in 2015, when the first week felt too complex and the second felt about right. The space session in 2016 drew mixed reviews: 14 of 20 teachers found it moderately or largely useful, and five found it of little or no use.

Each bar uses its own scale (0 to 4, 0 to 10, or percent), so compare week one with week two within a row, not across rows. The bottom chart shows the space session’s four answers. The printed percentages add to 95, which means one of the 20 teachers did not answer; the mean of 1.95 comes out right only when that teacher is left out.

The Science Teaching Efficacy Belief Instrument (STEBI, 25 items on a five-point scale) went the other way: mean personal science teaching efficacy fell from pre to post, and no change was significant. Teachers’ notebooks in 2015 moved “from excitement, to despair, to understanding,” and in 2016 some teachers felt overwhelmed early and expected to pick up Python quickly. Engagement rose and fell, some teachers preferred directed activities to open-ended inquiry, and the quote that titles the paper came from that 2016 struggle.

After each summer, teachers wrote lesson plans to teach the next school year, and in 2016 fellow teachers visited 21 classrooms to watch. Most science lesson plans used astronomy. In the observed lessons, seven of 21 did, and none built a lesson around real-world science research, the kind of work teachers had practiced in the summer. Planning a lesson and teaching it are different skills, and the second one needs practice too.

The seven observed lessons ran from elementary math (orders of magnitude) to high school physics (plasma and elliptical orbits), so astronomy did reach many subjects. In 2016, four teachers (13 percent) were unsure how they would use what they learned, and eight (27 percent) wrote only general statements about using it.

The focus of lesson plans shifted with each design: science and mathematics in 2014, science and astronomy in balance in 2015, and technology in 2016, when RAMPED’s sessions centered on technology. Observations were done by peers from the same PLD for at least 30 minutes each, guided by five open questions rather than a formal protocol. The Discussion counts seven of 21 observed lessons (33 percent) as integrated STEM and six of 21 (29 percent) as using astronomy as the vehicle for a STEM discipline.

• Teachers learned new STEM content in all three summers, and what they learned tracked closely with the activities they did.
• In 2016, teachers who could name a public astronomy data source rose from 15 to 90 percent.
• Teachers rated the programs highly: 97 percent of 2016 teachers were moderately or extremely satisfied after 16 days.
• Teachers’ confidence in teaching science did not rise during the summer.
• Astronomy appeared in about a third of the classroom lessons peers observed.
• Teachers asked for time to practice, fail, and try again, both in the summer and in the classroom.

• Content gains, uneven by topic. Five of seven 2015 topics rose significantly (p < 0.0001); 2014 teachers scored high on MOSART with persistent misconceptions about stellar energy and orbital eccentricity; 2016 pooled MOSART items rose with an effect size the authors call trivial.
• High perceptions. Three positive comments for every negative one; 2015 quality rose from 2.32 to 3.67 (of 4) between weeks; 2016 satisfaction reached 93 and 97 percent.
• No efficacy gain. STEBI means fell, with no significant change.
• Limited classroom transfer. Astronomy appeared in 5 of 6, 13 of 16, and 8 of 15 science lesson plans, and in 7 of 21 observed 2016 lessons; no observed lesson had an authentic, real-world science focus.
• Design recommendation. Astronomy-based authentic projects, structured support for content and technology, extension activities, and time for practice, collaboration, and sharing ideas.

Science standards in many states (the Next Generation Science Standards) ask teachers to connect science, technology, engineering, and math, and many teachers learned those subjects separately. This study shows that a strong summer program can teach the content and earn teachers’ enthusiasm, and that the step into the classroom needs its own support. The authors recommend building practice time into every program: time to code, to collect and analyze telescope data, and to try a lesson, fail, and try again.

Integrated STEM appears in the Next Generation Science Standards (NGSS) through shared practices and crosscutting concepts, and PLD is where most working teachers meet it. The three designs here met the standard criteria for effective PLD (content, authentic inquiry, reflection, partnership, and more than 50 hours), yet integration in classrooms stayed rare. Ultimately, the paper moves the question from whether teachers can learn integrated content in a summer to how programs support them through the school year that follows.

The study follows one PLD team through three design iterations and into classrooms, and the contrast between high perceptions and limited transfer is the paper’s main contribution. Its practical recommendations (four parts, ending with time for practice) are testable in a next iteration, and its eight future questions, from why teachers hesitate to write their own research questions to which authentic astronomy experiences transfer best, set out a research agenda for astronomy-framed PLD. The data sources also make a reusable template: pre- and post-items, a validated misconception test, an efficacy scale, notebooks, plans, and peer observation.

Notes from building this page

• The 2016 statistics fit n = 29 and d_av. t = 2.38 gives the printed p = 0.0244 only with 28 degrees of freedom (n = 30 gives 0.0241), and the printed 0.15 matches d_av = d_z √(2(1 − r)) = 0.153. The paired-test effect size d_z is 0.44, so “trivial” depends on the standardizer; reporting both, with the pre and post means, would settle the reading. The change is also reported as significant at p = 0.0244 while the methods set 0.001 as the threshold.
• The 15-to-90 percent item. If the 20 space-session attendees took the item (15 and 90 percent are exactly 3 and 18 of 20), every pairing of their answers gives an exact McNemar p between 6.1 × 10⁻⁵ and 7.3 × 10⁻⁴, above the printed p < 0.00001. Fisher’s exact test (3.4 × 10⁻⁶) and a two-proportion z test (2.0 × 10⁻⁶) on two independent groups fall below it. The printed p suggests an unpaired test or a larger tested group; the paper does not name the test or the n. These tests are the page’s.
• One space-session answer is missing. The usefulness percentages (5, 20, 45, and 25) are of 20 attendees and sum to 95; the printed mean of 1.95 reproduces over the 19 who answered (37 ÷ 19), and 14 plus five accounts for 19 teachers.
• Cohort labels on two counts. “Eleven 2014 LASSI teachers (50%)” planned to use remote telescopes and databases, yet the 2014 cohort had eight teachers; 11 of 22 is 50 percent, as are the neighboring 13 (59 percent) and eight (36 percent) of the 2015 cohort. The 2014 drawing-item list names eight scores while Table 3 lists seven of eight teachers sampled (the written list names seven). Respondent counts also shift within 2015: 41 percent fits 9 of 22, and 76 percent fits 16 of 21 but no count of 22.
• “Over 60 percent of questions.” Table 3 says significant gains appeared in over 60 percent of 2015 content questions; Table 4 reports p by topic (five of seven, 71 percent) with three to five questions each, so the question-level count cannot be checked from the paper.
• Scales and research questions. 2014 ratings use a 1-to-5 scale and 2015 ratings a 0-to-4 scale, so cross-year quality comparisons need a common scale. The methods mention three research questions and then list five.
• Integration counts. Results describe seven observed lessons that used astronomy with six specific instances; the Discussion counts the seven as integrated STEM and six as using astronomy as the vehicle. Table 10’s rows sum to seven. A short rubric for “integrated” would make the two counts comparable across iterations.

Try this in class

1. In pairs, open Star or quasar and find two cuts: one that catches every quasar and one that makes no mistakes. Record both cuts and how many stars or quasars each one gets wrong.
2. Visit the free SDSS SkyServer, look up five objects labeled stars and five labeled quasars, and compute u − g for each from the listed u and g magnitudes. Check which of your two cuts sorts them correctly.
3. Share your cuts with the class, then repeat step 1 with New sky. Compare how quickly each pair found a good cut the second time: that improvement is the “time for practice” the teachers in this study asked for.