From Allowed to Embedded: AI Policies Across 71 Engineering Courses

The authors call the design participatory action research: students did more than answer a survey. They audited their own courses. Each report described one course a student had taken, which of five kinds of AI policy it had, how clear that policy was, and what the student used AI for. Thirty students answered an open call through course announcements, department mailing lists, and word of mouth, and together they described 71 different courses. Most reports came from computer engineering (67) and computer science (44), with 16 from electrical engineering and one from general engineering.

Pick a policy and look at the largest number. Courses that ban AI still show students using it: 15 of 24 reports describe at least one use, most often to explain a concept or a lecture. The authors call this the shadow curriculum, learning that happens with AI whether or not the course plans for it. Move to Critical Use and every one of the nine reports describes using AI, and two in three describe critiquing its mistakes.

Each bar is a percentage of that policy’s reports, and the count beside it shows where the percentage comes from: 15 of 24 Disallowed reports is 15 ÷ 24 = 0.625, or 62.5 percent. Students could check several tasks, so the bars for one policy add to more than 100 percent. Compare the Debugging bar across the five policies: 29.2 percent under Disallowed and 88.9 percent under Critical Use.

The continuum orders policies by how deeply AI enters the course, from prohibition to required use with critique. The task profile follows that order for the tasks that demand the most from a student. Debugging rises from 29.2 percent (Disallowed) to 88.9 percent (Critical Use), verification of math and logic from 16.7 to 66.7 percent, and critique of AI errors from 16.7 to 66.7 percent. Explanation peaks in the middle of the scale instead: 72.3 percent of Allowed (Informal) reports against 55.6 percent of Critical Use reports.

Table 2’s column sizes (24, 21, 47, 27, and 9) sum to 128, and every printed percentage corresponds to exactly one whole-number count, which the bars show. The rows are binary items from a multi-select list, so a column describes the share of reports marking each task; it does not say how many tasks a report marked or which travel together. The Critical Use column rests on nine reports, the paper’s smallest group.

Reports grow from eight in Spring 2023 to 31 in Fall 2025, as more students joined and more courses settled on a policy. The authors read the period in three phases. In 2023, courses reacted: half of the first semester’s reports name no policy at all, and by that fall three in four describe informal permission. Through 2024 and early 2025, vague permission led while bans faded. By Fall 2025 the picture had split five ways, with Encouraged courses tied with informal permission at 29.0 percent and Critical Use, absent before Fall 2024, at 12.9 percent.

Pick a semester to see the policy shares the paper’s text gives and the number of reports behind each one. Fall 2025’s five shares (29.0, 29.0, 16.1, 12.9, and 12.9 percent) come from 9, 9, 5, 4, and 4 of 31 reports; added up they make 99.9 percent, because each was rounded on its own. Summer terms carry only two to nine reports, so one report moves a summer share by 11 percentage points or more.

Table 1 gives the sample per semester; Figure 1 gives the policy mix per semester as stacked bars, and the text states exact shares only for selected semesters. The Disallowed share falls from 35.7 percent (5 of 14) in Spring 2024 to 14.8 percent (4 of 27) in Spring 2025 and 12.9 percent (4 of 31) in Fall 2025. Encouraged (22.7 percent) and Critical Use (13.6 percent) first appear in Fall 2024. No Policy Stated never disappears: 16.1 percent of Fall 2025 reports still describe a course that says nothing.

Unique courses per semester sum to 110 against 71 unique courses overall, so many courses appear in more than one semester, and the same course can be reported by different students. The early semesters carry the phase claims on small denominators: the Phase 1 percentages rest on eight and 12 reports, and every undergraduate-only summer term on nine or fewer. The full Spring 2023 and Fall 2025 breakdowns are stated in the text; for the other semesters the page shows only what the text states and marks the rest.

Students rated how clear each policy was, from 1 (very unclear) to 5 (very clear). A ban felt clear (4.50). So did encouragement (4.37) and required, critical use (4.44). Informal permission, the most common policy in the study, averaged 2.45. Students found a firm “no” far clearer than a vague “yes.” The gap holds for undergraduate and graduate reports alike.

An average hides how answers spread out. The paper reports one spread, the standard deviation, for Allowed (Informal): 1.14, wide on a five-point scale, which means the answers for that policy spread widely across the scale. For the three explicit policies the averages sit within 0.13 of each other, and the paper’s tests find no real difference among them. The difference that matters is about two points, between informal permission and everything else.

The one-way ANOVA across the four compared policies gives F(3, 103) = 38.84, p < .001, and each Welch t-test of Allowed (Informal) against an explicit policy clears the Bonferroni-adjusted α of 0.0083 with an effect size near two standard deviations (d = −2.15, −1.96, and −2.09). A Kruskal–Wallis test, which uses ranks and suits a five-point item, agrees: H(3) = 58.7, p < .001. The authors note one caution: Allowed (Informal) is partly defined by vague wording, so low clarity there is partly built into the category.

The histograms are not in the paper. The page searched every possible 1 to 5 answer table for the four groups and kept only those matching all printed statistics; the last demo shows the search. One table survives, and every test in the comparison table is recomputed from it. Cohen’s d reproduces in its average-variance form (the mean difference over the root mean of the two variances), and H reproduces with the tie correction. The 95 percent intervals use a t quantile on each recovered group and assume independent reports, which the notes below question.

Read across the “At least one task” row. Use of AI rises with each step up the policy ladder, from 62.5 percent in banning courses to 100 percent in Critical Use courses. Even where AI was banned, the most common use was to explain a concept or a lecture. The authors describe this use as mostly study help that students did not consider cheating.

Switch to Count of reports to see the whole numbers behind each percentage. The Abstained row is the At least one task row subtracted from 100 percent: 9 of the 24 Disallowed reports abstained, and 15 used AI. Pick a policy to highlight its column, then find the task with the largest gap between Disallowed and Critical Use.

Two patterns stand out. Engagement with AI output rises with integration: critique of AI errors is 8.5 percent under Allowed (Informal), 25.9 percent under Encouraged, and 66.7 percent under Critical Use. Abstention falls the other way, from 37.5 to 0 percent. Where a course designs for critique, students report practicing it.

Each column is a separate policy group, and the reports within a column are not independent (one student can report several courses). The Wilson intervals below the table treat reports as independent draws and so understate the uncertainty; the Disallowed interval for any use runs from 42.7 to 78.8 percent even so. The paper offers no test across columns, and the page adds none.

Recovering the answers behind the clarity tests

The search enumerates every way to split each group’s ratings across the five answers (about 0.3 million tables), keeps those whose mean, and for Allowed (Informal) standard deviation, round to the printed values, then keeps pairs whose Welch t and Cohen’s d against Allowed (Informal) round to the printed values. The search keeps 57 Allowed tables, two Disallowed, two Encouraged, and one Critical Use table, and every one of the 228 combinations gives F = 38.84. Only one gives H = 58.7. The recovered table is the page’s reconstruction, not data the paper prints, and it would be confirmed or corrected by the authors’ item-level file.

• Outright bans on AI faded over the study: from a quarter of the first semester’s reports to 12.9 percent by Fall 2025.
• Vague permission (“allowed,” with no guidance) became the most common policy, peaking at three in four reports in Fall 2023.
• Students rated vague permission as the least clear policy, far below bans, encouragement, or required use.
• In courses that banned AI, 62.5 percent of the reports still describe using it, mostly to explain concepts.
• In courses that required students to use AI and check its work, every report describes using it, and students reported the most practice in critiquing its mistakes.

• Disallowed reports fell from 25 percent in Spring 2023 and 35.7 percent in Spring 2024 to 12.9 percent in Fall 2025.
• Allowed (Informal) peaked at 75 percent (9 of 12 reports) in Fall 2023 and was still 29.0 percent in Fall 2025.
• Mean clarity: Allowed (Informal) 2.45, against 4.50 (Disallowed), 4.37 (Encouraged), and 4.44 (Critical Use).
• 62.5 percent of Disallowed reports (15 of 24) describe AI use for at least one task.
• Critique of AI errors: 66.7 percent of Critical Use reports against 8.5 percent of Allowed (Informal) reports.

• Bans receded. Disallowed fell from 35.7 percent of reports (Spring 2024) to 14.8 percent (Spring 2025) and 12.9 percent (Fall 2025).
• Ambiguity filled the space. Allowed (Informal) peaked at 75 percent in Fall 2023 and remained the most prevalent category through 2024 and early 2025; by Fall 2025 it tied Encouraged at 29.0 percent.
• A clarity gap. Allowed (Informal) 2.45 of 5 (SD 1.14, n = 47) against Disallowed 4.50, Encouraged 4.37, and Critical Use 4.44; F(3, 103) = 38.84, p < .001; no significant differences among the three explicit policies.
• The gap holds by level. Undergraduate reports: 2.51 (n = 37) against 4.30 (n = 20); graduate reports: 2.20 (n = 10) against 4.57 (n = 7).
• A shadow curriculum. 62.5 percent of Disallowed reports describe AI use for at least one task, led by explanation (45.8 percent).
• Integration shapes the task. Debugging, verification, and critique rise toward Critical Use (88.9, 66.7, and 66.7 percent).

The authors ask engineering programs to replace silence and vague permission with explicit guidance, since every explicit policy, a ban included, rated far clearer than informal permission. Beyond clarity, they propose teaching with AI the way engineers treat any tool that can be wrong: “Trust but Verify.” In a “Verification First” course, assignments assume students have AI, and grading rewards checking its output. Ultimately, that turns the shadow curriculum into a planned one.

The authors propose extending the audit to other disciplines and institutions, adding faculty perspectives, and relating policy type to student performance. The companion paper, Trust but Verify, surveys 147 students in the same department on which tools they use and how often they check the output.

Students in engineering and computing will work with AI tools that are often right and sometimes wrong. Course policies decide whether they practice checking that work in class or alone. This study shows that the policy students saw most often told them the least, and that the courses which built AI into the work produced the most checking.

Course policy is cheap to change and sets the conditions for everything else an instructor does with AI. A department can rerun this audit each semester with the same five categories and one clarity item, and track whether its courses move from informal permission toward explicit, verification-centered designs.

The continuum gives programs a shared vocabulary for course-level AI policy, and the student-audit design captures what a syllabus review cannot: the policy as students understood it and the use it did not prevent. Because the clarity statistics are complete enough to recover their answer distribution, later waves or other institutions can compare against these 107 ratings at the level of individual answers.

Notes from building this page

• One answer table fits. Means and the Allowed (Informal) standard deviation alone leave 57 tables for that group; adding Welch t and Cohen’s d leaves 228 four-group combinations, all with F = 38.84; Kruskal–Wallis H = 58.7 selects one. In it, 14 of 47 Allowed (Informal) ratings are 1 (very unclear) and 21 are 3, with only six at 4 or 5; 23 of 24 Disallowed and 24 of 27 Encouraged ratings are 4 or 5. The bimodal Allowed distribution (a block of 1s and a block of 3s) may separate courses that students read as silent from courses they read as permissive, a split the sentiment items could test.
• Effect size and rank test conventions. The printed d values are average-variance d (−2.15, −1.96, −2.09); pooled-variance d gives −2.02, −1.87, and −1.84. H = 58.7 includes the tie correction (55.0 without). Stating both conventions would let readers reproduce the values directly.
• Reports are nested in students. Thirty students contributed 128 reports, about 4.3 each, and some courses appear in several reports. The t-tests, ANOVA, Kruskal–Wallis, and the page’s intervals all treat reports as independent; a model with a student random intercept, or a sensitivity analysis keeping one report per student and course, would need the student identifiers.
• Who classifies the policy. The same student classifies the policy and rates its clarity, so a course read as unclear can land in Allowed (Informal) for that reason; the paper names this overlap and treats the comparison among the three explicit policies as the more interpretable one. Those three do not differ (Welch p = 0.54, 0.85, and 0.80 on the recovered counts).
• Two summary statements to check against the tables. The conclusion credits Critical Use with the highest clarity, but its mean (4.44) is below Disallowed (4.50); ethical confidence, the other half of that claim, is not quantified in this paper. The text says critique was rarely reported outside Critical Use; Table 2 gives 25.9 percent under Encouraged and 16.7 percent under Disallowed.
• Figure 1 in numbers. The text states complete policy shares for Spring 2023 and Fall 2025 only. Critical Use totals nine reports, three in Fall 2024 and four in Fall 2025, so the other two fall in Spring or Summer 2025 if Fall 2024 is the first appearance. Printing Figure 1’s counts as a table would let readers check each phase claim.
• Scope of the sample. An opt-in sample that includes lab members probably over-represents students engaged with AI; the authors read the 62.5 percent shadow-use rate as an upper bound. The Wilson interval on that share (42.7 to 78.8 percent, page’s own) is wide even before accounting for clustering.

Try this in class

1. List five courses you have taken (or five classes this year) and place each on the paper’s five-level continuum: Disallowed, No Policy Stated, Allowed (Informal), Encouraged, or Critical Use. Rate each policy’s clarity from 1 to 5.
2. Pool the class’s answers. Compute the average clarity for each policy, and count how many of the Disallowed courses still had students use AI for at least one task.
3. Compare your averages with the The clarity gap demo, and write two sentences on which kind of policy your class found clearest and why.