Soft Skills, Hard Numbers: Six Questions Before and After One Class

Try each question before you reveal the answer. Almost no one could answer the conflict questions: before the session, the class averaged 5.56 percent on naming three types of conflict and 2.78 percent on naming four ways to approach one. Every engineering team depends on these skills, and few students had ever been taught them by name.

Answers earned partial credit, so an average of 21.11 percent on Q4 means the class gave about a fifth of the full answer on average, not that a fifth of the students got it right. The biggest gain was on Q1 (what ABET is): from 16.67 to 95.56 percent, a jump of almost 79 points.

Questions 1 to 3 test whether students know the accreditation framework that shapes their own degree; questions 4 to 6 test whether they have vocabulary for working with others. The full-credit answers come from ABET’s Criterion 3 student outcomes for the Engineering and Computing Accreditation Commissions (EAC and CAC) and from the conflict-management work the paper cites (Burrows and Harkness), which the session covered.

The items are open response and the paper does not describe its scoring beyond the full-credit column. Every printed mean and standard deviation, however, falls on a grid of steps of one fifth (Q1 to Q4), one third (Q5), or one quarter (Q6) for 18 students, and no list of whole-credit (right or wrong) answers reproduces them. The steps match the number of parts in Q3’s, Q5’s, and Q6’s full answers.

The study had two groups. In one class, 18 students answered the six questions on paper, played a team game tossing paper balls into targets for points, heard a short explanation of soft skills and conflict, and answered again. A second group of 37 students answered the questions once, online, at the end of the school year. Together they gave the authors a snapshot of 55 students, from freshmen to seniors.

The online cohort’s class standing is not printed; the page derives it by subtracting the in-class counts from the totals (seven freshmen, eight sophomores, 16 juniors, and six seniors). The one-in-four response rate and the self-selection it implies apply to that cohort only.

Every question rose. The class went from knowing almost nothing about conflict to naming most of the answer, and from a sixth of a correct definition of ABET to nearly all of it. The rows by class standing show the same story for seniors, juniors, and the one sophomore.

Switch to Seniors (14) and compare the pre-test bars with All 18: seniors, close to graduation, scored 14 percent on what ABET is and 2 percent on each conflict question. Switch to Juniors (3) or Sophomore (1) and remember the sample size: with one student, a single answer is the whole row.

Table IV’s averages are the in-class means with standard deviations; every item’s gain sits between 64 points (Q6, 2.78 to 66.67 percent) and 79 points (Q1). The post-test standard deviations for the conflict items (43.62 and 46.18 percent) are the largest in the table: most students named every type and approach afterward, and a handful still named none.

The paired t-tests use the same 18 students before and after, which removes between-student variation from the comparison. The printed p values are bounds rather than exact values; the reconstruction below shows which t statistics the printed tables allow. A post-test given at the end of the same session measures immediate recall of material just taught, which the authors acknowledge by naming retention as unmeasured.

You might expect students near graduation to know more about the skills their degree promises. They did not, by much: across the six questions, juniors and seniors averaged about five points more than freshmen and sophomores, and the authors’ test found no meaningful difference. Three or four years of engineering coursework had not taught these ideas by name.

The two groups are different sizes (16 and 39 students), so the page compares averages rather than totals. The largest gap is on Q2 (what ABET does): 11.25 percent for early students against 22.56 percent for upper-level students. On Q6 the early students are ahead.

Table V pools the first answer of every participant: the in-class pre-test and the online survey. An F-test returned p > 0.85, so the paper concludes that early and upper-level students come from the same population on this measure, which answers the second research question: years in the program did not add explicit soft-skill knowledge.

The printed p > 0.85 matches one computation exactly: a two-tailed variance-ratio F-test on the six item averages of each group (the spreadsheet function F.TEST), F(5, 5) = 0.839, p = 0.852. That test compares how much the six averages spread within each group, not how high they sit. Tests of level on the same six averages give p = 0.31 (two-sample t) and p = 0.055 (paired by item); both are the page’s own, and neither uses the 55 individual scores, which are not printed.

Rebuilding the class

For each item, the page enumerates every list of 18 scores (on the item’s step) whose class-standing averages round to Table II or III and whose mean and sample standard deviation round to Table IV. A sample standard deviation (dividing by n − 1) fits every item; a population standard deviation fits none. Students then pair within class standing in every possible order, and the page reports the smallest and largest paired t. Those bounds bracket the paper’s t without knowing which student is which.

• Before the session, the 18 in-class students averaged 2.78 to 21.11 percent on the six questions.
• After one session, they averaged 66.67 to 95.56 percent.
• Seniors scored 14 percent on what ABET is and 2 percent on each conflict question before the session.
• Early (16) and upper-level (39) students scored about the same before any instruction.
• Students told the authors the material would have helped earlier in their programs.

• Little explicit knowledge at baseline. In-class pre-test means of 16.67 (ABET), 11.11 (what ABET does), 8.89 (soft skills), 21.11 (partnerships), 5.56 (conflict types), and 2.78 percent (conflict approaches).
• Large immediate gains. Post-test means of 95.56, 80.00, 75.56, 86.67, 74.07, and 66.67 percent; paired two-tailed t-tests significant on every item (p < 10⁻⁴ to 10⁻⁷).
• No early advantage for upper-level students. Pre-test averages for 16 early and 39 upper-level students; F-test p > 0.85.
• Equal participation. The first author observed no difference between female and male participation in the session.
• Timing. Students reported that the content would have been more useful earlier in their programs.

Accreditors require every engineering and computing program to teach teamwork, communication, and ethics. This study shows that students can reach graduation without having learned the names and frameworks for those skills, and that a single, well-planned class can change that quickly. The authors argue for teaching them in the first year.

Professional skills are written into every accredited program’s outcomes, yet students in this study could not name them, and seniors did no better than freshmen. The authors draw two practical recommendations: teach the vocabulary of partnership and conflict explicitly, early (a first-year course or modules embedded in early courses), and give students real or hypothetical conflict scenarios inside their engineering projects, then assess teamwork, communication, and management directly. They also note that an emphasis on social skills may help programs attract women to engineering.

The six items make a short, reusable baseline instrument for a program’s own students, and the paper’s tables are complete enough to recover item-level score distributions, exactly for the conflict items and to a few candidates for the rest. The finding with the most leverage is the null early-versus-upper result: if three years of team projects do not produce explicit knowledge of partnership and conflict frameworks, that knowledge has to be taught on purpose. The authors propose surveying the literature’s models (dedicated courses, embedded modules, cooperative learning, and competitions) for their own university as the next step.

Notes from building this page

• Scoring steps. No item is consistent with right-or-wrong scoring. Steps of one fifth (Q1 to Q4), one third (Q5), and one quarter (Q6) are the coarsest that reproduce every printed mean, standard deviation, and class-standing average; finer multiples (tenths, sixths, eighths) also fit and give the same conflict-item scores. Stating the rubric would let readers reuse the instrument.
• One class, recovered. The search finds exactly one score list for Q5 and Q6 on both tests, one post-test list for Q1, and one pre-test list for Q3. After the session, 13 of 18 students named all three conflict types and four named none; 11 named all four approaches and five named none. All of the zeros are seniors. The conflict items’ post-test spread therefore reflects a split class rather than uniform partial learning, and a paired t-test on so bimodal a difference leans on its normality assumption; a sign test on how many students improved would make the same point without it.
• Paired t bounds. Every item has pairings that satisfy the printed bound. For Q1 to Q5 some consistent pairings do not (for example, Q1’s least favorable pairing gives p = 3.8 × 10⁻⁷ against the printed p < 10⁻⁷), so the bounds also say something about who improved.
• The F-test. The printed p > 0.85 reproduces as a variance-ratio test on six item averages per group, F(5, 5) = 0.839, p = 0.852. That test asks whether the averages spread equally across items, which differs from the research question (whether early and upper-level students know more). On the same averages, a two-sample t-test gives p = 0.31 and a test paired by item gives p = 0.055 (upper-level higher on five of six items); both are the page’s, and a test on the 55 individual scores would need the online responses.
• Two different occasions. Table V pools in-class pre-tests (on paper, in a classroom) with online answers given at the end of the year, and the early group of 16 contains one in-class student. The comparison mixes cohort, mode, and timing with class standing.
• Small details. The abstract says more than 50 students (55 in the text); the online survey went to “all undergraduate computer science students” in the methods and “all the undergraduate engineering students” in the acknowledgments; the elective is called both a computer science and a cybersecurity elective; the analysis mentions right-tailed chi-squared distributions without a test or statistic attached. The title names freshmen, while the in-class cohort was 14 seniors of 18; the paper’s case for first-year instruction rests on the seniors’ low baseline.

Try this in class

1. Give the six questions from Six questions to your class on paper, with no hints, and collect the answers anonymously.
2. Score each answer with partial credit: one fifth for each part of Q1 to Q4’s full answer you find, one third for each conflict type, and one quarter for each approach. Average each question across the class.
3. Teach the answers in one short lesson with a team game, ask the questions again the next day, and compare your class’s before-and-after averages with the bars in Before and after one class.