Beyond the Sign-In Sheet: Why Most Training ROI Programs Measure the Wrong Thing

Every training budget review starts with the same question. What did we get for the money?

Most learning and development teams answer it with the wrong data. They report how just many adjusters attended, how they rated the session/training provided, and a completion percentage pulled from the learning management system. None of that tells the CFO whether the training changed anything that eventually shows up on a loss run.

This is not a criticism of the people running the training programs. It is a description of how corporate training measures have worked for the better part of seven decades, and why that habit is now colliding with tighter budgets.

The numbers on measurement are not encouraging. The Association for Talent Development’s 2026 State of the Industry report, based on 2025 data from 340 organizations, found that less than a quarter of respondents actually measure whether their training programs achieved the organizational goals for which they were built. A smaller share still measures return on investment.

Meanwhile, average direct learning spends per employee fell from roughly $1,254 in 2024 to $846 in 2025, a drop of about a third, even as the average employee logged more training hours than the year before, 16.7 hours versus 13.7 (Association for Talent Development, 2026 State of the Industry Report). Organizations are spending less and asking employees to sit through more of it, without much evidence of what any of it produces.

For a workers’ compensation claims organization, that gap is not just an L&D problem. It is a claims cost problem. If a training investment cannot be tied to claim duration, litigation rate, or reserve accuracy, the finance committee has no defensible reason to protect it the next time the budget gets cut.

Symptoms Versus Findings

The perception that training is overhead is a symptom. It shows up as skepticism at budget time and as the first line item cut when premium volume softens. The objective finding, when anyone bothers to look for it, is usually different. Training tied to measurable claim behavior tends to survive budget reviews. Training measured only by attendance does not, because nobody in the room can defend it with a number.

A Framework, Not a Slogan

Donald Kirkpatrick published his four-level model for training evaluation in 1959, and it remains the standard reference point in the field (Kirkpatrick Partners, “The Kirkpatrick Model”). It is worth restating, because most organizations use only the first two levels without realizing the third and fourth exist.

Level one is reaction. Did the adjusters find the session useful and well delivered? This is what a post-training survey captures, and it is the easiest data to collect, which is exactly why most organizations stop here.

Level two is learning. Did the adjuster actually absorb the material, measured with a pre- and post-test or a scored case exercise, not a show of hands?

Level three is behavior. Is the adjuster applying what was taught on an actual file three or six months later? This requires a supervisor, or a file audit, to observe and document a change in practice.

Here at WorkCompCollege.com, we have direct antidotal data demonstrating that the certifications obtained went directly to a change in professional behavior and resulted in significant improvement in claims outcomes.

Level four is results. Did the organization’s numbers move? Claim duration, litigation rate, reopen rate, medical cost trend, reserve accuracy.

Jack Phillips later added a fifth level, converting the level four results into a financial return: program benefit minus program cost, divided by program cost (Training Industry, “Phillips ROI Methodology”). The useful part of the Phillips method is not the formula. It is the requirement to isolate training’s effect from everything else moving at the same time, such as a shift in claim mix, adjuster tenure, or a state fee schedule change. Phillips’s answer is a control group where practical, or a documented trend comparison where it is not. Either way, the estimate has to be conservative and written down, not asserted.

What a Claims Scorecard Looks Like

Applying this to adjuster education does not require new software. It requires picking numbers the organization already tracks and connecting them to the training calendar before the training happens, not after someone asks for a budget justification.

  1. Claim duration by adjuster cohort, trained versus untrained, or before versus after a specific course.
  2. Litigation rate on claims handled by adjusters who completed a defined training module within the prior twelve months.
  3. Reopen rate, which often reflects premature closure driven by thin documentation rather than genuine medical resolution.
  4. Reserve accuracy, comparing initial reserve to final paid on a rolling basis.
  5. Medical cost trends on comparable injury types, segmented by whether the handling adjuster completed relevant clinical education.

None of these requires a new system. Most claims organizations already have this data in their reporting stack. What is usually missing is the discipline to tag which adjusters received which training and when, so the numbers can be split and compared later.

The Measurement Gap Is the Opportunity

Because so few organizations measure past level one, a claims leader who builds even a basic level three or four scorecard has an advantage that has nothing to do with clinical sophistication.

It is better to do bookkeeping. When the budget conversation comes, that leader has a defensible answer instead of an attendance sheet.

This is not an argument for measuring everything. Specific training, particularly compliance refreshers, will always live at level one or two, and that is fine. The scorecard belongs to the training that is supposed to change claim outcomes, which describes most clinical and causation education.

WorkCompCollege.com builds its curriculum around the claims outcomes adjusters and supervisors are actually accountable for. If your organization wants help mapping a training calendar to a measurement framework like the one above, that conversation is worth having before the next budget cycle, not during it.

Sources: Association for Talent Development, 2026 State of the Industry Report (td.org). Kirkpatrick Partners, “The Kirkpatrick Model” (kirkpatrickpartners.com). Training Industry, “Phillips ROI Methodology” (trainingindustry.com).