How to Measure If Security Awareness Training Is Working

You measure whether security awareness training is working by measuring behavior, not the training. Track how many employees report suspicious email, how fast the first report arrives, how much real phishing your people flag, and whether they admit their own mistakes. Completion rates and raw click rates can't tell you if anyone acts differently.

Most programs can't answer the question

Ask a security leader if their training works and you'll usually get a completion rate. A 2023 study of US government programs by NIST researchers (Jacobs et al.) found exactly that: programs lean on compliance metrics like completion. Completion proves people opened the course. It says nothing about Tuesday at 4:55 p.m. when a fake invoice lands.

The harder problem is that the best recent research says standard training may not move clicks at all.

Rozema and Davis (WWW '26) followed 12,511 employees and found no significant training effect on clicks or reports. Their paper title says it plainly: anti-phishing training "still does not work." Ho et al. (2025) found that finishing annual training recently had no significant link to failing a phishing simulation. Most people spent minimal time on the training pages.

Compare that with vendor benchmarks. KnowBe4's 2025 Phishing by Industry Benchmark reports click rates falling from 33.1% to 4.1% after 12 months. Both can be true, and that is the point. A falling click rate can mean people learned. It can also mean the emails got easier, or people learned to spot one vendor's style. If you only track clicks, you can't tell which.

So stop asking "did the training work?" Ask "are people doing the thing we need them to do?" That question you can measure.

Click rate is only useful with a difficulty rating

Rozema and Davis rated every simulation with the NIST Phish Scale. Difficulty predicted clicks: about 7% on easy emails versus 15% on hard ones. Same people, double the clicks, purely from the email.

If you keep click rate, and you probably should, record the difficulty of every phishing simulation next to it. A 6% click rate on a hard email beats a 3% rate on an obvious one. Without the rating, a quarter-to-quarter drop is a story you're telling yourself.

Also watch what happens after the click. Lain et al. (IEEE S&P 2022), studying 14,000+ employees for 15 months, found embedded training shown after a failed simulation did not improve resilience and could increase susceptibility. Treat clicks as learning moments, not grades.

What to measure instead

Lain's team found the thing that did work: employee reporting acted as a fast, sustainable, crowd-sourced detection system. That's where your metrics should live.

SERR

Suspicious Email Reporting Rate (SERR) is the share of recipients who report a phishing simulation. Fortra's 2025 benchmark put average reporting at 10.5%, against a 5.42% click rate. Most employees still do neither. That silent middle is your real gap.

Time to first report

How long from delivery until the first person reports? This is the number your SOC or MSP actually cares about, because one early report can pull the email from every inbox. Rozema and Davis found a report arrived before the first click in 36-55% of simulations. That's the race you're trying to win, and we've written more on why reporting speed beats click rate.

Share of real phishing reported by employees

Simulations are practice. The real score is how many actual malicious emails your people reported, especially ones that got past your email filter. If employees flag real phishing before your tools do, the program is paying for itself.

Resilience by manipulation tactic

A single click rate hides patterns. Break results out by the tactic each email used. We track six: urgency, scarcity, trust, helpfulness, authority and social proof. You may find your team is solid against fake shipping notices and wide open to a message that sounds like the CEO. Tolsdorf et al. (2025) found susceptibility varied sharply by staff group in a hospital, so slice by team too.

Mistakes self-reported

This one gets skipped because it's awkward to count. Do people tell IT when they clicked, entered a password, or sent a file to the wrong person? A rising count is good news. It means people trust that speaking up won't get them punished. Amy Edmondson's work on psychological safety showed that safe teams admit errors and learn from them. If your program shames people, this number goes to zero and you lose your earliest warning.

A simple quarterly scorecard

Five numbers, reviewed once a quarter. We're not giving you goal numbers, because nobody has good public baselines for most of these. Direction is what matters.

• SERR: going up.
• Median time to first report: going down.
• Real phishing reported by employees: count going up, especially emails your filter missed.
• Click rate by difficulty band: flat or down within each band, not just overall.
• Self-reported mistakes: going up, then holding steady.

Keep completion in a footnote for your auditor. If you report to a board or a client, this 15-minute QBR template shows how to present these numbers without drowning anyone.

One honest caution. Gains fade. Reinheimer et al. (SOUPS 2020) found improvement was gone by six months. Lain et al. (2024) found the benefit of training came from the periodic reminder, not the content. If your scorecard slides two quarters after a big push, that's expected. It's telling you to send small, regular nudges.

Where Hook fits

Hook Security is a security awareness training and phishing simulation platform built for MSPs and SMBs. We built it around SERR as the primary metric, and Autopilot runs training and phishing simulations automatically each month so the nudges keep coming. It's the practical side of our Psychological Security (PsySec) approach: treat employees as partners in defense, then measure whether they act like it.

FAQ

What is the best metric for security awareness training?

Suspicious Email Reporting Rate (SERR) is the strongest single metric, because reporting is the behavior that stops real attacks. Pair it with time to first report.

Is phishing simulation click rate a good measure of training success?

Only if you rate how hard each email was. Research using the NIST Phish Scale found difficulty alone roughly doubled click rates, so an unrated click rate can rise or fall without any change in behavior.

Does training completion prove compliance or effectiveness?

Compliance only. Completion shows someone finished a course. Ho et al. (2025) found recent training completion had no significant link to how people handled phishing simulations.

How often should we measure security awareness results?

Track the numbers monthly and review them quarterly. Research shows gains fade within six months, so a quarterly view shows you when it's time for another nudge.