A Practical Guide to Your First AI Literacy Assessment

Every organisation running AI training is guessing about who needs what. An assessment replaces the guess with a map.¹1The guess is expensive — it usually means everyone sits the same course regardless of need, and the people who most need help are the least likely to say so.

The goal is not a score. It is a decision: where to spend the next training baht.

The guess is the norm, not the exception

If you are training without measuring, you are in the majority — and the size of that majority is documented.

Microsoft's 2024 Work Trend Index found that only 39% of people using AI at work had received any company training for it.¹ BCG's 2025 AI at Work study, covering more than 10,600 employees across 11 countries, found that only about a third of employees said they had been properly trained.² Salesforce's Slack Workforce Index in autumn 2024 put a number on how thin that training is: 61% of desk workers had spent under five hours in total learning to use AI, and 30% had received no training at all.^3◦SourceMicrosoft, 2024 Work Trend IndexSourceBCG, "AI at Work 2025"SourceSlack (Salesforce), Fall 2024 Workforce IndexBy the numbers61%of desk workers had spent under five hours in total learning AI — Slack Workforce Index, autumn 2024

McKinsey's 2025 Superagency in the workplace report adds the part that should worry anyone planning a curriculum: 48% of employees named formal training as the thing they most wanted, and more than one in five reported minimal to no support. The same study found leaders estimating that 4% of their employees used gen AI for at least 30% of their daily work, while employees themselves reported 13% — a threefold gap between what the top of the organisation believes and what the floor is doing.^4◦SourceMcKinsey, "Superagency in the workplace", 2025By the numbers4% vs 13%what leaders estimate versus what employees report for heavy daily gen-AI use — McKinsey, 2025

The Thai picture is consistent. AWS's 2026 Thailand study, run with Strand Partners across 2,000 respondents, found 56% of organisations naming a shortage of AI people and expertise as their main constraint, and 72% of AI-using organisations calling upskilling essential to their strategy.⁵² Almost everyone agrees training is the bottleneck. Very few can say who needs which training.SourceAWS with Strand Partners, "Unlocking Thailand's AI Potential 2026" — vendor-commissioned and self-reported (via Forbes Thailand)2Vendor-commissioned research — AWS sells the infrastructure this study encourages buying — and self-reported throughout. Useful as a directional regional picture, not an audited measurement.

If you operate in the EU, there is also a clock

The EU AI Act's Article 4 has applied since 2 February 2025, and it binds providers and deployers — not only the companies building AI, but any organisation putting an AI system to use in the EU. Supervision and enforcement by national market surveillance authorities began on 2 August 2026.⁶SourceEU AI Act Article 4, as amended by Regulation (EU) 2026/1744 (via European Commission)

Be careful how you read it, because the obligation changed recently and most of the commentary online has not caught up. The Digital Omnibus on AI — Regulation (EU) 2026/1744, in force since 27 July 2026 — rewrote Article 4 from a duty to ensure a sufficient level of AI literacy into a duty to take measures to support the development of it. The amended Article 4(1) mandates no specific level at all; the Commission's own guidance states plainly that it does not imply a certain level of AI literacy of any individual is required.^7◦SourceAI Omnibus entering into force, 27 July 2026 (via European Commission)DefinitionObligation of effort A duty to take reasonable measures towards an outcome, rather than to achieve a defined standard. Article 4 is now the former, which is why no certificate satisfies it.

So this is not a certification requirement, and nobody can sell you one that discharges it. What it is: a documented expectation that you took measures, calibrated to your people's technical knowledge, experience and training, and to the context the systems are used in. An assessment is the cheapest way to show you knew what you were calibrating to.³3And if you have no EU footprint, none of this binds you. It is still the clearest articulation available of what a regulator considers reasonable, which makes it a useful yardstick.

And if you operate in Thailand, a different law already binds you

Thailand has no AI act. What it has is the PDPA, and Chambers' 2026 Thailand guide is explicit that there is no specific or binding AI law governing the use of personal data in AI systems — so AI processing sits under the PDPA's ordinary duties: lawfulness, purpose limitation, data minimisation, transparency, security, accountability.⁸ Its compliance checklist includes training the employees who handle personal data, which is the nearest Thai equivalent to the EU's literacy duty — and the reason a Thai reader should not stop at the EU section above.SourceChambers, Data Protection & Privacy 2026 — Thailand

This has stopped being theoretical. The PDPC has moved from writing rules to enforcing them: five cases across the public and private sectors in August 2025 carried roughly THB 21.5 million in fines between them, and in November 2025 the regulator ordered a company to suspend iris scanning and delete the biometric data it had already collected.^◦ Failure to appoint a data protection officer is treated as an aggravating factor when something goes wrong.⁴By the numbersTHB 21.5min fines across five PDPC cases in August 2025, as reported in Chambers' 2026 Thailand guide4Chambers is a practitioner guide rather than the statute or a PDPC release, so attribute the dates and amounts to it. A draft Thai AI law does exist and mirrors the EU AI Act's risk tiers, but it remains a draft and binds nobody today.

An assessment does not discharge a PDPA obligation. What it does is tell you which cohorts are putting personal data into which tools — which is the first question a regulator asks.

Measure four dimensions, not one

A single "AI skill" number hides the gaps that matter.^◦ We assess four:By the numbers4dimensions we score, because one number hides the gaps that matter

• Foundations — what the tools are, what they can and cannot do.
• Applied use — can they get real work done with them today.
• Judgment and risk — do they know when not to trust an output.
• Workflow integration — is AI part of how the job is done, or a side experiment.

This is not an idiosyncratic split. The academic instruments converge on a similar shape: the 2026 LAK study by Zhang and colleagues built its measures on a Concept / Use / Evaluate / Ethics framework, and the widely used Meta-AI Literacy Scale separates understanding AI, using and applying it, detecting it, and AI ethics. The recurring insight across all of them is that knowing about AI and judging AI are different competencies, and an organisation can be strong in one and hollow in the other.^◦FigureResults map to cohorts, so training can be aimed where it actually lands.

Interestingly, AWS's own reading of its 2026 Thailand data landed on the same word we use for the third dimension: the skill it identified as decisive was judgment — using domain expertise to verify AI output and intervening when it is wrong.

Do not let people grade themselves

Here is the finding that should change how you run the assessment. Zhang and colleagues, presenting at the 2026 Learning Analytics and Knowledge conference, built both a self-reported and an objective, knowledge-based measure of AI literacy on the same four-part framework — and found that the correlation between them was consistently weak. Confirmatory factor analysis supported each measure as valid; they simply were not measuring the same thing.⁹⁵SourceZhang et al., "How to Assess AI Literacy: Misalignment Between Self-Reported and Objective-Based Measures", LAK 20265The population studied was K-12 teachers, not enterprise employees, so treat this as a measurement principle rather than a finding about companies. The principle travels: asking people to rate their own AI skill does not measure their AI skill.

Their profile analysis found six distinct groups, including a clear overestimation profile — people who rated themselves highly across every self-reported dimension and then scored lower on the objective measure. There was an underestimation group too, and a group whose self-assessment tracked reality.

This is the confidence–capability gap, and it is the reason a survey asking "how confident are you with AI?" is close to worthless as a training input.^◦ It is also the riskiest gap you can have, because confident people act on outputs without checking them. KPMG's 2025 study with the University of Melbourne, across more than 48,000 respondents in 47 countries, found 66% relying on AI output without checking its accuracy and 56% saying they had made mistakes because of AI.¹⁰DefinitionConfidence–capability gap When belief in one's AI skill outruns actual ability — the pattern most likely to produce unsafe use, and invisible to any self-rating instrument.SourceKPMG & University of Melbourne, "Trust, attitudes and use of AI", 2025

Practically: ask people to do something, not to rate themselves. Give them an output with a plausible error in it and ask what they would change. Ask them to describe a task they would not delegate to AI and why.

Report by cohort, not by name

The fastest way to kill honest answers is to make the assessment feel like a performance review. Report results by department and role, not by individual. You are diagnosing a system, not grading people.

This matters more than it sounds, because concealment is the default behaviour. The KPMG study found 57% of employees hiding their AI use or passing AI content off as their own. Slack's index found 48% of desk workers would be uncomfortable telling their manager they used AI — because it feels like cheating, or like admitting incompetence, or laziness. An assessment that reads as surveillance measures your employees' caution, not their capability.^◦FigureCohort reporting is what makes honest answers safe to give.

Say what you will do with the results before you collect them, and then do exactly that.

Turn the result into a plan

A readiness snapshot is only useful if it forces a choice. A good output looks like:

1. Which cohorts are ready for advanced, workflow-specific training.
2. Which need foundations first.
3. Where confidence outruns capability — the gap to close before you widen access to anything.

Then aim the training. Generic courses produce generic non-use; the BCG and Microsoft numbers above are what happens when training is bought rather than targeted.

What an assessment cannot tell you

Three honest limits, because a measurement sold as more than it is does real damage.

It measures capability, not behaviour. Knowing when not to trust an output is not the same as pausing to check on a Friday afternoon. Literacy is a precondition for good practice, not evidence of it. If you want to know whether the way work gets done has actually changed, that is a different measurement.^◦RelatedWhat a rollout system has to contain for capability to turn into practice

A snapshot ages fast, and not evenly. Tools change monthly and so does the frontier of what they handle well. A cohort that scored well six months ago may now be confidently using a model on tasks it has quietly got worse at. Re-assess every six to twelve months, and after any major rollout.

Objective measures are harder to run than self-report, which is exactly why most organisations skip them. Writing scenario items takes real effort, they need updating as tools change, and people resist anything that feels like a test. The honest trade-off: a self-report survey you actually run beats an objective instrument you never finish designing — as long as you never mistake the first for a measure of ability. Label it as perceived confidence and use it to find where to look, not what to conclude.

What to steal

• Assess four dimensions separately, and never average them into one number.
• Use at least one task-based item per dimension. Self-rating goes in the report as confidence, in its own column, next to capability.
• Report by cohort. Publish the reporting rule before you collect a single answer.
• Treat a high-confidence, low-capability cohort as the finding, not as noise. It is your first training priority and your largest risk.
• Re-assess on a date you set now, not when someone remembers.
• If you have EU operations, keep the assessment and what you did about it on file. The obligation is to have taken measures — so record the measures.