Problems → Prototypes · LENS
A Capability Map for Workforce Learning Systems
Large organizations often accumulate courses, credentials, and learning platforms without a shared model of the capabilities employees need to perform. Completion data then becomes easier to measure than readiness for real work.
This exercise runs locally using simple rules. Your entries stay on this page and clear when you reload. No AI service is called.
Evidence coverage
Select the evidence you actually have for this person. Coverage shows which tasks have been observed; it does not establish that performance met the standard.
Critical review of this working prototype
Strongest assumption: Mapping roles to observed tasks can replace indiscriminate course assignment with targeted evidence collection.
Likely failure: Checking that evidence exists can be mistaken for proof that performance met a standard.
Test that could change the design: Compare recommendations made from this map with recommendations from a course-completion list, then inspect actual task performance and unnecessary training.
Non-AI comparison: A manager reviews a role/task/evidence worksheet with a colleague.
Evidence to collect: Track task coverage, quality of the standard, observed performance, transfer, and training recommendations avoided.
Human control and access: Evidence absence is uncertainty, not a worker deficit. Keep observation coverage separate from judgments of readiness.
Changes made during review
- Rename the diagnostic to evidence coverage and clarify what checkmarks establish.
- Preserve separate selections when switching roles.
- Require review against standards even when all representative tasks have evidence.
This is a design and implementation review, not an empirical validation of learning outcomes.
Take the brief into your own AI environment
Copy a self-contained review prompt, then open the AI workspace you already use. The prompt asks for a falsification test, a non-AI alternative, evidence that distinguishes activity from learning, and human-system risks.
Preview the prompt
Act as a critical learning-engineering reviewer. Review the prototype brief below as a thin-slice experiment, not as a finished product. Return: 1. The strongest design assumption. 2. The most important plausible failure mode. 3. The highest-value falsification test for the next cycle. 4. One credible non-AI alternative that could address the same capability gap. 5. The next smallest prototype worth building. 6. What should be measured to distinguish engagement from learning, transfer, and real performance. 7. Human-agency, equity, accessibility, privacy, or governance concerns that should change the design. Be concrete. Separate evidence-backed claims from hypotheses. Do not reward novelty for its own sake, and do not assume more AI is better. COLLECTION: LENS TITLE: A Capability Map for Workforce Learning Systems CAPABILITY GAP: Large organizations often accumulate courses, credentials, and learning platforms without a shared model of the capabilities employees need to perform. Completion data then becomes easier to measure than readiness for real work. FIRST PROTOTYPE: Create a capability graph that links role outcomes to observable tasks, prerequisite knowledge, practice opportunities, and evidence. An AI coach can recommend practice from the graph and explain why, while managers and learning teams control the capability model. REPRESENTATIVE USE CASE: A new team lead is not assigned 'leadership content' broadly; the system identifies a need to run a difficult feedback conversation and recommends a short scenario, reflection, and live manager practice. LEARNING / HUMAN-SYSTEM FRAME: The learning architecture begins with performance and transfer rather than content inventory. AI supports navigation through the model, but human experts define what competent performance looks like and what evidence is credible. QUESTIONS ALREADY IDENTIFIED: 1. Can the organization agree on observable capabilities across roles? 2. Which evidence predicts performance on the job? 3. Does recommendation improve readiness or simply increase learning activity? FIRST-CYCLE FRAMING: Understand: Validate that the real gap is the ability to perform a defined workplace capability in context, not merely complete related training, not simply low engagement, low tool use, or a workflow inconvenience. Map: Map the system: employee, manager, learning team, role tasks, performance data, capability graph, and AI coach. Identify where the capability currently succeeds, breaks down, or is masked by other constraints. Instrument: Predefine evidence: task performance, transfer, time-to-readiness, manager agreement, recommendation uptake, and course reduction. A key disconfirming signal is: the graph becomes another taxonomy disconnected from actual work performance.
Open the underlying design brief and eight-step path
A first prototype
Create a capability graph that links role outcomes to observable tasks, prerequisite knowledge, practice opportunities, and evidence. An AI coach can recommend practice from the graph and explain why, while managers and learning teams control the capability model.
Representative use case
A new team lead is not assigned 'leadership content' broadly; the system identifies a need to run a difficult feedback conversation and recommends a short scenario, reflection, and live manager practice.
Questions worth carrying forward
- Can the organization agree on observable capabilities across roles?
- Which evidence predicts performance on the job?
- Does recommendation improve readiness or simply increase learning activity?
Evidence anchors
These sources motivate the design; they do not validate this prototype.
- Barnett, S. M., & Ceci, S. J. When and where do we apply what we learn? A taxonomy for far transfer. Psychological Bulletin. (2002)
Transfer as a design and evaluation target.
- Hattie, J., & Timperley, H. The Power of Feedback. Review of Educational Research. (2007)
Feedback design and evidence-producing practice.
- National Academies of Sciences, Engineering, and Medicine. How People Learn II: Learners, Contexts, and Cultures. (2018)
Broad learning-science anchor for learner/context/system modeling and transfer.
- Amershi, S., et al. Guidelines for Human-AI Interaction. CHI 2019. (2019)
Human-AI interaction design and calibrated reliance.
One possible first pass
- 01 Understand
Validate that the real gap is the ability to perform a defined workplace capability in context, not merely complete related training, not simply low engagement, low tool use, or a workflow inconvenience.
- 02 Map
Map the system: employee, manager, learning team, role tasks, performance data, capability graph, and AI coach. Identify where the capability currently succeeds, breaks down, or is masked by other constraints.
- 03 Design
Compare an AI intervention with simpler non-AI options. Specify what the human decides, what the AI may suggest, and which tradeoffs are acceptable.
- 04 Build
Create the smallest usable prototype around one representative task, with expert-curated content, visible uncertainty, and an easy human override.
- 05 Instrument
Predefine evidence: task performance, transfer, time-to-readiness, manager agreement, recommendation uptake, and course reduction. A key disconfirming signal is: the graph becomes another taxonomy disconnected from actual work performance.
- 06 Deploy
Pilot with a small, representative group in a near-real setting; preserve a baseline or comparison condition and document implementation conditions.
- 07 Evaluate
Look for capability growth and transfer, not just satisfaction or activity. Inspect subgroup patterns, human workload, errors, and unintended adaptations.
- 08 Refine
Let the evidence change the problem model. Default back to UNDERSTAND if the assumed gap was wrong; otherwise revisit the earliest step invalidated by the evidence.