Problems → Prototypes · LENS
Linking Frontline Training to Operational Performance
Organizations often measure training completion separately from the operational outcomes the training is supposed to improve. This makes it hard to know whether a performance problem is caused by knowledge, workflow design, equipment, incentives, staffing, or something else.
This exercise runs locally using simple rules. Your entries stay on this page and clear when you reload. No AI service is called.
This local decision checklist separates observations, hypotheses, and operating conditions. It cannot diagnose causes or workers. Use an anonymized task example; do not enter personnel, patient, or incident identifiers.
Critical review of this working prototype
Strongest assumption: Supervisors can supply sufficiently specific task observations to separate a testable learning hypothesis from reported operating constraints.
Likely failure: Low throughput or completion data becomes a worker-deficit label; unknown equipment conditions are silently treated as normal; mixed causes collapse into a training recommendation.
Test that could change the design: Give reviewers blinded cases containing equipment failures, staffing constraints, and genuine practice gaps. Compare false training recommendations and worker disagreement against a facilitated walkthrough without the tool.
Non-AI comparison: A worker-led task walkthrough with a supervisor and process owner using an observation checklist.
Evidence to collect: Compare errors per comparable task opportunity, rework, operating conditions, worker burden, delayed independent performance, and false training recommendations. Completion alone is not an outcome; the tool collects no telemetry.
Human control and access: Reported direct observation remains unverified. Supervisors can mischaracterize conditions or accommodations. Workers need a way to dispute interpretations; no ranking or employment decisions are justified. Enter anonymized descriptions only; exports contain those descriptions.
Changes made during review
- Kept unknown observations, reported system constraints, and training hypotheses in separate outputs.
- Withheld training hypotheses from proxy-only data, missing task observations, and undocumented supported-condition claims.
- Treated unchecked conditions and unsupported absent/present selections as unknown, preserving mixed explanations and prioritizing process-owner investigation.
- Added worker perspective, consent-based observation, accommodations, and an explicit prohibition on recreating unsafe conditions. Next smallest prototype: paired worker/supervisor review of one anonymized task observation.
- Applied parent review's privacy correction: disable the entire form fieldset until submit interception is registered, including a noscript explanation.
This is a design and implementation review, not an empirical validation of learning outcomes.
Take the brief into your own AI environment
Copy a self-contained review prompt, then open the AI workspace you already use. The prompt asks for a falsification test, a non-AI alternative, evidence that distinguishes activity from learning, and human-system risks.
Preview the prompt
Act as a critical learning-engineering reviewer. Review the prototype brief below as a thin-slice experiment, not as a finished product. Return: 1. The strongest design assumption. 2. The most important plausible failure mode. 3. The highest-value falsification test for the next cycle. 4. One credible non-AI alternative that could address the same capability gap. 5. The next smallest prototype worth building. 6. What should be measured to distinguish engagement from learning, transfer, and real performance. 7. Human-agency, equity, accessibility, privacy, or governance concerns that should change the design. Be concrete. Separate evidence-backed claims from hypotheses. Do not reward novelty for its own sake, and do not assume more AI is better. COLLECTION: LENS TITLE: Linking Frontline Training to Operational Performance CAPABILITY GAP: Organizations often measure training completion separately from the operational outcomes the training is supposed to improve. This makes it hard to know whether a performance problem is caused by knowledge, workflow design, equipment, incentives, staffing, or something else. FIRST PROTOTYPE: Create an AI-assisted performance diagnostic that combines a capability model with operational error patterns and supervisor observations. Before recommending training, it asks whether the gap is actually learnable and identifies alternative system levers. REPRESENTATIVE USE CASE: A manufacturing line shows repeated setup errors. The tool compares errors with task steps and finds that one issue is knowledge-based, another is caused by an ambiguous interface, and only the first should trigger practice. LEARNING / HUMAN-SYSTEM FRAME: The human–learner-system frame resists treating every performance gap as a training problem. When learning is appropriate, practice and feedback are tied directly to the target work; when it is not, the system recommends redesign or escalation. QUESTIONS ALREADY IDENTIFIED: 1. Can the diagnostic distinguish learning gaps from system failures with acceptable reliability? 2. Which operational measures are fair evidence of capability? 3. Does the prototype reduce unnecessary training and improve actual performance? FIRST-CYCLE FRAMING: Understand: Validate that the real gap is the ability to perform a frontline task reliably and diagnose whether failures are learnable or systemic, not simply low engagement, low tool use, or a workflow inconvenience. Map: Map the system: worker, supervisor, workflow, equipment/interface, operating conditions, performance data, and AI diagnostic. Identify where the capability currently succeeds, breaks down, or is masked by other constraints. Instrument: Predefine evidence: error rate, rework, training avoided, transfer, supervisor agreement, and false training recommendations. A key disconfirming signal is: the tool labels system problems as learner deficits or optimizes a metric workers cannot fully control.
Open the underlying design brief and eight-step path
A first prototype
Create an AI-assisted performance diagnostic that combines a capability model with operational error patterns and supervisor observations. Before recommending training, it asks whether the gap is actually learnable and identifies alternative system levers.
Representative use case
A manufacturing line shows repeated setup errors. The tool compares errors with task steps and finds that one issue is knowledge-based, another is caused by an ambiguous interface, and only the first should trigger practice.
Questions worth carrying forward
- Can the diagnostic distinguish learning gaps from system failures with acceptable reliability?
- Which operational measures are fair evidence of capability?
- Does the prototype reduce unnecessary training and improve actual performance?
Evidence anchors
These sources motivate the design; they do not validate this prototype.
- National Academies of Sciences, Engineering, and Medicine. How People Learn II: Learners, Contexts, and Cultures. (2018)
Broad learning-science anchor for learner/context/system modeling and transfer.
- Barnett, S. M., & Ceci, S. J. When and where do we apply what we learn? A taxonomy for far transfer. Psychological Bulletin. (2002)
Transfer as a design and evaluation target.
- Hattie, J., & Timperley, H. The Power of Feedback. Review of Educational Research. (2007)
Feedback design and evidence-producing practice.
- Amershi, S., et al. Guidelines for Human-AI Interaction. CHI 2019. (2019)
Human-AI interaction design and calibrated reliance.
One possible first pass
- 01 Understand
Validate that the real gap is the ability to perform a frontline task reliably and diagnose whether failures are learnable or systemic, not simply low engagement, low tool use, or a workflow inconvenience.
- 02 Map
Map the system: worker, supervisor, workflow, equipment/interface, operating conditions, performance data, and AI diagnostic. Identify where the capability currently succeeds, breaks down, or is masked by other constraints.
- 03 Design
Compare an AI intervention with simpler non-AI options. Specify what the human decides, what the AI may suggest, and which tradeoffs are acceptable.
- 04 Build
Create the smallest usable prototype around one representative task, with expert-curated content, visible uncertainty, and an easy human override.
- 05 Instrument
Predefine evidence: error rate, rework, training avoided, transfer, supervisor agreement, and false training recommendations. A key disconfirming signal is: the tool labels system problems as learner deficits or optimizes a metric workers cannot fully control.
- 06 Deploy
Pilot with a small, representative group in a near-real setting; preserve a baseline or comparison condition and document implementation conditions.
- 07 Evaluate
Look for capability growth and transfer, not just satisfaction or activity. Inspect subgroup patterns, human workload, errors, and unintended adaptations.
- 08 Refine
Let the evidence change the problem model. Default back to UNDERSTAND if the assumed gap was wrong; otherwise revisit the earliest step invalidated by the evidence.