Problems → Prototypes · LENS
A Training Harmonization Decision Assistant
Large health and service organizations want to reduce duplicated training, but units may differ in policy, workflow, equipment, and risk. Standardizing too aggressively can make training inaccurate; localizing everything can create high maintenance cost and inconsistent expectations.
This exercise runs locally using simple rules. Your entries stay on this page and clear when you reload. No AI service is called.
Compare manually entered rules and preview a source-linked practice branch. This local checklist does not read documents, infer meaning, verify policy, or generate AI scenarios. The starting rules are fictional examples, not clinical or operational guidance.
Use document titles, sections, and versions without sensitive content. Optional source links open only when you choose; links may contact external sites. JSON exports contain your entries.
Critical review of this working prototype
Strongest assumption: Educators can accurately declare applicability and decision keys so a source-linked comparison exposes consequential variation before consolidation.
Likely failure: Matching wording is mistaken for equivalent policy, edits inherit stale owner review, or a local exception silently overrides a common instruction; inaccurate keys can hide a real conflict.
Test that could change the design: Have source owners review blinded same-word/different-authority rules and overlapping local exceptions. Reject the design if educators consolidate them without checking versions and authority, or if local scenario errors rise despite faster updates.
Non-AI comparison: A source-owner maintained policy comparison matrix with a common module and manually reviewed local branches.
Evidence to collect: Measure owner agreement, incorrect consolidation decisions, update time, performance on unit-switch scenarios, and local exception errors. Reduced course count alone is insufficient. No telemetry is collected.
Human control and access: The tool cannot fetch or authenticate documents, infer semantics, verify approvals, resolve authority, or detect conflicts across different decision keys. Exact unit/role labels can omit relevant rules. User-controlled source links leave the page only on click. Avoid sensitive content in entries and exports.
Changes made during review
- Cross-review found edits retained stale review=reviewed. Delegated input/change handlers now reset only the affected row to pending when action, source, version, owner, scope, decision, unit, role, or URL changes; changing review itself preserves the user's selection. Delegation also covers newly added and reset rows.
- Kept every rule separately linked to its document/section, declared scope, owner, version, and optional HTTP(S) link; matching wording is explicitly unconfirmed equivalence.
- Compared overlapping system, role, and unit scopes for same-key instruction differences; unresolved conflicts hold affected target branches without selecting a winner.
- Required a target task/unit/role and source references, retained unit/role branches, and held unreviewed or owner/version-missing rules.
- Rejected executable URL schemes and embedded URL credentials; used textContent for all user text and cleared results/export on additions, removals, or edits.
- Added a full disabled-until-initialized form guard and noscript explanation after parent identified GET fallback leakage. Next smallest prototype: source-owner reconciliation of one conflicting pair followed by a cross-unit scenario test.
This is a design and implementation review, not an empirical validation of learning outcomes.
Take the brief into your own AI environment
Copy a self-contained review prompt, then open the AI workspace you already use. The prompt asks for a falsification test, a non-AI alternative, evidence that distinguishes activity from learning, and human-system risks.
Preview the prompt
Act as a critical learning-engineering reviewer. Review the prototype brief below as a thin-slice experiment, not as a finished product. Return: 1. The strongest design assumption. 2. The most important plausible failure mode. 3. The highest-value falsification test for the next cycle. 4. One credible non-AI alternative that could address the same capability gap. 5. The next smallest prototype worth building. 6. What should be measured to distinguish engagement from learning, transfer, and real performance. 7. Human-agency, equity, accessibility, privacy, or governance concerns that should change the design. Be concrete. Separate evidence-backed claims from hypotheses. Do not reward novelty for its own sake, and do not assume more AI is better. COLLECTION: LENS TITLE: A Training Harmonization Decision Assistant CAPABILITY GAP: Large health and service organizations want to reduce duplicated training, but units may differ in policy, workflow, equipment, and risk. Standardizing too aggressively can make training inaccurate; localizing everything can create high maintenance cost and inconsistent expectations. FIRST PROTOTYPE: Create an AI-assisted harmonization tool that classifies training content into system-wide, role-specific, and local-policy components. It generates shared scenarios with controlled local variants and shows exactly which source rule created each variation. REPRESENTATIVE USE CASE: Several outpatient units share a common patient-intake capability but differ in escalation policy. The tool produces one core module plus unit-specific decision branches rather than separate courses. LEARNING / HUMAN-SYSTEM FRAME: The design preserves common capability targets while representing contextual variation explicitly. Scenario practice and source-grounded feedback support transfer better than simply distributing harmonized information. QUESTIONS ALREADY IDENTIFIED: 1. What can safely be standardized? 2. Which local differences materially change performance? 3. Does the model reduce maintenance effort while improving accuracy and transfer? FIRST-CYCLE FRAMING: Understand: Validate that the real gap is the ability to perform a common professional task correctly across local policy variations, not simply low engagement, low tool use, or a workflow inconvenience. Map: Map the system: learner, educators, multiple units, policies, workflow differences, AI harmonization tool, and governance owners. Identify where the capability currently succeeds, breaks down, or is masked by other constraints. Instrument: Predefine evidence: content duplication, update time, policy accuracy, scenario performance, transfer, and local exception rate. A key disconfirming signal is: maintenance decreases but learners are taught a generic workflow that is wrong in important local contexts.
Open the underlying design brief and eight-step path
A first prototype
Create an AI-assisted harmonization tool that classifies training content into system-wide, role-specific, and local-policy components. It generates shared scenarios with controlled local variants and shows exactly which source rule created each variation.
Representative use case
Several outpatient units share a common patient-intake capability but differ in escalation policy. The tool produces one core module plus unit-specific decision branches rather than separate courses.
Questions worth carrying forward
- What can safely be standardized?
- Which local differences materially change performance?
- Does the model reduce maintenance effort while improving accuracy and transfer?
Evidence anchors
These sources motivate the design; they do not validate this prototype.
- Barnett, S. M., & Ceci, S. J. When and where do we apply what we learn? A taxonomy for far transfer. Psychological Bulletin. (2002)
Transfer as a design and evaluation target.
- McGaghie, W. C., et al. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? Academic Medicine. (2011)
Simulation, deliberate practice, and repeated performance with feedback.
- Hattie, J., & Timperley, H. The Power of Feedback. Review of Educational Research. (2007)
Feedback design and evidence-producing practice.
- Amershi, S., et al. Guidelines for Human-AI Interaction. CHI 2019. (2019)
Human-AI interaction design and calibrated reliance.
- National Academies of Sciences, Engineering, and Medicine. How People Learn II: Learners, Contexts, and Cultures. (2018)
Broad learning-science anchor for learner/context/system modeling and transfer.
One possible first pass
- 01 Understand
Validate that the real gap is the ability to perform a common professional task correctly across local policy variations, not simply low engagement, low tool use, or a workflow inconvenience.
- 02 Map
Map the system: learner, educators, multiple units, policies, workflow differences, AI harmonization tool, and governance owners. Identify where the capability currently succeeds, breaks down, or is masked by other constraints.
- 03 Design
Compare an AI intervention with simpler non-AI options. Specify what the human decides, what the AI may suggest, and which tradeoffs are acceptable.
- 04 Build
Create the smallest usable prototype around one representative task, with expert-curated content, visible uncertainty, and an easy human override.
- 05 Instrument
Predefine evidence: content duplication, update time, policy accuracy, scenario performance, transfer, and local exception rate. A key disconfirming signal is: maintenance decreases but learners are taught a generic workflow that is wrong in important local contexts.
- 06 Deploy
Pilot with a small, representative group in a near-real setting; preserve a baseline or comparison condition and document implementation conditions.
- 07 Evaluate
Look for capability growth and transfer, not just satisfaction or activity. Inspect subgroup patterns, human workload, errors, and unintended adaptations.
- 08 Refine
Let the evidence change the problem model. Default back to UNDERSTAND if the assumed gap was wrong; otherwise revisit the earliest step invalidated by the evidence.