Problems → Prototypes · AILE
XR + AI for Safe Practice of Complex Workflows
Complex technical workflows are hard to learn from manuals and slide decks because learners must coordinate spatial, procedural, and decision knowledge. High-fidelity simulation can be expensive, while low-cost digital training often lacks realistic practice.
This exercise runs locally using simple rules. Your entries stay on this page and clear when you reload. No AI service is called.
Critical review of this working prototype
Strongest assumption: Prediction and visible state changes support sequence understanding and fault recovery better than reading a procedure.
Likely failure: Trial-and-error completion or hint following is mistaken for transfer; a previously passed test becomes stale after a change.
Test that could change the design: Compare with paper fault cards, then remove hints and change a supervised mock station’s fault. Reject the design if independent recovery does not improve.
Non-AI comparison: Paper sequence cards, diagnostic trace and trainer-led tabletop troubleshooting.
Evidence to collect: Blocked attempts, predictions, hints and evidence-led recovery are local observations; assess transfer independently and compare cost with paper practice.
Human control and access: Clearly labeled 2D precursor without XR or AI; fictional power constraints are not equipment instructions. No timing or motion requirement; native keyboard controls and textual state labels. No remote collection.
Changes made during review
- Added an actual state machine, injected signal fault, diagnostic inspection, evidence-gated repair and release gate.
- Invalidated passed verification whenever configuration, wiring or power isolation changes state.
- Added predictions, observable action history, hints, reset and export; no procedure-competence score.
- Next smallest experiment: a second fault variant with trainer-reviewed transitions before evaluating real workflow transfer.
This is a design and implementation review, not an empirical validation of learning outcomes.
Take the brief into your own AI environment
Copy a self-contained review prompt, then open the AI workspace you already use. The prompt asks for a falsification test, a non-AI alternative, evidence that distinguishes activity from learning, and human-system risks.
Preview the prompt
Act as a critical learning-engineering reviewer. Review the prototype brief below as a thin-slice experiment, not as a finished product. Return: 1. The strongest design assumption. 2. The most important plausible failure mode. 3. The highest-value falsification test for the next cycle. 4. One credible non-AI alternative that could address the same capability gap. 5. The next smallest prototype worth building. 6. What should be measured to distinguish engagement from learning, transfer, and real performance. 7. Human-agency, equity, accessibility, privacy, or governance concerns that should change the design. Be concrete. Separate evidence-backed claims from hypotheses. Do not reward novelty for its own sake, and do not assume more AI is better. COLLECTION: AILE TITLE: XR + AI for Safe Practice of Complex Workflows CAPABILITY GAP: Complex technical workflows are hard to learn from manuals and slide decks because learners must coordinate spatial, procedural, and decision knowledge. High-fidelity simulation can be expensive, while low-cost digital training often lacks realistic practice. FIRST PROTOTYPE: Prototype a lightweight 3D or XR scenario with an AI coach that observes task sequence, asks the learner to predict the next step, and provides graduated feedback. The first version should simulate only one high-value workflow rather than build a broad virtual world. REPRESENTATIVE USE CASE: A new employee practices configuring a piece of enterprise equipment in a virtual environment, including one realistic fault that requires diagnosis rather than rote sequence-following. LEARNING / HUMAN-SYSTEM FRAME: Multimedia should reduce extraneous processing and make invisible relationships visible, while simulation enables repeated practice without real-world risk. Feedback should move from guidance toward independence as competence grows. QUESTIONS ALREADY IDENTIFIED: 1. Which elements require spatial simulation and which can remain 2D? 2. Does immersive practice transfer to the real workflow? 3. Is added realism worth the cost and cognitive load? FIRST-CYCLE FRAMING: Understand: Validate that the real gap is the ability to execute and troubleshoot a complex workflow safely and independently, not simply low engagement, low tool use, or a workflow inconvenience. Map: Map the system: learner, trainer, workflow, equipment model, XR interface, AI coach, and workplace safety constraints. Identify where the capability currently succeeds, breaks down, or is masked by other constraints. Instrument: Predefine evidence: sequence accuracy, diagnosis quality, transfer to real task, error recovery, time-to-competence, and simulator cost. A key disconfirming signal is: the simulation feels realistic but does not improve real-world performance beyond simpler practice.
Open the underlying design brief and eight-step path
A first prototype
Prototype a lightweight 3D or XR scenario with an AI coach that observes task sequence, asks the learner to predict the next step, and provides graduated feedback. The first version should simulate only one high-value workflow rather than build a broad virtual world.
Representative use case
A new employee practices configuring a piece of enterprise equipment in a virtual environment, including one realistic fault that requires diagnosis rather than rote sequence-following.
Questions worth carrying forward
- Which elements require spatial simulation and which can remain 2D?
- Does immersive practice transfer to the real workflow?
- Is added realism worth the cost and cognitive load?
Evidence anchors
These sources motivate the design; they do not validate this prototype.
- Mayer, R. E., & Fiorella, L. Introduction to Multimedia Learning, in The Cambridge Handbook of Multimedia Learning, 3rd ed. (2021)
Multiple representations, multimedia, and cognitive-load-aware design.
- McGaghie, W. C., et al. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? Academic Medicine. (2011)
Simulation, deliberate practice, and repeated performance with feedback.
- Hattie, J., & Timperley, H. The Power of Feedback. Review of Educational Research. (2007)
Feedback design and evidence-producing practice.
- CAST. Universal Design for Learning Guidelines, Version 3.0. (2024)
Accessibility, learner agency, multiple representations, and barrier-aware design.
One possible first pass
- 01 Understand
Validate that the real gap is the ability to execute and troubleshoot a complex workflow safely and independently, not simply low engagement, low tool use, or a workflow inconvenience.
- 02 Map
Map the system: learner, trainer, workflow, equipment model, XR interface, AI coach, and workplace safety constraints. Identify where the capability currently succeeds, breaks down, or is masked by other constraints.
- 03 Design
Compare an AI intervention with simpler non-AI options. Specify what the human decides, what the AI may suggest, and which tradeoffs are acceptable.
- 04 Build
Create the smallest usable prototype around one representative task, with expert-curated content, visible uncertainty, and an easy human override.
- 05 Instrument
Predefine evidence: sequence accuracy, diagnosis quality, transfer to real task, error recovery, time-to-competence, and simulator cost. A key disconfirming signal is: the simulation feels realistic but does not improve real-world performance beyond simpler practice.
- 06 Deploy
Pilot with a small, representative group in a near-real setting; preserve a baseline or comparison condition and document implementation conditions.
- 07 Evaluate
Look for capability growth and transfer, not just satisfaction or activity. Inspect subgroup patterns, human workload, errors, and unintended adaptations.
- 08 Refine
Let the evidence change the problem model. Default back to UNDERSTAND if the assumed gap was wrong; otherwise revisit the earliest step invalidated by the evidence.