Application class: Providing people a model of the real world
Virtual 3D models can be used to train people in the operation of complex systems in high risk applications such as medical or emergency services. VR can also be used as a low-cost and accessible supplement to the practical portions of many other education and training courses by simulating the lab environment and allowing students to experiment without needing to access expensive equipment.
Existing alternatives
Obvious alternatives that should form a baseline of comparison for use cases with this application class are
- field training, such as practising surgery on real people and conducting emergency service operations with experts supervising, or shadowing experts while they are on duty.
- training in a testing facility using real infrastructure, props, and body doubles to simulate a workplace scenario
Potential benefits
- Low-risk alternative to field training
- Medical: For medical training, VR is a low-risk alternative to field training, such as asking trainees to conduct real surgical operations. This can also create more opportunities for hands-on training, so trainees can conduct no-consequence virtual surgical procedures that can be easily reset after mistakes.
- Emergency services: For emergency service training or any other high-risk training, VR simulations offer a safe, repeatable, updatable and likely cheaper alternative to a fully-equipped testing facility. Multiple participants in the same virtual environment could cooperate to rehearse complex tasks. .
- Educational: VR experiences that simulate car engines are an ideal interactive tool that enables students to understand the inner workings without having to leave their seat.
Potential harms
- Over-reliance decreases sensitivity: over-reliance on training simulations — especially incomplete or inaccurate ones — may increase the likelihood of mistakes in the field.
- Additionally, a trainee who has exclusively trained in simulations with no field experience may be conditioned to undervalue the severity of their mistakes or actions in the real world.
- VR-based training simulations should therefore be used as a supplement to all other forms of training, not in lieu of it.
Example applications
Application: Police and firefighter training in building entry and navigation
Description: VR-based training programs for the emergency services that simulate real-life scenarios.
Role of government: Application provider
Rationale: The emergency services deal with high-risk, time-sensitive operations on a daily basis. Effective training for all staff is paramount, however it can be difficult to provide frequent and effective training for all necessary staff. This could be due to the resource and setup costs required for equipping the training facilities, the need for all staff to be on site for the training, as well as the potential dangers in these scenarios e.g. firefighting and simulated hostage situations.
Assumptions: These VR simulations are reflective of a realistic workplace scenario, with relevant interactive capability.
Specific considerations: VR simulations for emergency services may be more effective in large, open spaces, especially if all trainees are in the same physical area. As mentioned, these VR training experiences should be used as a supplement to real-life training, never as a substitute.
Existing implementations:
- Leicestershire Fire & Rescue Service (UK) have produced VR films for trainees to experience a real-life firefighting scenario [137]
- Zurich City Police have been undertaking tactical training using VR [138]
- New York Police Department (NYPD) are using VR to train unit response to active shooting scenarios [139]
- The Department of Homeland Security created a system called Enhanced Dynamic Geo-social Environment (EDGE) for first responders to practise coordinated responses and fine-tune strategies. [140]
Application: VR medical training in hospitals
Description: VR-based surgery training to upskill medical professionals
Role of government: Application provider
Rationale: Training medical staff could mean conducting procedures on a functioning 3D body simulated in VR rather than a real patient, where mistakes in the latter could result in life-long health complications. The easily duplicable, resettable VR simulations — in combination with VR's remote capabilities — means all training can be conducted remotely and with a low risk of physical injury.
Assumptions: The simulations must be reflective of a realistic workplace scenario, with appropriate interactive capability. Additionally, if the training is conducted remotely, all trainees have sufficient internet access and the low network latency required for them to immerse in the training.
Specific considerations: As mentioned, these VR training experiences should be used as a supplement to real-life training, never as a substitute.
Existing implementations:
- University College Hospital (London) has adopted a VR surgery simulator to train its medical staff [137]
- The University of Melbourne's Virtual Reality Surgical Simulation laboratory has developed a VR simulation for ear surgery [142]
- The University of Toronto's medical school is using made-in-Canada VR technology to train surgeons [143]
Application: Emergency response and disaster relief planning
Description: A VR or AR live tracking 'sandbox' map for strategic disaster and emergency response planning
Role of government: Application user
Rationale: Emergency responses can be high-risk. Strategic placement of team members can be crucial to the success of the operation, and knowledge of the live locations of these units can ensure action in the most efficient and safest manner. By generating a dynamic 3D replica of the target environment with a live feed of the location of all units deployed in the area, that is accessible through either VR or AR, the response team managers can determine the best strategy moving forwards. An example is during large-scale bushfires. It could be beneficial to see a live rendering of the bushfire front and its pattern of spread, along with the live locations of firefighter troops in the area, so that effective strategies can be developed.
Assumptions: There is a low-latency method for troops to broadcast their location to the application that runs the 3D rendering and tracking.
Specific considerations: The rendered sandbox map does not require high fidelity, as long as it portrays the pattern of spread and the location of each of the troops to the requirement of the strategists.
Existing implementations: There are no large-scale VR response procedures similar to the description above, however there are pilots for disaster response and mass-casualty VR response training in the US. [144]
[137] Virtual Reality Training For The Emergency Services - BBC Click. (2020).
[138] VR Tactical Training of the Zurich City Police with Refense. (2021).
[139] NYPD using VR to train for active shootings and real-life scenarios. (2019).
[140] US Department of Homeland Security - Science and Technology. Enhanced Dynamic Geo-Social Environment. US Department of Homeland Security - Science and Technology (2022).
[141] Ars Electronica. Impressions of Challenge: Emergency Error Battle / Oö. Landes-Feuerwehrverband (AT), Rosenbauer International AG (AT), Ars Electronica (AT), g.tec medical engineering GmbH (AT). (2018).
[142] O’Leary, S. Virtual Reality Surgery Simulation. Melbourne Medical School.
[143] How virtual reality is improving surgical skills during the pandemic. (2021).
[144] Hsu, E. B. et al. State of Virtual Reality Based Disaster Preparedness and Response Training. PLoS Curr 5, ecurrents.dis.1ea2b2e71237d5337fa53982a38b2aff (2013).
[145] Etter, C. Game Science Center exhibit: Augmented Reality Sandbox. (2014).