Application class: Bringing people together
This class encompasses metaverse applications that enable social activities: engaging in conversations, conducting meetings, or interacting with other people. Such applications typically allow participants to interact with digital avatars of other people in a shared environment. The environment could be entirely virtual if using VR, or a real environment with AR.
Building larger shared virtual environments provides the possibility of scaling levels of co-location: for example, whilst a virtual meeting room allows participants to directly converse, a virtual office building or city block gives the opportunity for different styles and frequencies of interaction.
Existing alternatives
Obvious alternatives that should form a baseline of comparison for this class of applications include
- co-locating in the real world, for example meeting in person or sharing an office space
- other digital communications tools like phone calls, email, video conferencing etc.
Potential benefits
- Convenience: In many circumstances bringing people together with metaverse technology will be cheaper, easier and faster than doing so in the real world, especially for people in remote or distant locations.
- Prevention of disease transmission: Virtual co-location removes the risk of disease transmission (which is particularly pertinent during the Covid-19 pandemic).
- Better social immersion than 2D video conferencing:
- VR: The immersive property of VR environments may provide direct benefits in the quality of the social interaction compared to other forms of remote communication like video conferencing. Spatial audio, for example, allows small groups to huddle together and have separate conversations in the same virtual room as they might in a real gathering.
- AR: Similarly, interactions enabled by AR are reported to be more similar to natural face-to-face interactions than screen-based alternatives. This may leave users feeling more attentive, focused and motivated. [119] AR can also help improve spatial reasoning and visualisation abilities, facilitate kinesthetic learning, and increase creativity by allowing users to explore and absorb new knowledge, and solve novel problems. [120]
Potential harms
- Lack of access control: Depending on the level of accessibility of the metaverse application, it may be difficult to control who is permitted to attend. This issue may arise particularly when using large, 'open world' platforms such as Meta's Horizon Worlds or The Sandbox to hold events.
- Anonymity may encourage anti-social behaviour: Like many online situations, physical remoteness and possible anonymity of participants may encourage anti-social behaviour which can be difficult or impossible to police in large settings.
- Excludes users who cannot wear headsets: Though users that can't afford headsets or are unable to wear them for extended periods may still be able to participate (using their phone, for example), this risks creating a two-tier experience that excludes or disadvantages those individuals.
- Excludes remote users: Lack of access to high quality network infrastructure is most common in remote areas, potentially excluding the very people that stand to benefit most from these applications.
Example applications
Application: Virtual nursing home visitations
Description: VR visitation rooms for residents of nursing homes where in-person visitations are infeasible.
Role of government: Application user
Rationale: During quarantine and lockdown, in-person family / friend visits to facilities such as hospitals, aged care homes and prisons were not permitted. NSW Government staff indicated that this was particularly problematic for people in nursing homes, as visits from family & friends could be a crucial social interaction. By bringing people together using VR, real-life in-person visitations can be replicated without requiring family and friends to travel to the physical location.
Assumptions: Nursing homes adopting this strategy are equipped with the VR hardware and network infrastructure required to run these virtual environments in a realistic, immersive manner.
Specific considerations: In-person visitations may be preferable to VR visitations. VR visitations should be seen as an opportunity to increase the number of interactions between residents and their families rather than replacing in-person meetups.
Existing implementations:
A similar AR-based nursing home visitations study yielded positive results. [121]
A number of companies have already begun to stake a claim in this space, offering VR experiences to aged-care residents. [122] - [125]
Application: Virtual tech precinct for NSW
Description: A dedicated virtual space for tech startups, problem owners, domain experts and academics to come together and innovate within.
Role of government: Platform provider or application provider
Rationale: By creating a 'virtual tech precinct' in a virtual world, the NSW Government can encourage collaborations between departments involved in new technologies and private companies in the VR space.
Metaverse virtual spaces combine many of the benefits of Zoom or Teams spaces (global reach, inexpensive to run) with the social aspects and spontaneity of in-person encounters that are often crucial for innovative endeavours.
Assumptions:
A sufficient number of people with access to VR headsets would be interested in such a hub to attain a critical mass of participants.
Participants are willing to wear a headset for however long they intend to participate in the hub.
Specific considerations: The global reach of these spaces can be a double-edged sword. International talent can contribute to the space, but local talent would also have access to similar workspaces set up anywhere else in the world. Any virtual tech precinct will need to invest heavily in developing its community to make it an attractive location for potential collaborators to base themselves.
Existing implementations: Horizon Workrooms offers elements of what is required for a virtual workspace. The application provides virtual meeting rooms in which crowds of up to 16 people can interact. The application lacks any sense of physical connectivity between rooms which one might expect in a hub. Avatar movement is restricted to teleportation between preset locations within each space, preventing elements of socialising that arise through less structured mingling.
Application: VR Service NSW
Description: A 'virtual NSW Service Centre' which can be accessed through VR, allowing users to access the same services they would in a real service centre, but from any location.
Role of government: Application provider
Rationale: In this service, users enter a virtual room containing avatars of NSW employees, that they would be able to converse with to conduct their business with the NSW Government. This service could be accessed from the home and could be focussed towards groups with mobility issues such as the elderly or remote communities who may prefer a face-to-face interaction with a representative rather than through a website or over the phone.
Assumptions: The government would need to provide this service functionality as a stand-alone VR app, or as part of a widely-adopted metaverse application. It would need to be bespoke, and would need to have the endorsements of the NSW Government in terms of regulations, behaviour management etc.
Specific considerations: This strategy would only be effective if there is large-scale community take-up of VR, where the public are comfortable spending significant time periods in VR and conduct activities there as part of their daily lives. The need for a digital verification method would be crucial in this application to prevent malicious actors from stealing the identities of others. Safeguards are currently in place for the online and phone services. These may be a suitable starting point for a VR centre too.
Existing implementations: None in the public service, and there are no other similar examples globally. However, as discussed in Government interest and adoption, some international governments are investigating ways to involve themselves in the metaverse.
Application: 'Over-the-shoulder' live collaboration during surgery
Description: The use of AR headsets as a 'telementoring' medium, for example engaging remote experts while undertaking unfamiliar medical procedures.
Role of government: Application user
Rationale: This approach could be life-saving if we consider high-complexity and time-crucial surgical procedures with experts on the other side of the world. A local surgeon could use their AR headsets to contact a specialist during an unfamiliar procedure. This call would be superimposed on the worker's headset and a camera feed showing their environment would be transmitted live to the expert. This allows both expert and worker to have a common line of sight, therefore a common frame of reference when offering advice, allowing for greater collaboration and a reduced chance of miscommunication.
Instead of the patient needing to travel for the right care, a local surgeon could put on an AR headset and communicate remotely to an expert who coaches them through the procedure.
Assumptions: The reception in the area of work is able to handle live video feed uploads and downloads, and audio or video conferencing is of an acceptable quality.
Specific considerations: For surgeries, low network latency would be a requirement.
Existing implementations: The University of California San Diego and Naval Medical Center San Diego (NMCSD) collaborated on an Augmented Reality Technology-Enabled Remote Integrative Surgery (ARTEMIS) project. [126] In this project, 'expert surgeons in remote sites use VR to access a 3D reconstruction of a patient's body, and instruct novice surgeons on complex procedures as if they were together in the operating room. Novice surgeons in the field can focus on saving a patient's life while being guided through an intuitive AR interface'. A separate study on the use of AR for 'telementoring' and remote assistance in spine surgeries also showed promising results. [127]
Application: VR courts
Description: As an alternative to video conferencing, some court sessions could take place in a VR space.
Role of government: Application provider
Rationale: VR technology could offer a number of benefits in the courtroom setting.
- Jurors can visit a virtual reconstruction on the crime scene giving them a deeper understanding and additional context about key events.
- Defendants could be given the opportunity to be represented by a generic avatar of neutral appearance to conceal traits that are irrelevant to the trial such as gender or race but which may influence jurors.
- Virtual courtrooms could offer a safer setting for defendants or witnesses whereby they can choose who they see when giving their testimony.
Assumptions: The changes to the court and its operations still comply with all of the legal requirements that the trial must adhere to.
Specific considerations: There is precedent for Australian courts becoming early adopters of new technologies. The Australian High court was one of the first in the world to admit evidence via video link. [129]
Existing implementations:
- VR reconstruction for jurors [130]
- Fair administration of justice for groups affected by prejudice [131]
[119] Alzahrani, N. M. Augmented Reality: A Systematic Review of Its Benefits and Challenges in E-learning Contexts. Applied Sciences 10, 5660 (2020).
[120] Diegmann, P., Schmidt-Kraepelin, M., Eynden, S. & Basten, D. Benefits of Augmented Reality in Educational Environments - A Systematic Literature Review. 16 (2015).
[121] Abels, E. A. M. et al. Augmented Reality-Based Remote Family Visits in Nursing Homes | ACM International Conference on Interactive Media Experiences. in ACM International Conference on Interactive Media Experiences 258–263 (2021). doi:10.1145/3452918.3465502.
[122] NeuronsVR. NeuronsVR Home. NeuronsVR (2022).
[123] SilVR Adventures. Virtual Reality Experiences for Seniors. Silvr Adventures (2022).
[124] WyndVR. Virtual Reality for Seniors. MyndVR.
[125] Rendever. Virtual Reality for Seniors. Rendever (2022).
[126] Weibel, N. et al. ARTEMIS: Augmented Reality Technology-Enabled reMote Integrated Surgery. HXI - Human-centered eXtended Intelligence (2022).
[127] Cofano, F. et al. Augmented Reality in Medical Practice: From Spine Surgery to Remote Assistance. Frontiers in Surgery 8, (2021).
[128] Navy Medicine. NMCSD ARTEMIS Project. (2020).
[129] Williams, D. R. Use of video recordings and video links by courts and tribunals. Western Australian Law Review 17, 257–271 (1987).
[130] Yeo, A. Jurors could use VR to visit crime scenes, and help them reach a verdict | Mashable. Mashable (2021).
[131] Meltzer, I., Zhang, K. & Lim, J. A New (Virtual) Reality for the Courtroom. Sydney University Law Society (2020).