A NSW Government website

NSW Government
Digital NSW

Application class: Giving people experiences
Relevant metaverse properties

This class of applications explores objects and experiences in the form of interactive persistent virtual worlds and 3D visual overlays.

Virtual worlds are 3D rendered environments that users can interact with while in VR. These worlds can be replicas of the real world to encourage tourism, provide a unique form of creative expression for artists, act as a form of general entertainment for users, amongst other applications.

In addition to contextual information overlay, AR can be used to add virtual 2D or 3D elements to a person's view of the real world. An example is adding a holographic rendering of a poker machine to a person's environment while they are using AR, so they can play without needing to be in a physical casino, or superimposing live digital visuals onto a real installation without needing to build the physical infrastructure.

Existing alternatives

Obvious alternatives that should form a baseline of comparison for use cases with this application class are

  • engaging with the real world, e.g. visiting the real tourist destination rather than a duplicate in a virtual world
  • building the infrastructure for the 2D or 3D visuals in real life, e.g. projecting live digital visuals onto the physical scene using projectors

Potential benefits

  • Virtual worlds provide artistic freedom: Virtual worlds do not need to be bound by realism, and can openly manipulate visual elements such as gravity, colour, and shape. This creates endless new possibilities for artists and developers.
  • Virtual worlds are cheaper to create: Creating expansive virtual worlds can be significantly cheaper than developing these worlds in real life. In VR, the worlds can be built anywhere, by anyone. Individual buildings do not need purchase of real-estate or land (unless they are being built in a pre-established virtual world, such as Decentraland or The Sandbox) . Buildings and trees can be cloned efficiently, forgoing construction OH&S concerns due to their digital nature, making them quicker to complete and easier to modify upon completion.
  • AR provides unique tourism opportunities: By augmenting a real environment with dynamic and novel visuals, one can establish an entertainment experience that is not bound by the restrictions of reality, and also without needing to duplicate the world in virtual reality (see the example applications below). This results in a cheaper and more flexible setup without also de-incentivising tourists from visiting the real location.
    • For example, an AR-based Vivid Sydney event would be better for incentivising tourists to visit Sydney — as the AR would only work while looking at the points-of-interest in person — compared to a VR-based Vivid Sydney that duplicates Sydney in a virtual world.
  • Novel opportunities for artists: Digital art in VR and AR form also creates more opportunities for artists to showcase their work, as the art would only be available digitally. These art installations may not need to meet locale restrictions such as awareness of the city skyline or neighbouring obstructions.

Potential harms

  • Discourages in-person tourism: By providing virtual duplicates of tourist destinations and points of interest, in combination with the novel immersive opportunities presented by VR, it could discourage people from visiting the real-life location. Designers of the virtual world should be cautious to create a duplicate that intrigues, rather than de-incentivizes, tourism to the real-life destination.
  • Difficult to monitor for harmful behaviour: Virtual worlds can be difficult to regulate and monitor, which means harmful behaviour could be difficult to control.
  • Lack of regulation results in legal ambiguity: The lack of regulation could create a previously-unexplored and possibly legally ambiguous relationship between a real life location (e.g. a bar) and the services used by patrons while in the location (e.g. a virtual pokies machine). Especially for illegal activities, would the location be held accountable for allowing patrons to use these activities in the establishment, or would it be separated as part of the user's own internet activity?
  • AR visuals could be distracting: Supposing AR technology becomes widely-adopted, any augmentations to reality — artistic or informative — would need to have public safety in mind, so pedestrians would not be distracted or have their vision obscured in a dangerous way. An example is while they are crossing a busy street, or stopping in a busy walkway, to watch an AR light show.

Example applications

Application: An augmented reality (AR) enhanced Vivid Sydney

National Museum of Singapore
National Museum of Singapore's 'Story of the Forest' exhibit, where audiences can use an app to interact with the illustrations in the exhibit [150]

Description: Enhancing Vivid Sydney events and points of interest using AR

Role of government: Application provider / Platform provider

Rationale: NSW's annual Vivid Sydney event could be enhanced in AR. AR technology can add dynamic visual elements to points of interest in Vivid Sydney, with greater artistic freedoms compared to the real world. The NSW Government could develop a mobile application where users point their cameras at a Vivid POI and the AR feature renders superimposed visuals exclusive to people using the app. This approach could similarly be used for other points of interest across NSW to increase tourism, as these AR features would only be viewable when the user is in a direct line of sight from the venue, ensuring that the locals (e.g. nearby restaurants, bars) can benefit from this tourism as well.

Alternatively, the NSW Government could create a platform for AR Vivid, and local companies and artists could adopt the platform to create their own AR scenes with standards and regulations set by the NSW Government.

Assumptions: There is sufficient reception in the area to download and render these dynamic visuals on mobile devices. Additionally, the app is free to download and is provided or endorsed by the NSW Government. If the NSW Government chooses to become a platform provider for the AR Vivid service, it will need to ensure all of the external AR offerings have been approved before going live on the public-facing app.

Specific considerations: Vivid Sydney attracts a lot of people each year. Supposing the app requires an active internet connection and everyone is using their phones to view the AR graphics, there will need to be sufficient bandwidth to handle the large number of people in the same area. The graphics will need to be at an appropriate resolution without significantly draining battery life and internet data. The NSW Government will also need to be transparent if it is collecting any user data from these experiences. AR also may not work under low-light, so lighting in the surrounding areas should be considered.

Existing implementations: There are no existing implementations that align to the description above, however AR as a medium for enhancing public artworks has been explored.

  • Ohio's Akron Art Museum's 'Interplay: Art Play for All' exhibit where scanning a QR code via a mobile phone renders an interactive piece of artwork using AR [146]
  • The National Gallery of London placed QR codes all over the streets of London which renders an AR artwork when scanned by a mobile device [147]
  • The National Museum of Singapore hosted a 'Story of the Forest' exhibit with 3D animations that visitors can interact with by downloading an app [148]
  • The Museum National d'Histoire Naturelle in France hosts an experience called 'Revivre' that renders extinct animals in AR. Visitors use a Microsoft HoloLens to view the 3D AR animations [149]

 

 

Application: Virtual tour of the Sydney Opera House

Description: Generating a 3D interactive replica of the Sydney Opera House in a virtual world to encourage tourism.

Role of government: Application provider

Rationale: With the NSW Digital Twin already in development, there is potential to upload the 3D rendering of a NSW point of interest into a virtual world. An example landmark is the Sydney Opera House. Users could explore detailed and to-scale renderings of the Sydney Opera House in the virtual worlds through their VR headsets. Increased public interest in the area could encourage tourism, especially if the VR Sydney Opera House visit is also able to promote interest in surrounding areas.

Assumptions: The rendering would be available for free to the public.

Specific considerations: There may be important legal considerations that will have to be addressed regarding the digitisation and online display of the Opera House and its various art pieces.

 

Existing implementations: There are several 360 VR tourism videos already available for free on the public domain, however they are static videos and not interactive. Examples are guided tours of Venice [151] and aerial VR shots of Angel Falls in Venezuela. [152] Google Arts and Culture created VersaillesVR, a free interactive tour of the Palace of Versailles available through the gaming platform Steam [153], [15], however this is a standalone game and is not part of a virtual world.

 

 

Application: High school student VR tour of a manufacturing facility

3D replica of a manufacturing facility
Generating a 3D replica of a manufacturing facility in a virtual world could yield positive educational outcomes. [156]

Description: Replicating industrial facilities in virtual worlds for use as a remote educational tool

Role of government: Application provider

Rationale: Providing replicas of industrial areas — such as manufacturing facilities, foundries, or refineries — in a virtual world could provide educational benefits. For high school students studying subjects like physics and engineering, visits to these facilities could prove useful for their studies and provide insights into their career pathways. However, where these facilities are not operated locally or they are not open to the public, having a VR alternative for students to interact with may be cheaper and more manageable than a school trip to a real facility, especially if it is interstate or international.

Assumptions: For virtual school trips, students would require access to a VR headset, with appropriate bandwidth to explore these virtual environments.

Specific considerations: These replicas should not be used in place of a trip to a real site, particularly if a school trip is feasible and the facility is safe to visit.

Existing implementations: Similar to the 'Virtual tour of the Sydney Opera House' application, there are VR videos with tours of manufacturing facilities available for free in the public domain, such as one created by the American Fuel & Petrochemical Manufacturers. [155] However, they are static films and cannot be interacted with.

 

 

Application: AR gambling bars

Description: A NSW pub or club could offer AR 'virtual gambling', where for example, patrons interact with virtual pokies placed in the bar by augmented reality headsets.

Role of government: Regulator

Rationale: The AR pokies machines could replace physical pokies machines. The cost to build an AR pokies machine would be negligible compared to a real-world version. Removing the physical pokies machines would also result in more available bar space to host other forms of entertainment.

Assumptions: Wide adoption of AR interfaces commonly worn in public

Specific considerations: Creation of an AR gambling bar creates a new and possibly legally ambiguous relationship between the real location and the virtual location. There is an unclear division of responsibilities between the bar and the metaverse pokies application provider, and it is difficult to police any relationships between a physical space and the AR content digested by users in the space. Additionally, lowering the barrier of access to gambling in public venues may increase opportunities for problem gambling.

The regulatory role of the government would require resolving questions such as:

  • Under what circumstances would the poker machines 'in' the bar for the purposes of licenced gambling venues?
  • What if an augmented reality provider places the pokies in the bar without the bar's permission?
  • Do limits on numbers of physical poker machines apply?
  • How would safe gambling rules be applied?

Existing implementations: There are no existing examples of AR gambling bars in the nature described above, however there is a lot of public interest in this area.

 

 

[146] Akron. Akron Art Museum Launches Interplay: Art Play for All Augmented Reality Experience. Akron Art Museum (2020).

[147] AOL AUGMENTED ART GALLERY. (2021).

[148] Story of the Forest. (2016).

[149] Museum Nal Hist Naturelle. Revivre, Extinct Animals in Augmented Reality. Muséum national d’Histoire naturelle official website.

[150] Shing, C. Y. Story of the Forest. (2017).

[151] Venice, The Floating City: A Guided VR Tour - 8K 360 3D Video. (2021).

[152] 360°, Angel Falls, Venezuela. Aerial 8K video. (2017).

[153] Google Arts & Culture. Versailles: The Palace is Yours. Google Arts & Culture.

[154] Google Arts and Culture & makemepulse. VersaillesVR | the Palace is yours on Steam. (2019).

[155] Refining Virtual Reality Tour (VR). (2019).

[156] Feruz110. 3D model. (2018).