#4 Tour.Go: Project Summary

Welcome to this final blog post for Tour.Go. I finished the project recently in May, and wanted to create this final entry in this blog series to demonstrate the core functionality and discuss challenges encountered. I will also provide the solutions adopted to achieve the project purpose and aims.

Tour.Go is an educational augmented reality app for tourism and sight-seeing. It was designed with the intention to encourage individuals to take an interest in cultural heritage and history.

Monument Detection

One of the most difficult, but vital features in the application was allowing users to detect monuments to read information and view 3D models in augmented reality. This was a system mentioned in previous posts, but there were additional steps to ensure intended functionality.

An unexcepted shortfall of using a mixture of geo-location comparison and AR was that the lighting calibration would occasionally move the object positions in the world space. This result was overlooked, and should have been considered earlier. Another aspect of using AR to position real-world waypoints to represent landmarks and monuments was that the starting camera position is always local. The figure below demonstrates the problem.

You can see that the above figure demonstrates the start orientation is always the same, no matter the rotation of the device.

The nature of AR experiences having a global orientation causes real-world waypoint positions to be assigned incorrectly. One suggestable solution would orientate the world objects to be always northward facing, taking into account the compass value of the device. Although this might work in practice, it could present additional complexities to the chosen implementation, considering the logic of disabling and reusing objects if location changes dramatically to only show waypoints in a given radius. There was some investigation to provide this implementation, but unfortunately it was unsuccessful. Due to time constraints, I could not redesign the monument detection, and so instead created a calibration display.

The calibration display allows the user to rotate their device to the correct orientation before loading the AR scene and all waypoint objects. Using the device compass, the calibration reaches one-hundred percent when the device is facing the assigned target rotation.

The addition of the calibration display meant that waypoints were accurately positioned to represent the real-world locations of monuments and landmarks (points of interest).

The 'Calibration Manager' is responsible for measuring the current rotation of the device and comparing it to the target rotation. When the target rotation is reached, the scan scene is activated for monument detection.

AR Object Manipulation Updates

Freely available 3D scans of iconic London monuments were available on SketchFab. This allowed for highly detailed models to be viewable in AR. However, the quality of these models also presented a challenge due to the file size. To ensure commercial viability and consumer satisfaction, the goal was to limit the application's impact on device storage. The chosen solution to this problem was using Fire Store cloud storage from Firebase.

A screenshot of the Cloud Storage interface in the Firebase console.

The result of using this system allows for asset bundles to be stored in the cloud and can then be downloaded at runtime when requested. An asset bundle is a compressed file version of a given prefab. The original file format of .glb was used due to its high performance in augmented reality scenarios and its popularity for 3D scans of real-world places/buildings.

The first step is to retrieve the URL of the file from cloud storage using the Firebase API in Unity C#. Once the file has been retrieved, the script downloads it and converts it from an asset bundle into the prefab. The advantage of using prefabs and asset bundles is that the parameters assigned in the components of the object is passable, meaning that the facts and information spheres present in the monument experiences can be easily assigned.

Users can select information spheres to read three interesting facts and a description concerning the monument. This was especially useful for large landscapes or landmarks with multiple points of interest.

An example of the display that appears when the user selects a blue information sphere.

One of the challenges encountered with 3D AR objects is the positioning in the physical space. To ensure ease of a accessibility and user understanding, the object will always be instantiated in front of the camera. This then allows the user to scale, position or rotate the object more easily.

Quiz System

The quiz system was a key aspect of the application that promoted cultural and historical landmark information and learning. I chose not to include an in-game reward to counter cheating or discourage interest/engagement in the topics themselves. It was only after the fact that I realised I could have included a timer that would have prevented the former issue. Each quiz had a specific theme including 'The Great Fire of London' or 'The Victorian Era', and so I decided to order the questions randomly to avoid users remembering the sequence of answers rather than the information. Overall, the quiz system proved to be an effective educational tool in end-user testing. The use of audio for correct and incorrect answers also contributed to immersion and made the quiz experience more exciting. Additionally, adding quiz music could lend into the experience well.

Badge Collection System

Badges enhance the users experience of discovering monuments. The system aims to categorise monuments into themes and encourage app engagement using the 'collect them all' approach. Users can see the progress of each badge on the collection screen.

When a badge is completed, a notification will appear on the top of the scan interface. This direct method of displaying user progress aims to continue to encourage retainment with the application. Finally, badges are a way to signal user progression and give the effect of completing a journey.

The badge system works using the real-time database as part of the Firebase API. Here, the badges that users have unlocked are listed under their unique identification number. The number assigned to each entry represents the progress to the badge. Each badge is given a maximum progression value, allowing for customisation of the overall objective to unlock the badge.

The class BadgeManager accesses the data applied in the real-time database and is also responsible for updating user data when a badge is unlocked. The class is also responsible for looping through the entries in each collection to display on the badge collection menu.

Guild Menu

The guild menu works similar to the badge system as it retrieves input from the real-time database and displays it to the user. Therefore, the guild manager compares all users and assigns their icon, username and guild colour to the leader board. The leader board for users that discover the most monuments aims to introduce social competition and continuous engagement with sight-seeing and discovering monuments.

When a user selects a guild, an index is assigned to the entry that corresponds with the user on the real-time database. The user can select the guild upon first opening the guild menu from the main screen. Additionally, users can select their player icon from a variety of pre-defined avatars in the player profile.

The main inspiration for guilds was from games such as Pokémon Go, an AR title that set the standard for interactive augmented reality experiences. Each guild is attributed a score, increasing when a guild member discovers a monument. For this, I created a visual bar chart that shows the guilds in 1st, 2nd and 3rd place. This is achieved by setting the size delta of the image rectangle transform and positioning the image component using the unity anchor UI system.

Summary

I enjoyed working on this mobile AR application as part of my final project, and am considering making improvements following its initial completion. I achieved a variety of systems and mechanics that I planned from the beginning, and was successful in executing an educational app for tourists. The project also provided challenges and learning new technical skills.

Thank you for reading, and I hope you found this recent post interesting.