ARviation is an ETC student project based on augmented reality, with the goal to create an experience that sparks the interest of children in diverse careers within aviation. We aimed to do this by presenting the basics of the science and engineering behind airplanes, primarily by showing how an airplane’s components work to help it achieve flight. Therefore, students will be exposed to the “world” of designing and building airplanes. Our client, MuseumLab, designed a children’s museum exhibit in the Center of Aviation Technology and Training for Hosanna House, another organization with the mission of fostering education in underprivileged communities within Wilkinsburg. Our experience was created to complement this exhibit.
The team, integrated by Wei Cheng (Jimmy) Chen, Chang (Ivy) Liu, Xiaoying Meng, María Laura Mirabelli, and Yu (Eric) Zhu, designed a multi-stage experience that incorporates the following materials, both digital/virtual and physical: an app titled BAM! Build A Mustang (developed with Unity, using AR Foundation), installed on iPads provided by our client; decks of cards (composed by eighteen playable cards and an organizational card); laminated, letter-sized sheets with a Morse code guide; laminated, poster-sized sheets with an airplane diagram and card “slots”; and AR markers printed as decals. The team also provided our client with an educators guide and technical documentation, to aid in the preparation of educators facilitating the experience and of staff in charge of maintaining the iPads.
Multiple aspects of this project went well. From the beginning, the design of our experience was straightforward, which let all team members easily focus on implementing their own tasks (functions for developers, assets for artists and designers, etc.). Our team members were willing to provide their ideas and give others constructive feedback, given we created a friendly environment that fostered conversation. Additionally, when we encountered problems, we were able to iterate and improve by going to faculty office hours, talking to outside groups and discussing internally. We also did a good job with keeping the design and development direction aligned with the client’s needs, e.g. AR experience, making use of a runway in the exhibit space, intriguing kids’ interests in an aviation career, etc. We were able to accomplish goals we set for ourselves at the start of the semester by maintaining a clear understanding of what the overall goals of the project were. With the time limit, we did not have to abandon any major part of our design and polished them to a good standard.
Although we made a well-functioning application, there are also still some known problems with its latest version: if installed on an Android tablet, it will freeze after running the app for a few minutes; the app has to be built in Unity’s development mode, otherwise AR photos shot in a specific scene may not be displayed properly; very occasionally, AR Foundation fails to detect AR markers. There were also flaws in our development schedule that we could have improved, given that a few bugs only showed towards the end of the semester. We would have avoided this situation by setting up several checkpoints for the product to ensure it is releasable and robust at each checkpoint. Another thing we didn’t do well is the modularity in our source code. As there were many iterations throughout our development, we made many changes and layers of functionalities in our code. While the best way to improve the modularity of the code is by setting up a standard framework for the codebase, it is hard to imagine the code without having a concrete idea of our project’s potential changes. Instead, we could leave time on each iteration to restructure the code and reduce its redundancy.
In terms of the design schedule, we could have playtested with our focus group earlier. We playtested with ETC students a few times before we playtested with kids; even though the ETC playtesting sessions were helpful to a degree, they were never as useful as playtesting with kids because the behaviors, educational levels, and interests of the playtesters were dramatically different. If we had playtested with kids earlier, we would have changed our design much earlier from manuals to cards, leaving more time to refine the latter. We also wasted time on finding free 3D models for our airplane and its components, incorrectly thinking this would be faster than building them from scratch. After weeks of research and working with the models, we realized that their wireframes were too messy to put textures on. Instead, we could have found good models to purchase online earlier or model them ourselves from the beginning to save time.
For overall organization and producer responsibilities, we tried implementing several tools to help organize our work that were unsuccessful, such as task lists and daily check-ins. We initially tried to use these so that we could work independently and remotely, but soon realized that our team worked best when we were all in the same project room, available to ask each other questions, discuss aspects of the project, and clear out any confusions or miscommunications on the spot. Finally, this team was set up with two co-producers with other roles (María was also doing UX/experience design and Xiaying was doing 3D art), which turned out to be difficult for both. We came to the conclusion that the producer role is best fulfilled when one person focuses solely on it, without any other additional tasks.
We learned many lessons throughout this project. For technology, we experimented with the ALPS beacon system, which is an indoor ultrasonic ranging technique that can be used to localize modern mobile devices. We did not adopt the technology in this project for the following reasons: ALPS provides the coordinates for devices, but our design was best served with the position and orientation of AR markers; ALPS has not been integrated into Unity, lacks documentation and technical support, and only works on iOS devices. However, the beacon system allows high precision without drifting issues and therefore might be useful in other location-based projects. Although we didn’t use ALPS, it was inspiring to see what the newest AR technology can do and its limitations. We used AR Foundation instead, which makes AR really easy to implement in Unity. However, as a platform in active development, it also has its flaws: it has version-compatibility issues; it includes ARkit (iOS) and ARcore (Android), but the details of the two low-level implementations may not be the same, leading to subtle differences; it requires specific marker designs, otherwise its readings are incorrect; when using two reference image libraries, AR Foundation does not work properly. Having learned this, we can now gauge what type of experiences AR Foundation is best suited for.
In terms of design, the biggest lessons concerned the difficulty of creating educational experiences that balance presenting challenges for users with their level of complexity and being fun and engaging to maintain users’ interest. We struggled with creating manuals and then cards in which the difficulty of the science and engineering behind aviation could be condensed into information snippets digestible to school children. We ourselves had to become familiarized with this knowledge to then be able to adapt it for our target audience without any misinformation. Then, this content had to be presented in an interactive way that allows users to approach it with ease but always learning from it. Our experience design process, which took a lot of inspiration from game design, taught us the complexities of achieving this, in terms of multiple iterations, playtesting sessions, and failed attempts to reach a successful outcome.
In conclusion, this semester provided a great opportunity for the members of team ARviation to learn about experience design for educational content, AR technologies, and overall teamwork and project development. Our clients at MuseumLab have reiterated how impressed they are by our work, which is the most important outcome we strived to achieve. Our app-guided experience will begin to be used at the Center for Aviation Technology and Training during the summer of 2022, and we were kindly invited by our clients to drop by and see it be actively used by school groups then. We look forward to learning about the experience outcomes for its target audience, and are grateful for the successful process we underwent this semester.