Category: Dev Blogs

Overview

• Searights Tollhouse Visit
• Experience Design
• Tech Updates
• Design Production
• Challenges
• Next Week’s Goals

Searights Tollhouse Visit

On Monday, we visited Searights Tollhouse, the same location as our 3D model from the Portals Project. We got to learn about how the tollhouse keepers lived and worked in that building, how they kept records, and some fun facts about the building and the National Road in general!

Experience Design

This week, we focused on developing some narrative for our experience based around the tollhouse, since we have the realistic model for it.

We met with both Carl Rosendahl and Chris Klug this week to discuss constraints and effective AR design and narrative design, respectively. The biggest piece of advice we received was to define very tight constraints that we should follow, given how open-ended our project requirements are, as well as to lean into the fact that this is still a fictional experience rather than trying to make everything look extremely realistic if that is too difficult.

We thought one engaging interaction that people would find entertaining is if they could talk to the horse, so along with the rigging for the horse being done this week, we also have an animation for the horse’s dialogue.

Tech Updates

Building off some interactions that we came up with last week, we began to build some prototypes for a couple mechanics that we can focus the gameplay experience around.

For this week, we focused on using Niantic’s object detection to allow players to collect tolls from real-world objects like cars or other people. At this point, the mechanic of collecting tolls is still not very convincing as being particularly engaging, and there is also a major drawback with the AR package, which is that the object detection has no way of keeping track if you have already seen a specific object once it has left the frame and come back in. As a result, the player could theoretically keep looking away and looking back at the same object to infinitely collect tolls.

Due to these issues—and also because we don’t want to encourage the player walking on an actual road with lots of cars or people while playing—we decided that any toll collection mechanics that we pursue should be limited to in-game characters, where we can better control the states, perhaps with some extra easter egg mechanic of collecting tolls from other people who walk in frame only if time permits.

The other major mechanic that was integrated this week was the ability to render a large rectangular environment on the road, in the direction of the actual ground detected during gameplay, and ensuring that there is enough unobstructed ground (at least a set minimum length and width) in whatever open area the player’s phone is pointing before the environment is spawned. This was a challenge because Niantic’s package does not automatically provide the data about the orientation of meshes being generated on the detected ground layer. As a result, we had to create a backend system that finds the mesh objects and uses a principal component analysis (PCA) technique to find the primary direction of the road and then calculate the length and width along that axis.

We also implemented a flexible dialogue system using the Ink Package for Unity, which will allow our narrative designer and programmers to easily collaborate on adding in dialogue that interacts heavily with the in-game functions.

Design Production

This week, we finalized our poster, half-sheet, team logo, and team photo!

Challenges

The main challenges for this week were working within the constraints of the Niantic Spatial Package, as detailed above, in implementing the mechanics that we need in order to properly render a scene in a way that looks good and then build gameplay on top of that. Having overcome some of these issues, we can start creating playable builds for next week and start receiving playtesting feedback.

Next Week’s Goals

• Attend Tuesday’s playtest night on main campus and have a playable build that lets us receive feedback on our current system and interactions
• Improve the rendering of the scene in AR based on where the player is standing and where the road is

Overview

• 3D Models
• Tech Demo
• Challenges
• Next Week’s Goals

3D Models

This week, we gained access to many of the 3D models that we will utilize in our project. This includes the scan of Searights Tollhouse from the Portals Project, as well as a Wild West-themed asset bundle that includes some human characters, horses, wagons, and contemporary buildings from the era of the National Road.

Our artists will be focusing their efforts on arranging a layout of these buildings, as well as rigging and animating them for our app design. They will also be working on creating realistic textures for the road at different stages of history, UI and other 2D art for in-game interactions, and visual effects to enhance the experience.

Tech Demo

With the new models ready to test out, our programmers began creating prototypes, testing out spawning objects and having them move around on the ground in an AR scene.

One challenge that we needed to overcome was that most of the tutorials and demos available online showing how to set up an AR scene with Niantic’s package show how to detect the ground and then have the user manually spawn in objects (usually by throwing them into the scene). However, this approach does not work for our project, since we want to have more complex scenes that exist on their own without player input.

We were able to build a prototype that uses the location of the Niantic semantic layer meshes in the scene to place objects on the ground without the user needing to click on the screen or do anything, and this foundation will let us build up more complex prefabs that can contain all the objects that will spawn on the ground around the player and that the player can interact with.

Challenges

As mentioned above, a major challenge for this week was taking our tech exploration with the Niantic Spatial Platform and converting what we learned into a demo that would be useful for our project, one where objects can be placed on the ground using Niantic’s semantic layer data automatically, without any user input, so that we could spawn more complex scenes and environments around the player

Next Week’s Goals

• Field trip to Searights Tollhouse
• Meet with subject matter experts Carl Rosendahl (Distinguished Professor of Practice and the Director of ETC’s Silicon Valley campus, who has a lot of experience with VR and AR startup projects) and Chris Klug (ETC faculty, extensive narrative and tabletop design experience) to seek their advice
• Build prototypes for AR interactions that we designed and create a demo that can be tested at the IDeATe/ETC weekly Playtest Night on main campus
• Finalize in-progress team promo material including the team logo, half-sheet, poster, and team photo

Overview

• Quarters Walkaround
• Synthesizing Feedback
• Challenges
• Next Week’s Goals

Quarters Walkaround

In preparation for our Quarters Walkaround on Wednesday, we came up with two distinct directions that our project could go in, which we wanted to receive feedback on. One is an indoor tabletop experience, which is very flexible to be played almost anywhere, and allows us to imbue our product with significantly more narrative and educational design. The second is a fully outdoor experience, which takes advantage of the Niantic Spatial platform in a way that the first does not require, and which has the benefit of making the player feel more immersed in the environment, as if they are actually in the scene, and provides a sense of scale and awe.

Going into Quarters, one thing we were sure about was that we needed to manage scope and we didn’t want to do essentially 2 projects at once. In other words, we will need to pick one of the two directions to commit to. We should think about ways to simplify both versions, such as using pre-built, realistic 3D assets in the outdoor version. Another element to consider is the players’ motivation for caring about developing the road. One possible design decision is to make the player take on the role of a tollhouse keeper to become more invested.

For our experience and potentially transformational goals, we thought about how educators could use this app or game either before or after field trips to sites like the National Road Museum to get kids more interested or keep them interested in the National Road and related historical topics.

Synthesizing Feedback

After Quarters, we organized the feedback from each group into an affinity diagram to pull out the most common sentiments.

The questions and areas to focus on that we got from the experience can be summarized by 4 general categories:

1. Scope: One location is definitely better than trying to do multiple, and we should aim for a vertical slice. Different faculty had different preferences and opinions on whether an indoor or outdoor experience would be more successful.
2. Enjoyment/Value for the Audience: How will our experience be fun for teens in our target audience who have no prior familiarity with the National Road? What is fun and interesting in general about what we are designing?
3. AR/Technology: What does AR give us that other platforms can’t achieve? How do we balance making something that shows off the tech with something that is actually fun for our audience?
4. Location/Context: Is it actually beneficial for this experience to work from anywhere? Where should our audience play this? Should it be part of a bigger educational activity (like a road trip or visit to the National Road Museum)?

In our instructor meeting we discussed how to take this feedback and move forward, so that we could begin creating prototypes and playable builds as soon as possible.

We need to further explore the capabilities of AR technology, since this project was pitched to be an AR experience, so it’s a given that we will use AR and cannot change that component. The rationale for choosing AR was the opportunity to provide movement and interaction to old-school, static dioramas in the museum.

As for the location, our decision to make this experience playable anywhere makes sense because very few people know what the National Road is, and we want this to be a tool that can introduce them to it. If we made a tabletop experience, we would need to really land the gameplay and create something unique and innovative in order to have a successful project that meets our goals, as we would be cutting out a big piece of the pitch if we didn’t use Niantic’s semantic layer detection outside.

On the enjoyment and audience value side, having a narrative will be a lot more difficult and not work as well if we choose the direction of an outdoor experience, but we want to start with narrowing down our player experience goal and deciding what we want the player to feel. As inspiration, we looked at the Oregon Trail tabletop game, where a large aspect of that game is intentional frustration, and the game successfully achieves its emotional and experiential goals.

Ultimately, we decided to pursue an experience that is played outdoors, so that we continue to use the Niantic Spatial Platform, which was a big aspect of the original pitch, and that we could lean into the ways that life-scale immersive environments could be an interesting medium for an interactive experience.

Challenges

The biggest challenge for this week was committing to a design direction that we felt most aligned with our project and personal goals, balancing the possible risks and benefits of each version of our design, as well as which we felt we would learn the most from pursuing.

Next Week’s Goals

• Create several tech probes that show possible interactions and just rendering a scene in general. Start incorporating visuals from the artists and see what’s actually exciting in this space
• Follow up with Portals Project team to meet and get models for Searights Tollhouse and other buildings
• Make progress in anticipation for halves on the art, programming, and historical research sides

Overview

• Field Trip: National Road Museum
• Historical AR Research
• Niantic SDK Experimentation
• Storyboards
• Challenges
• Next Week’s Goals

Field Trip: National Road Museum

This week began with an exciting field trip across state lines to the National Road & Zane Grey Museum in Ohio! There, we got a crash course on the history of the National Road and the evolution of transportation in the United States from the very knowledgeable Betsy Taylor.

The museum has a very impressive diorama spanning over 100 feet long which represents over 700 miles of road across more than a century of growth and change. The dioramas—all hand-painted and sculpted—tell a compelling story of who was traveling across the road over the years and the types of buildings and towns that emerged to support commerce and traffic. Supplementing the dioramas are real examples of the vehicles used to traverse the road, including the iconic Conestoga wagon, along with early automobiles and bicycles before the invention of the bike chain. The charm of the dioramas and the surprise of witnessing the true scale of the real historical artifacts are emotions we want to capture with our project.

On the way to and from the museum, we also got to briefly stop by an S-bridge, which is a type of structure built to connect segments of the road that were not fully aligned on either side of a river—these were common since the National Road was built in small, disjoint portions rather than as one continuous stretch. We also saw sections of National Route 40, the modern legacy of this history.

Historical AR Research

We began researching existing augmented reality applications, particularly ones that had historical premises or otherwise were part of educational projects. One experience recommended to us by our project instructor, Mike, is the Gettysburg AR Experience which displays notable scenes from the Civil War (like the Battle of Gettysburg and Lincoln’s Gettysburg Address) in life-scale AR. We want to take inspiration from the immersive nature of rendering large artifacts right in front of a player, which they can interact with through their screens.

We also found our own examples and evaluated what features made AR a unique medium which we should try to incorporate into our design. A lot of historical examples that we looked at either opted to render—or project a rendering onto—one artifact at a time, which players could walk around and see from many angles. Alternatively, some experiences were very heavily tied to a specific location. With our design, we wanted to try something unique from these examples that we found because we want to give users more freedom to experience our app from many locations while still seeing some artifacts, like wagons and buildings, that are very large in life-scale.

Niantic SDK Experimentation

The team got access to 4 Android phones that we could use to test and run our builds, and we began experimenting with the Niantic SDK to see what it was capable of. The main feature we were most interested in was the semantic layer detection, so that we can render life-scale objects based on where the road or ground is that a player is standing on. We found that Niantic supports both highlighting the semantic layers on UI canvases and also generating mesh objects that can be textured to look like a road.

Storyboards

We met with Jesse Schell last Friday to discuss storyboarding and get his advice for designing virtual reality and augmented reality experiences. This week, we pursued his advice in brainstorming, and we separately listed out a lot of possible learning and/or transformational outcomes, interesting experiences, and cool features of AR as a launching-off point for us to pick the elements that we are most interested in.

Based on our visit to the museum and our brainstorming process, we created a storyboard of an experience with 4 stages, the first three dedicated to showing short segments of the National Road’s progression while being constructed, and the final one being longer and showing a developed town along the road. We came up with interactions and achievements that would progress the levels, such as cutting down trees to clear a path for the road, building an S-bridge to cross a river, and building a tollhouse once the road’s construction was nearly finished. We will continue refining this design, so that we can receive feedback from faculty at Quarters next week.

Challenges

The main challenge of this week was simultaneously researching the history of the National Road and the capabilities of AR technology and then combining all of our findings into a design that matches our project goals.

Next Week’s Goals

• Preparing for Quarters presentation on Wednesday
• Synthesize Quarters feedback and decide on a solid direction to begin prototyping
• Prepare to attend Transformational Design Workshop
• Reach out to Portals Project to request relevant their 3D models from scans
• Begin working on fast sprints for prototyping to see what works

Overview

• Project Kickoff
• Research & Brainstorming
• Composition Box
• Challenges
• Next Week’s Goals

Project Kickoff

We’re excited to be kicking off our project! The requirements are very open-ended, and there’s no preconceived notion of what this project will become. Our only constraints are that we make

1. an augmented reality application for phones (preferably using Niantic’s SDK), which
2. is centered around the historic National Road.

While we don’t have a specific client for our project, we plan to contact and visit local organizations that have done relevant work, such as the National Road Museum in Zanesville, Ohio, Searight’s Tollhouse in Fayette County, and the Perennial Project in Brownsville, Pennsylvania, which captures 3D scans of historical buildings.

Research & Brainstorming

The team started researching and brainstorming possible settings and AR interactions that could get an unfamiliar player invested in the topic of the National Road. We wanted to find some interesting stories about the National Road as well as learn about how to design good AR experiences, especially ones that are historical and educational.

What is the National Road, anyway? It was the first federally-funded highway in the United States, with construction spanning from 1811 to 1837, which connected Cumberland, Maryland, on the Potomac River to Vandalia, Illinois, and facilitated westward travel for thousands of Americans. The current U.S. Route 40 is the lasting legacy of this highway. During the decades of its development, as well as the subsequent decades of evolving transportation, the road transformed from a footpath cleared entirely by manual labor to a vital source of commerce fueling the prosperity of many cities that popped up along its path. In the experience we are building, we want to capture this progression and evolution and show how this road shaped a nation and its people, and vice versa.

Why use augmented reality and Niantic’s spatial platform? The National Road Museum is a wonderful educational attraction that uses charming and beautiful hand-painted dioramas to showcase the road in all of its glory, with knowledgeable facilitators who can share stories and answer questions. However, one goal of this project is to explore more futuristic methods of communicating history, using devices that almost every person has on them all the time. AR allows us to turn a phone into a portal through both time and space, and it can show people historical artifacts with the proper sense of scale, making players feel like they can directly see and interact with objects from a different time as close to physically interacting with them as possible. Niantic’s SDK is particularly good at detecting different semantic layers, such as ground, foliage, buildings, and sky, so we can use this feature to render the actual road itself and show how it changed over time.

Composition Box

We finished out the week by completing a composition box and attending the Playtest to Explore workshop to get some feedback on our initial ideas. We wanted to target our app to teenagers aged 12-16, who might be learning about American history in school and potentially visiting stops along the National Road, including the museum. These were some of our player experience goals:

• Let the player experience the life of Americans along the National Road by visually presenting the buildings and environment around the road.
• Players will come away with a better understanding of the history and culture of the National Road across the decades of its development.
• Players should feel like they embody their role as someone who lives in this time and place in history.

During the workshop, we asked our peers what they thought was the most interesting part of life in early 1800s America as people were moving west, and the responses we received centered around interesting anecdotes about the time and learning about how the modern cities we know today were different in that time.

Challenges

The biggest challenges of the week were synthesizing all of the new research and information we learned and coming up with a direction and goals for what we are going to build over the next few weeks.

Next Week’s Goals

• Take a field trip to the National Road Museum
• Research and experiment further with Niantic’s SDK, following their tutorials for setting up the tech
• Flesh out game design for quarters, doing research and attending the transformational design workshop
• Contact the Perennial Project and request access to 3D model of Searight’s Tollhouse