With Halves behind us, Beyond Frame is entering a new stage of development.

The feedback we received was encouraging, and our prototype has demonstrated that the core experience is technically possible. Visitors can enter a painting, move through its layers, interact with elements of the artwork, and take home a personalized print.

However, one piece of feedback raised an important question:

What is the difference between interacting with a painting and actually feeling like you are inside it?

This week, we began exploring that question through new interaction concepts, improvements to visitor integration, and a more intentional relationship between the digital artwork and the physical installation.

Rethinking Our Interactions

One of the most consistent messages from Halves was that our interactions could go further.

Our existing forest painting already supports several interactions, including depth-based movement, hiding behind trees, falling leaves, and environmental responses to visitor movement.

These have established a foundation, but we want to avoid repeating the same interactions across every painting.

Instead, we are beginning to think about painting-specific interaction design.

Each painting has its own atmosphere, composition, and visual language. We believe its interactions should emerge from those qualities rather than being added as a universal set of features.

For our second painting, which uses a pointillist style, we are exploring wind as a central interaction.

Visitor movement could generate wind that affects different parts of the artwork. Elements might move, the visitor’s appearance could respond, and the environment could gradually settle when the visitor stops moving.

We are also exploring a third painting with water as a significant visual element.

Potential interactions include creating ripples through footsteps, generating splashes, seeing a reflection in the water, or influencing the movement of a sailboat.

These ideas also introduce new design challenges. If a visitor blows a hat off a figure or moves a sailboat, how does the environment return to its original state?

Rather than treating these as purely technical resets, we want to consider how the transition back can become part of the artwork’s behavior.

The third painting has not yet been finalized, but these experiments are helping us establish a clearer direction for future interaction design.

Making Visitors Feel Part of the Painting

Another important area of feedback was the visual integration of the visitor.

During playtesting, one visitor described feeling that they were interacting with a painting, but not necessarily standing inside it.

Several technical issues may contribute to this feeling.

Body segmentation can produce noisy edges, small visual artifacts, or outlines around the visitor. Differences in perspective and scale can also make a person appear to be placed on top of the artwork rather than naturally belonging within its composition.

We are now investigating ways to make this integration more convincing.

One possible direction is to prioritize smoother visual blending, even if that means sacrificing a small amount of segmentation precision.

Another is to reconsider how much of the painting we display during the interactive experience.

Instead of always showing the entire painting, we could transition into a closer view of a selected region when the visitor enters.

Full painting → enter the frame → focus on an interactive region → explore → return to the full painting

This approach could help us achieve more believable scale and perspective, although it may also limit how much of the artwork visitors can explore at once.

It is a tradeoff we want to test rather than decide in advance.

Guiding Without UI

Another recurring challenge has been helping visitors understand where to go and what to do.

Because Beyond Frame is intended to feel like entering an artwork rather than playing a conventional game, we want to avoid obvious interface elements.

Instead, we are exploring environmental cues.

One direction is to connect the physical entrance frame with a corresponding digital frame or glimpse into the painting.

Rather than presenting an empty entrance, parts of the artwork could already be visible within the frame, inviting visitors to approach it.

After entering, visitors might initially appear partially hidden behind elements of the painting, immediately demonstrating that they can move through its depth.

We are also reconsidering the physical layout of the entrance.

The frame should become the natural destination when visitors enter the room, with the surrounding space directing them toward it rather than relying on written instructions or obvious floor markers.

Lighting could help visitors find the correct starting position, while the frame itself could provide a more intuitive transition into the experience.

Interactive objects could also reveal themselves only when necessary.

For example, a tree might begin to move slightly as a visitor approaches. A branch could lean toward them, or a very soft glow could briefly appear around an interactive element.

Rather than explicitly telling visitors where to interact, we want to create moments that invite curiosity.

The goal is not to communicate:

“Interact here.”

But instead:

“Something seems to be happening over there. Maybe I should move closer.”

Bringing Sound and Lighting Into the Experience

We are also continuing to develop sound and lighting as parts of the interactive environment.

Instead of treating these systems as separate effects, we want them to respond to the same actions happening inside the painting.

For example, a visitor generating wind in the second painting could trigger movement in the artwork, corresponding environmental sound, and subtle changes in the room’s lighting.

Similarly, movement through water could create ripples on screen, footsteps or splashes in the soundscape, and changes in reflected light.

Our next technical goal is to connect DMX lighting control with Unity and continue developing multi-channel audio.

Different paintings should eventually have distinct environmental sound and lighting designs.

The forest painting, for example, might emphasize leaves, footsteps, distant voices, and sunlight filtering through trees, while a water-based painting could create an entirely different atmosphere.

The goal is for these layers to work together, extending the artwork beyond the boundaries of the screen.

Refining the Physical Installation

The physical frame is also undergoing further development.

Our current temporary structure has helped us test the experience, but the final installation needs to feel more intentional, stable, and visually connected to the artwork.

We are investigating a stronger freestanding structure with decorative moulding inspired by traditional painting frames.

This distinction is important to the concept: visitors are not simply walking through a doorway.

They are walking through the frame of a painting.

The layout also needs to support visitor safety, camera tracking, and a smooth transition from the museum-like entrance into the digital environment.

A More Considered Physical Takeaway

Our printed takeaway is evolving as well.

The current direction combines a personalized photo print with a designed envelope or sleeve.

The client responded positively to the idea of keeping this design simple and elegant, while including relevant information about the artwork and the museum.

We also want the design to remain adaptable. Different museums could use their own branding without requiring us to completely redesign the physical takeaway.

This allows the print to remain both a personal record of the visitor’s experience and a meaningful connection to the original artwork.

Preparing for Our Next Playtest

Our next major milestone is the October 31 playtest.

Rather than simply asking whether individual features work, we want to understand how the complete experience feels to visitors.

Some of the most important questions include:

• Do visitors feel like they are inside the painting?
• Can they discover interactions without explicit instructions?
• Do the interactions feel appropriate to the artwork?
• Does the environment encourage curiosity and exploration?
• Does the visitor feel visually integrated into the painting?

We also want to experiment with interactions that are more dramatic than our current design.

Our instructors encouraged us to test beyond our usual comfort zone. Some of these interactions may ultimately be removed, but trying them could help us discover a better balance between subtlety and surprise.

Beyond the immediate prototype, we have also begun exploring opportunities to present Beyond Frame outside ETC, including preparing a proposal and technical requirements for Meshwork Festival.

These possibilities are exciting, but our primary focus remains on strengthening the experience itself.

Looking Ahead

At the beginning of the semester, our main challenge was figuring out how to place a visitor inside a painting.

Now, we are increasingly concerned with what it means to belong inside that painting.

That requires more than body tracking and interactive elements.

It requires believable visual integration, carefully designed movement, intuitive guidance, and environmental responses that feel natural to the artwork.

As we continue developing our second painting and exploring a third, we want each experience to have its own identity while remaining connected to the central idea of Beyond Frame.

We don’t just want visitors to see themselves in a painting. We want them to feel, even briefly, that they have become part of its world.