Introduction

Clients:
Renee Cox – Renee Cox is a photographer, artist, lecturer, and political activist.  Her works capture the identities and beauty within her subjects. She is using human bodies, both nude and clothed to portray black womanhood and deconstruct stereotypes.

Jessica Gaynelle Moss – Jessica is the curator for our exhibit. is an artist, independent curator and arts consultant. She runs her own arts consultancy, curates exhibitions and performances. 

The Mattress Factory – The Mattress Factory is a contemporary art museum located in Pittsburgh. The Mattress Factory has pioneered the development of alternative art forms through site-specific installations, video, and performance art. 

Team:  Hui Feng, Wenbo Guo, Yingzi Qu, Zijun Lin, Xuan Nie, Anjali Shah

Instructors:  Moshe Mahler, Heather Kelley

Faculty Consultant:  Ricardo Washington

Project Description:

In the spring 2022 semester we got the opportunity to work with The Mattress Factory, independent curator Jessica Gaynelle Moss and internationally acclaimed multidisciplinary artist Renee Cox. For this project, the team worked with Renee Cox to design and develop an installation that merges the power of digital media with Cox’s artistic vision and sensibilities for exhibition at The Mattress Factory in June 2022.

The team researched possible technical solutions for single or multi-channel installations, head mounted experiences, projection mapping, and other possible platforms that serve both the installation feasibility and the artistic integrity of the project. The team designed, implemented, and executed a production pipeline that brings this collaborative artistic vision to life.  This project is exploratory by nature and the direction of the visuals and overall user experience were determined, in part, by the skillsets of our project team.

Project Goals:

1. Embodiment: We incorporate audiences’ body with Renee’s photography, have them move and use their body gestures to control interactions, so that we can build up a bridge between Renee’s art piece and the audience’s own body.

2. Art Space: The interactions expand the 2D form for Renee’s artwork to spatial formats. Interacting with the artworks, people are stepping into the immersive space; observing others’ action and movement in the space, guests are expanding their individual experience into an environment – the art space.

3. Recreation: Guests’ interactions and poses serve as part of the art pieces, complete the artwork by recreating and regenerating new art formats.

Deliverables:

We delivered an exhibit spread across 2 rooms in the Annex Gallery at the Mattress Factory. The first room has an interactive experience where the guests can influence Renee’s art works and the second room is an immersive experience where the guests can believe to be inside a Utopia created by Renee. 

What went well?

Pleasant Client Relations:

We communicated with our clients – Renne, Jessica and the Mattress Factory very often. Along with our formal weekly meetings, we spent time with Renee outside of work which helped us better understand her artistic vision and expectations. The team bonded very well with Renee and knowing each of the team member outside of work and their talents let our clients trust at every stage of the project. Renee being the visionary artist that she is, allowed us to be mentored and motivated by her throughout the project and also learn many life lessons like ‘ Make a fuss’ or ‘Always ask for what you need’. 

Working with an Artist:

The artist, Renee, was actively involved in our development process while giving us the creative freedom that we needed to experiment. The exhibit based on her works, she was very clear of her vision for the project which helped us narrow down our research for the right immersive technologies and get very clear feedback. She was very decisive and understanding of the team’s challenges and limitations. She was always open to new ideas, helped us build upon all our pitches and our early prototypes inspired her to weave the narrative that the exhibit will be based on. We worked well together with the artist which allowed us to fulfill all her expectations and our clients are satisfied with the project.

Tech Exploration and Ideation:

The team explored various technologies at the beginning of the semester to find what suited the artist’s integrity best. Our team’s talent allowed us to explore interesting technologies like holographic imagery, projection mapping, motion sensing and zoetropes. Our client, Renee, was so excited by all our prototypes that she wanted to incorporate bits of everything in the final product. Our prototypes inspired the artist’s narrative for the exhibit which very easy narrowed down our technology that was projection mapping for an immersive experience and Kinect for an interactive experience. The team never fell short of ideas neither at an early stage nor closer to refining the project. We quickly iterated feedback from the guests and our instructors to add in any missing elements that would improve the guest experience. During the last 4 weeks we came up with an entirely new feature of a following interaction in second room that would allow the first room to narratively and aesthetically communicate with the second room. 

Tech support at ETC and MF:

We faced many challenges with tech through our development phase such as syncing 5 4K projectors with 1 PC or syncing more than one Kinect together or mapping videos in Unity amongst the many others. We were supported by the IT team at the ETC through and through. They helped us procure sample materials and equipment. They helped us build a custom PC and work with hardware like SLI Bridges and Spyder Color Calibrator. The Mattress Factory also provided us with utmost tech support while we installed on site. From using live mapping softwares to installing hardware, they helped us solve all our challenges very seamlessly.

What could have been better?

Team Composition:

Our team had 6 members with only 1 programmer, Hui. This led us to plan our scope very carefully and keep it small. As Hui was alone responsible to develop all interactions for the first room and map the videos in Unity for the second room we could not take up any complex programming challenges or design interactions that would overload her. Whereas we had 3 artists and a producer who also had an arts background. The last member was a game designer who took up any tech research, installation and documentation work to help the programmer. 

Reaching the Final Design:

After halves our process became slow as we started exploring new ideas to improve the guest experience. We wanted the first room to influence the experience of the second room. We spent a lot of time exploring how we could make this happen using technology like using physical sensors to detect the audience size to influence the intensity of projections in the second room or networking the PCs in the first and second room. It took us a few weeks to realize what we were looking for is way beyond the scope of our 15 weeks project with only 1 programmer. Though we developed a new feature of the following interaction it did not align well with the immersive experience in the second room and we decided to not use it. We were also informed mid way through the project that our client will not be able to provide us with the open files of her work which meant sus recreating all the animations for the second room which was an added overload on our timeline. 

Playtesting: 

As our exhibit is spread across two rooms with a specific guest flow designed based on the layout of the Mattress Factory’s annex Gallery, it was difficult for us to recreate the same experience in our project room due to shortage of space. We were able to install the first rooms exhibit in our project room but due to the small space we could not test for how intuitive our experience was as we had to guide the playtesters to move to the next space. For the second room’s projection mapping we used the Cave at ETC to test our projections but we had only 3 projectors installed at the cave whereas our final exhibit would have projections on 5 dimensions – left, right and center wall, the floor and ceiling.

Hardware Challenges:

We faced a lot of hardware challenges throughout the project. Our team lacked good knowledge of hardware but we were able to solve them with the help of the IT team at the ETC. 

1. Kinect – We started building for Kinect 2 when we realized that Kinect Azure was a better version to sync more than 2 Kinects together. Midway we had to spend 2 weeks to translate our codes from Kinect 2 to Kinect Azure. We also had to overcome a lot of challenges to sync the Kinects together and allow motion detection of large size crowds for a gallery space. 

2. Projectors – We had to build a custom PC with 2 graphic cards to support 5 4K projectors. We tried using an SLI Bridge to make the GPUs communicate which did not work out. It was also a challenge to match the color of projectors which we resolved by using a Spyder color calibrator. 

We resolved a lot of other problems like mapping videos in Unity, controlling motion detection for large size crowd, covering end to end walls with projections making sure the equipment did not interfere with the guest experience amongst others. 

Lessons Learned

Our team was composed of members with diverse backgrounds. Not necessarily everyone had experience with location based experiences. We learnt to understand our strengths and how they could be utilized in different mediums of entertainment. For example how a game designer can help improve the guest flow of a location based exhibit. All of us learned more about hardware, new softwares and developing pipelines. We started to let our team members do what they do best, which is create amazing things and not question their decisions. 

Post halves, we had this overwhelming fear that we might not deliver and fail at installing both the rooms within the given timeline. We then realized that failure was a viable option and we were not alone in making this work. With the help of our instructors we learned to plan and manage our scope better. Each of us also realized that there were 5 other team members that were trying their best to make this as best as it could be. We also started to enjoy the work again and we ended up with a project that was appreciated by our clients and fulfilled their expectations.

Conclusion

The exhibit will be showcased at the Mattress Factory’s Annex Gallery. The opening of the show will be in June 2022 and the show will run for a duration of 6 months. 

Categories:

Tags:

Comments are closed