Making Interactive Art

This summer, we built an interactive art installation in the middle of a college campus — a journey designing, manufacturing, and installing Abilene Christian University’s Lightwalk. Now that it’s complete, and since the opportunity came by way of sharing knowledge, I thought I would do the same here and pause to reflect on our process and lessons learned along the way.

Vision

The vision for the Lightwalk installation at Abilene was nearly two years in the making when we first had a conversation with their team. In that time, a good amount of consideration had already been given to various aspects of the installation, including a concerted effort from Abilene to prototype their vision and actually bury it in the ground.

We knew the installation would be located below-grade on the East side of a jagged concrete path and consist of many “reeds” or light poles that would illuminate. The twist to all our previous work was that it would also have to be hackable so students could continue to improve the installation over time. In all frankness, it would be a one-of-a-kind installation that couldn't easily borrow from stock components — it was custom art on a massive scale.

Constraints

We visited the site and kicked the project off on-site in Abilene, Texas. We settled on a rough vision, discussed high-level functionality and settled on an installation date which was exactly 11 weeks later (avoiding the worst of a Texas summer, but still a relatively quick turnaround all things considered). Here were the project constraints as we understood them at the time:

Primary aspects:

1. A hardware layer you can see and not hesitate to touch.
2. Firmware that coordinates and animates the entire installation.
3. Fleet management for health monitoring and code GUI for students.
4. A responsive web app for pedestrians to control the installation from.

In addition it needed to be:

1. Striking. It needs to look stellar both day and night and from all angles.
2. Durable. It needs to survive life outdoors on a college campus.
3. Flexible. Literally. It needs to survive everything from curious toddlers to frisbee accidents to go-carts.
4. Interactive. It needs to respond to the movement of pedestrians as they walk past.
5. Synchronized. It needs to coordinate complex effects without appearing visually out-of-sync.
6. Hackable. It needs to be a platform that enables students to write their own effects.
7. Mobile. It needs a mobile website where visitors can request their own effects in real-time.
8. Theft proof. It will look like 350 lightsabers sticking out of the ground… temptation for college pranksters.

Design for Installation

We knew the real push would come in the final days and hours when on-site logistics meshed with last-minute testing. They say “no plan survives first contact with implementation”, and I’d agree. This could’ve created a real pressure cooker-type situation. So, hoping to avoid that crisis, we designed our components and software with an installation-first mindset that would also make ongoing maintenance easier. This began with a distributed system architecture.

In this architecture, we provided power and data to a chain of five nodes (computer “brains” boxed in the ground). Each node in turn controlled ten reeds. Added up, this worked out to 7 chains, 35 nodes, and 350 reeds. All of the nodes were exactly the same, and all of the reeds were similar because they only differed in length (different lengths between 2 - 4ft). We anticipated manufacturing problems would result in a 5-10% failure rate at the assembly level, so we built 40 nodes and ~400 reeds and connected everything together with about 70 custom-molded cables. As a result, installation was as simple as putting together about a dozen different parts and swapping out those that didn’t work. 

The entire installation (35 nodes) will boot and vote to determine which of the nodes should become master. This master node effectively becomes the spokesperson for the entire installation. Typically, the lowest node (by unique identifier) becomes master. But imagine something mechanical happens to the installation— a golf-cart drives through it… or the sprinkler guys take an excavator through the middle. If there is ever a physical break in the communication bus, effectively splitting the installation into two or more smaller segments, each isolated segment will immediately notice the change and vote for a new master. After voting these master nodes then attempt to connect over wifi to our cloud service.

When a visitor queues an effect for the installation this cloud service relays that command to all master nodes. Those master nodes then relay the command to slave nodes along the bus and thereby coordinate when to begin the effect. This all happens very quickly and provides a reliable means for ensuring the installation remains functional even when the unexpected takes place.

Substitute Hardware Brawn with Software Wit

Lightwalk has two primary features: lights and interactivity. This is obviously an over-simplification, but these were the two features that ultimately needed to be most closely coupled together. A third feature, a tempering omnipresent feature that became a constant design constraint was a notion of robustness. This feature dealt with the physical, the tangible. What we termed the hardware layer needed to withstand brutal environmental conditions outside in Texas for two years. As a result, our challenge was to balance the need for the installation to be both functional as well as bombproof. Our approach was to simplify the hardware and invest heavily in smart software. This enabled our hardware team to focus on designing and sourcing fewer (but more reliable) components and our software team to focus on building robust software for a redundant and distributed architecture. Here are two examples where software wit simplified the hardware layer and improved overall robustness:

Robust Comms

Lighwalk leveraged a common CANBus across all nodes for communication. We wrote a simple protocol above the CANBus protocol to abstract our message routine, enable nodes to work together, and ultimately display complex effects at >120Hz. Take that, Javascript in the browser! Further, this protocol allows nodes to broadcast motion interactions (when a person walks past) to all nodes. This enables installation gamification -- where intelligent effects triggered by motion dynamically respond to motion happening anywhere, or everywhere, along the installation.

Smallest (#winning) Motion Sensor

We had a few options when it came to detecting motion, and we considered everything from machine vision to sonic sensors. We found that most of the off-the-shelf solutions for detecting motion were fragile, large, and obvious. However, we wanted a form factor that was small and discreet… something robust we could integrate into the installation itself. This was a tall order when considering the small diameter of a single reed. Consequently we looked at different sensors as well as separating the sensor from software-level motion-processing and motion-triggering tasks which normally take place on volume-greedy silicon. This ultimately led to our final approach which tightly coupled software with hardware by keeping the sensor above ground and processing below ground. We leveraged very small PIR sensors that were incorporated into the reeds themselves and processed raw sensor data with firmware on the connected node so we had total control over determining when there was motion and when to broadcast motion interactions.

Some Assembly Required

But engineering alone couldn’t bring something like this to life without a concerted manufacturing and QA effort. The final mile, or in our case the entire second month of the project, was largely dedicated to building and testing reeds and nodes. This was as much a manufacturing effort as it was a relationship building effort. I’ll explain.

The Multiplier Problem

A challenge we faced was the sheer scale of the installation. Everything that needed to be accomplished either needed to be done once, 40 times, or 400 times. And the vast majority of tasks lived in those final two buckets. Because we were up against hard deadlines and strict performance criteria we needed to simultaneously tackle design, sourcing, and assembly all at once and all in-step with one another. This meant bringing on trusted partners we could collaborate with during design so we could most fully leverage their expertise and capacity during assembly.

Tight Feedback Loops

The things we looked for in partners were the same things you might consider in any important relationship. Fundamentally we needed to both speak a common language, have quick access to decision makers, and be able to dutifully execute against a schedule. We’ve found that this often looks like the partner that may not be the most well known but is the hardest working. This work ethic, or mutual commitment to the project, enabled our teams to work closely together and quickly iterate on solutions until they were both performant and manufacturable.

Reed Assembly

The process of manufacturing reeds was one example. One partner provided reeds which began as two individual LED boards put back-to-back and soldered to a flexible IP-68 rated custom cable assembly from another partner. This subassembly then mated with a custom turned aluminum cap from another partner. Viget added a diffusion tube which created the functional lighted assembly we could test and provide final QA on. The resulting reeds could be mass-produced, were optically awesome, weatherproof, UV resistant, and we hoped college proof.

Final Thoughts

Creating large-scale interactive art is itself an art. I characterize it as a balance between creative design and practical engineering. It’s a collaboration of the two disciplines so that they meet somewhere in the middle and only really compromise on the truly unreasonable. Yes, this sets the bar high and keeps the aim on a truly inspiring and immersive experience. Yes, this makes it more difficult. But, in my opinion, interactive art should both capture the imagination of the visitors it hopes to wow as well as the engineers that bring it to life. Artists, software engineers, hardware engineers, and supply chain folk alike all enjoy a unifying challenge. And, we had the absolute pleasure of collaborating with Abilene Christian University while acting as their trusted partner to craft an interactive experience  we hope their visitors will enjoy for a long time to come.