Cyborg Control

Interactive Workshop Window Installation

In collaboration with Kati Hyyppä
Supported by Senatsverwaltung für Kultur und Europa
Berlin, 2021

A Pandemic Installation

Our workshop, which is a former shop space in eastern Berlin, has a large window facing the street. We're used to people peeking in, but during the coronavirus pandemic the window has become an even more important way to maintain a dialogue with the neighborhood. Last spring we displayed a hacked plotter for people to interact with. This spring the window was taken over by a remote-controllable toy cyborg with a little doll head. Over several weeks, a miniature world kept growing around the cyborg, so there was always something new to discover. People could walk the cyborg to different locations with their smartphones and let the cyborg interact with its theatrical environment by pushing all kinds of buttons and triggering sensors.

Working on the cyborg world

The cyborgian world, which eventually filled the entire window, transferred ongoing pandemic themes into an imaginary microcosm. It was constructed largely using random materials that we had at home. At “Salon 3000” the cyborg could get a hair treatment and at “Späti 51”, a Berlin-style convenience store, an alien would greet him. While the actual Berghain nightclub remained closed, it was possible to dance off in “Borghain” - after visiting a “Test Center”.

The installation in our workshop window

Adjunct to the club was the laboratory of “Biochip Corp.”, named after organic TV junk-food (“Bio” = “Organic” in German), but then again, it also looked like an evil corporation which is doing experiments with microchips in syringe-shaped incubators!? Fortunately, the laboratory had a solid authorization system and everytime the cyborg tried to enter the lab by typing a random access code into the keypad, the lab's alarm went off!

More cyborg world elements in the making

The world domination control console was located prominently in the center of the window. Its exact function remained unclear, but when the robot pushed its button, a cylindrical shaped world started to spin, clearly indicating that this shop window decoration is an imaginary world, which is not to be confused with our spherical earth. On top of the cabinet, a parabolic 5G party umbrella pointed towards a suspended toy plane. Every time the world domination button was pushed, the plane started to fly around, spreading funny, glittery trails.

One family even brought a laptop to control the cyborg (before Felix figured out why the interface always crashed on iPhones)

As you can probably imagine, the installation got quite some attention from the neighborhood. Many enjoyed seeing the world growing over time, kids wanted to play again and again with the cyborg and some people were curious about the cultural references and what exactly we wanted to express with that installation. It certainly caused some interesting conversations!

Decoration detail with micro-Illuminatus holding an electronic chip in his hands

What did the artists want to tell us with that?

Hacking Robosapien V 1.0

We were really lucky to rediscover an old Robosapien in our cellar, which we got from a neighbor a few years back, and not any other toy. Robosapien is a legendary robot, developed by NASA robotics physicist Mark W. Tilden in the early 2000s. It is really well designed and easy to modify. There even exist quite a few tutorials already for controlling it by an Arduino. The hack boils down to simulating the infrared signal of the original remote control. The robot is operated by sending commands as voltage pulses via the output pin of a microcontroller to the IR input of the Robosapien. Mark Craig has figured out all those commands and there's a complete list on his website.

Main board of Robosapien V1.0 with the “Head” connector on top

Hardware modification

We decided to go for a NodeMCU V1.0 (ESP12-E Module) as a controller. It has WiFi capability, it can run its own web server, it also works with 3.3V (as the controller of the Robosapien does), it can be easily programmed with the Arduino IDE and most important: it was in our random controllers drawer. The ESP is directly connected with the IR input of Robosapien's main board. That's the white wire of the “head”-connector. We have just cut that wire, extended the end that goes to the Robosapien main board and soldered it onto pin 3 of the NodeMCU. At the end, we also added a red Terminator-style LED in the eye of the cyborg that is toggled by NodeMCU's pin 1. And both, the robot and the ESP, are powered by an external PC power supply. The empty battery compartments are filled with stacks of big washers for additional weight, as otherwise the cyborg would tip over while walking.

An ESP-webserver beats in the chest of the hacked Robosapien as its new heart

Firmware

There are a few different codes on the ESP. First of all, there's the Arduino code which makes the ESP become a hotspot with a web server, but there is also the HTML/Javascript interface that's delivered to the smartphones of the people who control the cyborg. When people tap on the arrow-buttons on their phone screen, the vibration motor rumbles and the javascript sends a command via WebSocket protocol to the ESP. The Arduino code receives that signal and toggles the IR output pin accordingly.

Shooting the photo for the control interface

A great discovery while developing the public interface was the Arduino SPIFFS library. This “SPI Flash File System” lets you store all the web interface data (HTML, CSS, JS) as separate files on a simple filesystem in the ESP's flash memory. That's a lot more comfortable than baking everything into the Arduino code. If you plan to do something similar, you might want to have a look at the newer “LittleFS” library, though, as SPIFFS is already deprecated.

The control interface itself was just a photo with arrows made of cardboard

When hitting those arrows, the phone would rumble and send commands via WebSocket protocol to the ESP

Fails & Discoveries

It was astonishingly straightforward to modify the Robosapien so that it can be controlled publicly via a web interface. Cutting one wire and soldering it to the ESP was basically enough and everything seemed to work right away. But after a while, we discovered that iPhone users constantly failed to operate the cyborg. That was a bit hard to debug, as we don't have an iPhone. Thankfully our friend Felix, an iPhone and ESP user, offered to have a look at the problem. He discovered that toggling the phone's vibration motor via Javascript makes the interface crash. He added a few lines of code and now it also works for iPhones - but feeling the rumbling effect when pushing an arrow is a feature which only Android users can experience.

“Now for iPhone”

The instruction sign informed people about how to control the cyborg

Another problem occurred after several months of operation: Lately the robot often has problems to power up correctly. The same goes for the ESP. Sometimes it completely fails to provide a hotspot. It looks like the program on the controller wouldn't run at all, but from time to time everything works normally again. The problem is unclear. Is the cyborg reaching a state of consciousness and does it start to rebel? Or would it have simply been a good idea to put an optocoupler between the ESP and the Robosapien's main board? It's a mystical problem and as long as the cyborg doesn't try to kill us while it's supposed to be powered off, there's at least no reason for us to consult with an exorcist on that matter!

Thanks

While we made this installation, Kati and I received an art grant from the Berlin Senate (Senatsverwaltung für Kultur und Europa). It paid our bills and gave us the freedom to make this installation for the neighborhood. Thanks a lot for this support! Also big thanks again to Felix for figuring out the iPhone issue and to our neighbor Heike for donating the robot!

Additional resources

TAKE ME TO A RANDOM PROJECT!