Process Documentation

Taction is the result of many iterations, conflicts and problem solving decisions. Its development process highly influenced by three crucial factors that helped us define the concept and realize a finalized outcome. These three factors are:

  1. First presentation (proposal) crits
  2. Observing our peers interact with the object
  3. Technical difficulties (the many, many of them)

Our initial concept consisted of a multi-modal form of interaction, in which multiple agents would interact through the cushion-like objects as a complex ecosystem. This early concept would allow music to influence the behavior of the cushions, while at the same time making the cushions reactive to participants and to each other. This interaction modality posed a number of challenges and UX design oversights, so after presenting the concept to the class and meeting with Elio and Sabine to discuss the feasibility of the project, we arrived at the conclusion of reducing the modality of interaction. This reduced interaction mode maintained the influence of music on the cushions in terms of vibration through surface transducers, as well as enabling the visual behavior (light) of the cushions be affected by the participants through pressure sensors. Therefore, the cushions could be seen as becoming alive through music and interaction with humans, creating a two-dimensional interactive mode. A major decision that came when deciding to reduce the modality was to abandon the communication between cushions in order to prevent an over-saturation of behavioral and interaction.

For our first prototype, we decided to maintain a simple aesthetic in terms of the look of the cushion, just to simply get an idea in terms of size, feel and space for circuitry; however, while showing this faceless cushion to our peers in the classroom, we noticed that everyone related to the object in different ways. The most common response people had to the object was that of a pet-like connection, similar to a cat or a dog. We believed this was a powerful response and thought we could leverage from it by maintaining a formless outlook that allowed a free interpretation, while at the same time providing the it with very life-like characteristics, such as a soft exterior and added weight inside the object to give a life-like sense of weight.

Our limitations were mainly within the range of budget and skill, but in the course of overcoming them many design and technological decisions were made which shaped our final result. One of our first struggles came when dealing with the FFT (Fast-Fourier Transformation) of our sound input. The library that exists is not very well documented, at least in terms of giving less experienced users the opportunity to understand how to extract important data from it. Our intention was to get separate frequency values that we could send to the different pillows, so that the pillows would vibrate at the frequency of the music being played. Unfortunately, this was not possible, as we could not find a way to obtain valuable frequency measures from the library. Instead, we were able to retrieve the amplitude of different frequencies and use the FFT bins to send this amplitude to the different cushions. From the receiving end, a pre-programmed frequency would play at different volumes depending on the amplitude-values received, which in turn would make the cushions vibrate according to sound levels. In terms of making the surface transducers vibrate without making aesthetically displeasing sounds, many research expeditions were set into motion and ultimately found their way into using a DAC (digital to analog converter) and a small amplifier to convert an encoded sound file into low frequency sounds for bassy vibrations. Much more could be said about the troubles encountered with using the amp and the DAC together as a way to control the volume, but for the sake of simplicity, we can say that a wrongly used MOSFET was the way to success.

Another important feat was succeeding with a broken serial communication, product of the highly praised FFT library. This programming challenge was an almost unsolvable puzzle attempted by great minds such as that of Elio Bidinost and Sabine Rosenberg, and finally resolved by a valuable member of our team, Sophia Lim. See our video documentation for more information on this topic.

In case there are questions regarding why there are two Arduinos inside the cushions, the answer is as simple as: Arduino’s lack of multi-tasking power. Once all of the separate working pieces were put together, everything broke. The transducer was barely vibrating, and the LEDs triggered by our home-made pressure sensors were barely lighting. Our first instinct was to remedy this by adding power through external batteries to the transducer and the lights, but the problem persisted. We soon discovered this was a problem of memory and processing power. Thus, we resorted to making the containing box of the circuitry large enough to house two Arduinos with their respective batteries inside. This was our not-very resourceful approach, but it worked for the time being.

Due to the sequential nature of serial communication (where only one receiver could receive data at a time), the delays from the libraries and the amount of components in the circuit, a disruption in the playback of sound (vibration) occurred. This issue crept up on us last minute. Previous to the presentation day, we had successfully confirmed that the data being sent from the Teensy controller to each of the Arduinos was coming in a proper rate. We conducted a test with every one of the transducers and everything worked fine; however, when connecting all of the transducers to all of the Arduinos, so that they would all be playing at once, everything broke again. We attempted to remedy this issue during the week post-final presentations, and the only way we could superficially patch this up, was by preventing the Arduinos from receiving constant data, and playing each of their frequencies at a given amplitude for at least 2 to 3 seconds, so that a processing and communication overload would not occur.