This is me, my hair, my lip, my smell, the way my head moves and the body I live in. The sense of self is not fixed but continuously updates in response to the present information. When the body senses itself internally and localizes its actions, it provides the basis for a material sense of self existence. At the same time, the mind registers the sense of an agency with free will, thesense of being, the cause of voluntary action. The present self continuously becomes the past,and by the time we look into it we are in another present, consumed with planning the future.
Inwards to Outwards is my thesis work in MIT Media Lab that describes my theory and practice that concern the sense of self. Two projects, Being A Tree and Masque, are included as examples of altering the perception of self through multi-sensory stimulation of exteroceptive signals and false feedback of interoceptive signals. Through these two systems, I aim to provide the audience a new, though temporary, relationship with themselves.This thesis incorporates my technical contribution in human computer interaction as well as my artistic inquiry. The goal is to reconfigure the tools of technology, not for exploitation but for the recovery of human feelings, affects and emotions.
I hope the thesis delivers more than just the systems and study data. This thesis is about the moments of self-(re)organization and creating ripples in the fabric of self. It pays close attention to our shared psychological, emotional, cultural, and perceptual approaches to the inner and outer world and tries to bring light back to the sensitivity of self.
“How are you feeling?”
The answer could be either a mental state or a body condition: “I am so excited,”“my back hurts” or both: “I am too tired to enjoy the movie.” Thinking of ourselves, we examine both our body and mind. But where do we sit between body and mind? Where is the self? How does the biological machinery - body - establish the foundation of our consciousness, self-awareness and subjectivity?
When the body senses itself internally and localizes its actions, it provides the basis for a material sense of self existence. Interoception is a term that is used to include the sense of the physiological condition of the entire body and the ability of visceral afferent information to reach awareness and affect behavior. It connects the mind and internal body state, operates between the unconsciousness and consciousness.
A.D. Craig, a leading researcher in the area of interoception, argues that interoception is intimately linked to the sense of self and proposes a time-shifting progression of self image that is constantly updating. This allows us to adapt in the ever changing environment and recognize ourselves along the way. The image here is a cartoon illustrating the cinemascopic model of awareness based on time-shifting global emotional moments (top), which can explain subjective dilation of time (middle), and which provides a possible basis for subjectivity with a comparator buffer (bottom) that can be loaded with the present global emotional moment for comparison with any other from the past or future, but which is always one tick behind when compared with the present moment. (Craig A. D., The sentient self, 2010)
In Hand With Reflecting Sphere, a self-portrait by Dutch artist M.C Escher, the artist and his surrounding environment are trapped in the reflective sphere. He, along with us the viewers, looks into the curved reflection of himself and the studio he lived in. The reflection is distorted but recognizable, false but true, real and dreamlike. To register the image in the mirror as ourselves, we rely on the expected, stable visual contents and synchronized actions. The reflected image is bound to follow its original. But the reflected image is nonetheless dynamic, possibly turning in opposition to what it was in the past.
To further investigate the relationship between the internal self and a perceived self, I’d like to extend the concept of mirror, to the overall sensory inputs we collect from the outside. As a person makes a sound by either interacting with the surroundings (eg. tapping, walking) or producing it directly (eg. voice, respiration), the sound will be heard by the person again. These sensory feedback, both interoceptive and exteroceptive, guides us to construct the sense of self.
Masque is a psychoacoustic system that manipulates the user’s perception of their own respiration by providing false auditory feedback. By deliberately misrepresenting the wearer’s sense of their own breathing, Masque can cause bias in behavioral and cognitive experiences without any explicit instructions or stimuli. Users hear synchronized respiration sound from Masque as their own and react naturally to the synthetic body signals.
Masque uses a digital temperature sensor chip for its fast data acquisition and high resolution (0.1°C) to sense respiration. The sensor measures the temperature right in front of the nostril at a frequency of 43Hz to detect the exhaling and inhaling activity of the user. With high precision and fast speed temperature sensor, Masque detects the user's breathing activities and plays back a mediated breathing sound synchronously through a bone conduction headphone. The mediated breathing sound is real-time synthesized, thus its breathing rate can be modified by the user at any moment.
How could we bring people to truly empathize with the natural world, facing devastating but ever distant problems such as glaciers melting in the Arctic, water shortages in California, or the deforestation of the Amazon? What would change if an audience could be transported into the center of the story— if one could step into the shoes of another living being?
TreeSense is sensory VR system that transforms you into another life form, a tree.
With arms turning into branches and the body into a trunk, the participant experiences being a seedling rising through the dirt, sprouting branches and growing to full size, until finally witnessing its fate firsthand. By tapping into various senses, TreeSense delivers a vivid body illusion of being another identity to create a personal connection with it. Our research looks into novel sensations such as electronic muscle stimulation, as well as a comprehensive sensory alteration through visual, sound, scent, vibration, temperature and wind, which unlocks a higher level of realism. As an audience member exclaimed, “you know it’s not real but your body really believes it!”
Can tattoos embrace technology in order to make the skin interactive?
The DermalAbyss project was the result of a collaboration between MIT researchers Katia Vega, Xin Liu, Viirj Kan, and Nick Barry, and Harvard Medical School researchers Ali Yetisen and Nan Jiang.
DermalAbyss was a proof-of-concept that presented a novel approach to bio-interfaces in which the body surface would be rendered as an interactive display. By replacing traditional tattoo inks with biosensors whose colors change in response to variations in the interstitial fluid, it blended advances in biotechnology with traditional methods in tattoo artistry. The biosensors create a direct access to the compartments in the body and reflects inner metabolic processes in a shape of a tattoo. It could be used for applications in continuously monitoring such as medical diagnostics, quantified self, and data encoding in the body.
Publication: Katia Vega, Nan Jiang, Xin LIU, Viirj Kan, Nicolas Barry, Ali Yetisen, Pattie Maes and Joe Paradiso. "The Dermal Abyss: Interfacing with the Skin by Tattooing Biosensors" In Proceedings of the 2017 ACM International Symposium on Wearable Computers. ACM, 2017.