Danlin Dany Huang
Interactive media foralternative soma experience.
Body Oracle
Danlin Huang, Ke Huang
Key Words
Concept
In today's information age, we predominantly communicate through text—messages, emails, and web pages. According to Mehrabian's 7-38-55 rule , spoken words convey only 7\% of meaning, while tone of voice comprises 38% and body language accounts for 55%. Though emojis help express emotion, digital communication remains fundamentally "disembodied" , lacking the immediate bodily cues essential for deep human connection. This heavy reliance on online exchanges weakens our ability to express and interpret body language, resulting in misunderstandings, communication breakdowns, decreased empathy, and diminished social support and well-being.
Our linguistic limitations compound this issue: both Western and Eastern vocabularies lack precise terminology for proprioception and social proprioception—our sense of body positioning in space, both individual and collective. This linguistic gap limits our ability to articulate and understand crucial concepts of embodied nonverbal interaction.
In response, we propose "Body Oracle," a computational linguistic speculative design project. We envision an "alternative present" with an alternative language system capable of describing complex proprioceptions. Inspired by Chinese oracle bone inscriptions (OBI), one of the oldest forms of Chinese writing, we draw upon their elegant pictographic nature where simplified strokes mirror the objects they represent. Through strokes that sketch bodily positions, we capture spatial relationships in their most intuitive form. Based on this inspiration, "Body Oracle" introduces AI-generated hieroglyphic characters that represent corporeal and intercorporeal poses, mapping the spatial relationships between bodies. These intuitive, cross-linguistic, cross-cultural signs—similar to emojis—may expand our ability to conceptualize, understand, and communicate bodily experiences.
Body Oracle Caster
Body Oracle Caster is an interactive AI-generation tool. It features a screen and a camera that immediately transforms user’ movements into AI-generated Body Oracle characters. These characters mimic the pictorial style of ancient Oracle Bone Inscriptions (OBI) while reflecting each user’s unique motion.
To construct a comprehensive dataset, volunteers were instructed to replicate a series of poses derived from authentic OBI characters, capturing both individual and collaborative postures. The system employs a multi-step process underpinned by three core components:
-
Pose Estimation: The camera feed is processed through MoveNet—a TensorFlow Lite model that facilitates real-time pose estimation. Once the system determines that a user is maintaining a stable pose (i.e., minimal keypoint movement), the extracted skeleton data proceed to the next stage.
-
Initial Rendering (pix2pix cGAN): A conditional generative adversarial network (cGAN) based on the pix2pix framework is then applied to the 2D skeletal arrangement. This model was trained on paired pose–character data, enabling it to transform skeletal configurations into coarse, hand-drawn visuals suggestive of OBI glyphs.
-
Style Refinement (Neural Style Transfer): The pix2pix output is further refined using a neural style transfer network. This module incorporates oracle-inspired stylistic elements, adjusting line weight, curvature, and brush-like texture to enhance the resemblance to ancient script. The resulting pictograms retain anatomically relevant features while aligning with canonical OBI aesthetics.
Neural network model dataset
Body Oracle Archive
Body Oracle Archive is a continually expanding collection of somatic hieroglyphic characters. When the Caster detects a pose that does not match any existing character in the Archive, it employs cGANs and Neural Style Transfer to create a new symbol that adheres to OBI aesthetics. A large language model (GPT4o) then assigns a corresponding pronunciation based on ancient Chinese phonetics. Over several months, we collected extensive pose data from both public participants and professional dancers, yielding a comprehensive movement dictionary that links each character to its pose and pronunciation.
Body Oracle Translator:
Body Oracle Translator is a handheld mixed reality (MR) device that helps audiences experience the Body Oracle language system. It recognizes and analyzes body poses in real time, then references the Archive to display the corresponding Body Oracle characters while providing audible pronunciation of the recognized characters.
We created a binocular-style, oracle bone–inspired MR mask to encase the HoloKit, a smartphone-based, open-source, optical see-through stereoscopic headset. The translator’s shell is made from a ready-made oracle bone. An iPhone is mounted inside, and the mask design leaves the phone’s camera unobstructed, enabling environment sensing while preserving the headset’s optical see-through view for users.
When a user holds this MR translator up to a human body, a fine-tuned YOLO11 Pose Estimation\footnote model (running on the iPhone) identifies the individual’s pose in real time and compares it against previously generated Body Oracle characters in the Archive. By pressing the button at the top of the phone’s screen, the device audibly pronounces the recognized character. The resulting character and its pronunciation are displayed at the center of the translator.
Audiences are experiencing Body Oracle Translator in TEI’25 Art and Performance Exhibition
future archaeology
We present this speculative design as a "future archaeology" experiential exhibition that traces the evolution of these hieroglyphs and envisions a world where proprioception forms the foundation of linguistic expression. Through this immersive "alternative present," audiences are encouraged to expand their bodily awareness. The experience creates an illusion that the "Body Oracle" language has existed for thousands of years—deeply embedded in our culture—inhabiting a world transformed by bodily communication.
Our contributions are threefold.
Revitalizing Ancient Heritage: We draw on the wisdom of ancient Chinese Oracle Bone Inscriptions, adapting their pictographic simplicity through AI generation to create an intuitive, cross-cultural language that fosters collective bodily awareness.
Experiential Alternative Present: We combine soma design and speculative design to immerse audiences in an interactive environment, prompting them to explore how a body-centered language system might alter their perception of self and others.
Pathway to Somaesthetic Futures: Our audience feedback, gathered from exhibition participants, suggests that engaging with Body Oracle enhances somatic awareness and bodily conceptualization. This points toward a future where these somatic characters could complement traditional text-based communication to foster collective bodily awareness.
As Wittgenstein famously asserted, "The limits of my language mean the limits of my world." Drawing on the principle of linguistic relativity, we contend that introducing these alternative somatic hieroglyphic signs may reshape our collective semantic cognition of bodily awareness. By acting as a sociolinguistic mediator, Body Oracle serves as a "true oracle" that can help us conceptualize, describe, and ultimately transform our social reality through a reimagined embodiment of language.