HCI Deep Dives

How can blind users navigate and understand virtual environments? This paper compares two haptic approaches for conveying where virtual objects are located: vibrotactile feedback (buzzing sensations) versus skin-stretch feedback (gentle pulling on the skin). The researchers tested both methods with blind participants to see which better communicated the position and movement of objects in VR space...

When you hit a baseball hard, you don't just feel the moment of impact—there's a lingering tingle that follows. This paper explores how to recreate that aftereffect in VR to make impacts feel more realistic. The researchers built a VR baseball bat that delivers not just the initial hit, but also vibrotactile and thermal feedback that simulates the tingling sensation after a strong swing. In a stud...

How hard is your brain working in VR? This paper presents a machine learning system that predicts cognitive load in real time using behavioral and physiological signals collected through VR headsets. Trained on data from 738 participants across various VR experiences, the model achieves remarkably low error (MAE of 0.11) in predicting mental workload. The system analyzes markers like eye movements...

Watching a dance performance engages your eyes and ears—but what if you could also feel the dancer's movements? This paper presents algorithms that automatically convert motion capture data from human dance into haptic feedback, letting viewers physically experience the performer's movements through vibrotactile devices. The approach cleverly separates external movements (the dancer traveling thro...

Taking a break at work should leave you refreshed—but often it doesn't. MindSpace is a pneumatically controlled haptic device worn on the upper chest that combines two calming sensations: gentle breathing-like movements and deep touch pressure similar to a hug. In a study with 24 participants performing demanding cognitive tasks, those who used MindSpace during their breaks showed significantly fa...

What if your clothes could power your wearable devices? texTENG introduces a maker-friendly framework for creating textile-based triboelectric nanogenerators (TENGs)—fabrics that harvest energy from everyday human motion like walking, arm swings, or even breathing. While TENGs have shown promise for sustainable wearable power, adoption has been slow due to hard-to-find materials and complex fabric...

Micro-expressions are fleeting facial movements lasting just 40-200 milliseconds that are believed to reveal concealed emotions. But can these subtle expressions actually be measured objectively? This study provides the first direct electromyographic (EMG) evidence that micro-expressions are real, quantifiable muscle activations—not just visual artifacts. By placing electrodes on participants' fac...

Music has long been known to evoke powerful emotions, but can machines truly understand and predict these emotional responses? This survey paper takes stock of the field of music emotion recognition (MER), examining the datasets, computational models, and persistent challenges that shape this research area. The authors review how emotion is represented—from categorical labels to dimensional models...

Virtual body extensions like wings or tails offer exciting new experiences in VR, but using them naturally—especially parts you can't see, like a tail—requires proprioceptive feedback to sense position and force without relying on vision. This paper introduces the "Imaginary Joint," a novel approach that uses skin-stretch feedback at the interface between your body and a virtual extension. A weara...

Electro-tactile interfaces—which deliver tactile sensations through electrical stimulation of skin nerves—offer unique advantages like fast response times, thin flexible form factors, and the ability to simulate textures, softness, and even coldness. But designing them has been notoriously difficult, requiring deep electronics expertise and custom hardware. eTactileKit changes that. This open-sour...

What happens when you watch yourself perform in VR—and then have to critique that performance? This study explored self-reflection in motor skill learning using a Karate training task. Participants were embodied as a "trainer" avatar and asked to give verbal feedback on a "trainee"—which was either their own 3D-scanned appearance or a stranger, performing either their own recorded movements or an...

Can fake heartbeat sounds trick your body into relaxing? This system generates pseudo-heartbeat audio at rates slightly slower than your actual heart rate to induce calm and support meditation. Using a contactless radar sensor to detect real heartbeats, it creates slower auditory feedback (10-30% below actual BPM). Tested with 120 participants at SIGGRAPH Asia 2024, results showed that hearing slo...

Learning to identify musical pitch intervals usually requires tedious rote practice. Purrfect Pitch offers a new approach: a wearable haptic vest that translates sound into touch. When users hear two musical notes, they simultaneously feel vibrations at corresponding vertical positions on their back—leveraging our natural "high/low" pitch metaphor. In a study with 18 participants, those using the...

Processing high-resolution video with AI requires massive computational resources. GazeLLM offers an elegant solution inspired by human vision: use eye-tracking to focus only on what matters. By cropping first-person video to a small region around the user's gaze point, the system reduces pixel input to just one-tenth while achieving task comprehension equal to or better than full-resolution video...

Traditional quadrotor drones pose safety concerns with their spinning blades. Cuddle-Fish takes a different approach: a helium-filled soft robot with bio-inspired flapping wings that's safe enough to touch, hug, and interact with physically. In testing with 24 participants, people spontaneously engaged in affective behaviors like patting, stroking, and even hugging the robot. Users reported positi...

Our bodies experience a wide variety of kinesthetic forces as we go about our daily lives, including the weight of held objects, contact with surfaces, gravitational loads, and acceleration and centripetal forces while driving, to name just a few. These forces are crucial to realism, yet simply cannot be rendered with today’s consumer haptic suits, which primarily rely on arrays of vibration actua...

Xiaru Meng, Yulan Ju, Christopher Changmok Kim, Yan He, Giulia Barbareschi, Kouta Minamizawa, Kai Kunze, and Matthias Hoppe. 2025. A Placebo Concert: The Placebo Effect for Visualization of Physiological Audience Data during Experience Recreation in Virtual Reality. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI '25). Association for Computing Machinery, New Y...

Perceiving and altering the sensation of internal physiological states, such as heartbeats, is key for biofeedback and interoception. Yet, wearable devices used for this purpose can feel intrusive and typically fail to deliver stimuli aligned with the heart’s location in the chest. To address this, we introduce Heartbeat Resonance, which uses low-frequency sound waves to create non-contact haptic...

To enhance focused eating and dining socialization, previous Human-Food Interaction research has indicated that external devices can support these dining objectives and immersion. However, methods that focus on the food itself and the diners themselves have remained underdeveloped. In this study, we integrated biofeedback with food, utilizing diners’ heart rates as a source of the food’s appearanc...

When several individuals collaborate on a shared task, their brain activities often synchronize. This phenomenon, known as Inter-brain Synchronization (IBS), is notable for inducing prosocial outcomes such as enhanced interpersonal feelings, including closeness, trust, empathy, and more. Further strengthening the IBS with the aid of external feedback would be beneficial for scenarios where those p...

Yulan Ju, Xiaru Meng, Harunobu Taguchi, Tamil Selvan Gunasekaran, Matthias Hoppe, Hironori Ishikawa, Yoshihiro Tanaka, Yun Suen Pai, and Kouta Minamizawa. 2025. Haptic Empathy: Investigating Individual Differences in Affective Haptic Communications. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI '25). Association for Computing Machinery, New York, NY, USA, Art...

Riku Kitamura, Kenji Yamada, Takumi Yamamoto, and Yuta Sugiura. 2025. Ambient Display Utilizing Anisotropy of Tatami. In Proceedings of the Nineteenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '25). Association for Computing Machinery, New York, NY, USA, Article 3, 1–15. https://doi.org/10.1145/3689050.3704924   Recently, digital displays such as liquid crystal...

Hu, Yuhan, Peide Huang, Mouli Sivapurapu, and Jian Zhang. "ELEGNT: Expressive and Functional Movement Design for Non-anthropomorphic Robot." arXiv preprint arXiv:2501.12493(2025). https://arxiv.org/abs/2501.12493 Nonverbal behaviors such as posture, gestures, and gaze are essential for conveying internal states, both consciously and unconsciously, in human interaction. For robots to interact more...

K. Brandstätter, B. J. Congdon and A. Steed, "Do you read me? (E)motion Legibility of Virtual Reality Character Representations," 2024 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Bellevue, WA, USA, 2024, pp. 299-308, doi: 10.1109/ISMAR62088.2024.00044.   We compared the body movements of five virtual reality (VR) avatar representations in a user study (N=53) to ascertain h...

The Oscar best picture winning movie CODA has helped introduce Deaf culture to many in the hearing community. The capital "D" in Deaf is used when referring to the Deaf culture, whereas small "d" deaf refers to the medical condition. In the Deaf community, sign language is used to communicate, and sign has a rich history in film, the arts, and education. Learning about the Deaf culture in the Unit...