Can a VR recreation of a concert make you feel like you were actually there? We explore this question in our latest paper, presented at the Augmented Humans conference 2026 in Okinawa, and the answer surprised us: less realism actually works better.
What happens when the line between a human performer and a machine blurs on stage? We are presenting this work at the Augmented Humans conference 2026 in Okinawa, and it is one of those projects that keeps surprising us.
The setup: an artist wears sensors and controls a robotic arm, turning their own body into a live audio-visual mixer. Their movements, their physiological signals, all of it feeds into the sound and visuals in real time. The performer becomes something in between – not quite human soloist, not quite machine, but a hybrid that makes you question who or what is actually creating the art.
Virtual reality promises to transport us to events we could never attend in person. Concert recreations in VR have become increasingly sophisticated, aiming to capture not just the sights and sounds of a performance, but the atmosphere created by a live audience. But what happens when we try to recreate that collective energy using physiological data from the original audience? And does it matter if that data is real or fake?
Flying robots are becoming more and more common. Yet, they also come with safety concerns. Quadrocopter drones, the most popular type, have fast-spinning propellers. Their rigid structures and loud noise levels make them unsuitable for close-range human interaction. Yet, can we design flying robots that people can safely touch and interact with?
Our recent work, lead by Mingyang Xu, introduces Cuddle-Fish, a soft floating robot that takes a different approach to aerial robotics. Instead of using propellers, the robot combines a helium-filled soft body with low-frequency flapping wings inspired by animal movement. It’s also the first of its kind that fits in normal indoor environments (like appartments and homes). Its design eliminates the danger of spinning blades while maintaining the ability to move through indoor spaces.
I’ve been experimenting with the Audio Overview of Google’s NotebookLM to create a podcast about Human-Computer Interaction (HCI), and I have to say, it’s surprisingly useful! Using their Audio Overview feature, I’ve uploaded several HCI publications, and NotebookLM has turned them into conversations.
Additionally, I also summarized most of my publications. Have a listen.
Google just announced that they allow to customize the Audio Overviews. I’m pretty excited about this feature, as the standard summarization is a bit too casual for me. Here is an example from the podcast without customization:
A research concept from 2016 might be in the spotlight in the next years, as described and discussed in the Dagstuhl Seminar “Eyewear Computing – Augmenting the Human with Head-mounted Wearable Assistants”.
Frisson is a feeling as well as an experience of physical reactions such as shivers, tingling skin, and goosebumps. In this work, we propose Frisson Waves, an exploratory real-time system to detect, trigger and share frisson in a wave-like pattern over audience members during music performances. The system consists of a physiological sensing wristband for detecting frisson and a thermo-haptic neckband for inducing frisson.
Very happy to be invited as one of the panelists presenting in a three-day online symposium on Liveness organised by the Neurolive EU Project. The symposium brings together artists, humanities scholars and cognitive neuroscientists to explore how liveness is conceptualised, measured and practiced across the arts and the sciences.
I’m speaking on the 24th, here’s the abstract of my talk and my slides (carefull ~17 Mb pdf)
Abstract: In my research, I combine design and technology to augment human senses, looking for novel interaction paradigms. Engineering and computing have often been used to mimic or surpass some human abilities (for example autonomous driving, playing Go). Such efforts appear to put humans and computers in a competitive relationship, as emphasized in AI vs. Human game competitions. Once fantastic fear of AIs “replacing” human workers is now taken much more seriously and discussed in the public sphere. My research proposes a different approach to the human-computer relationship by applying a cooperative and empowering framework, using wearable computing to actively augment human capabilities.
The role of the audience in stage performances is changing from passive spectators to contributors making the performance interactive by using different techniques. In this talk, we investigate the connection of audience physiological data to the experienced performance in three performance events with a total of 98 participants. We identified memorable performance moments by assessing, Electrodermal Activity (EDA) showing that the audience’s responses match the choreographer’s intention. Through Heart Rate Variability (HRV) features related to parasympathetic activity, we identified dramatic shifts that are connected to the choreographic development of the performance. Our results show how the audience’s physiological responses are linked to the choreographic development of the performance. Based on the findings, we contribute a discussion of the registered physiological phenomena and implications of the audience’s responses analysis to performance and choreography design in general. Furthermore, we walk through the dataset collected from the performance events.
On 16-17 March 2020, we co-organized the Augmented Humans conference in Kaiserslautern.. Due to the circumstances related to COVID-19, we moved to a fully virtual event and cancelled the physical event on 13th March. The whole team was struggling and especially thanks to all of the chairs it came together quite well.
On Sunday, just we met (only 2 people, disinfection materials, social distancing and mask wearing in place) to pre-record Stelarc’s Keynote
Abstraction Beats Realism: Physiological Visualizations Enhance Arousal Synchrony in VR Concert Recreations. Meng, Xiaru and Ju, Yulan and Hoppe, Matthias and Han, Jiawen and He, Yan and Minamizawa, Kouta and Kunze, Kai. The Augmented Humans International Conference 2026 (AHs 2026), March 16–19, 2026, Okinawa, Japan. 2026. Bibtex.
Embodied Responses to Posthuman Performance: A Mixed-Methods Study of Physiological and Emotional Audience Reactions. Kunze, Kai and Xu, Mingyang and Peng, Danyang and Ogasawara de Oliveira, Lucas and Shao, Rose and Meng, Xiaru and Hoppe, Matthias and Barbareschi, Giulia. The Augmented Humans International Conference 2026 (AHs 2026), March 16–19, 2026, Okinawa, Japan. 2026. Bibtex.
Philipp
Lepold
Karlsruhe Institute of Technology, Karlsruhe, Germany
,Tobias
Röddiger
Karlsruhe Institute of Technology, Karlsruhe, Germany
,Tobias
King
Karlsruhe Institute of Technology, Karlsruhe, Germany
,
Kai
Kunze
Keio Media Design, Yokohama, Japan
My overall goal: I want to give people tools to foster inert talent and empower new skills.
You can find up-to-date works on my publication list
Evaluation of Facial Expression Recognition by A Smart Eyewear for Facial Direction Changes, Repeatability and Positional Drift. Masai, Katsutoshi and Kunze, Kai and Sugiura, Yuta and Ogata, Masa and Suzuki, Katsuhiro and Nakamura, Fumihiko and Shimamura, Sho and Inami, Masahiko and Sugimoto, Maki. To be published in ACM Transactions on Interactive Intelligent Systems (TiiS). 2017. Bibtex.
A collection of iOS apps I’ve developed over the years.
Numeriqo
Number puzzle game with math cages. Fill grids so each row and column has unique numbers while solving arithmetic constraints. Sizes from 3x3 to 5x5.
Landing Page · View on the App Store
Numeriqo Pro
Advanced number puzzle game with grids up to 9x9 and additional features.
‘Blanks’ is a simple application helping you to learn new English words. It shows you word definition and gives 4 choices as answer. Select one of the choices by dragging it on the hole in the paper. If you are correct, a green hock is displayed. If the selected word does not fit to the definition, you will see a red cross.
The word definitions are from Wordnet. The application developer is not associated in any way with Wordnet.
Kai Kunze is a pioneer researcher in the HCI field, augmenting humans using technology. His most significant research contributions are in Eyewear Computing and Placement Robust Activity Recognition. His current research includes also Digitalizing Human Emotions and Amplifying Human Senses. Kai Kunze published over 254 papers at high profile conferences and journals (e.g. CHI, TOCHI, UIST, IEEE Computer)1. His publications are well-cited, with a total citation count of 4868 (H-index of 40), according to Google Scholar. He is actively involved in the research community, most notably as PC Co-Chair for TEI 2017, and Local Co-Chair of CHI 2021.
On the 9th August I had the opportunity to take part in a Miraikan Science Quest. It’s an open event at Miraikan, the Science Museum in Tokyo, to encourage a dialog between the public and researchers. This Science Quest was a premier, as it was the first ever held in English ;)
My second Siggraph was pretty amazing again. Unfortunately, we had another emerging technology exhibit, so my time joining the main conference was limited.
I’m excited and happy to be one of few non-Japanese researchers to receive a JST Presto (Sakigake) project grant, on the Topic Open Collective Eyewear.
It’s a strange feeling to have UbiComp and ISWC so close to my home. Amazing organization and impressive research and meeting old friends.
back @ this year’s CHI 2016. We just have 3 Late Breaking Work submissions accepted and it seems they are currently for free download at the ACM website
We took a risk in organizing the Eyewear Computing Seminar as we deviated largely from the standard model, yet I believe it payed off.
It felt nice to be back in Kansai for the biggest conferences in my field.
Overall very pleasant and inspiring event, nice to see some old friends and meet new ones.
I'm impressed mostly by the Hacking/Making Studios and the interactivity/demo exhibits. Siggraph is my new favorite research conference.
Katsutoshi Masai, one of my Master students had the idea to track facial expressions using low cost sensors in glasses. Quite nice work ;)
I’m pleased to announce the completion of my Ph.D. from the University of Passau under the guidance of my advisors:
Principal Advisor: Prof. Dr. Paul Lukowicz Secondary Advisor: Prof. Dr. Hans Gellersen
My doctoral thesis has been published and is available atOpus Bayern. The pdf is open access, so feel free to read it (careful it’s a 19 MB pdf): Compensating for On-Body Placement Effects in Activity Recognition as pdf
However, the sources were not available. Finally, I got around to push the latex sources of my dissertation up to github.
This project demonstrates real-time accelerometer data streaming from mobile devices using WebSocket technology. The system combines a Node.js web server with a Processing.org visualization interface. When a mobile device connects to the server through its browser, the visualization displays a distinctive red cube among randomly positioned background cubes. The cube features a dynamic transparent aura that responds to the intensity of the device’s movement - the more vigorously you shake your phone, the more pronounced the visual effect becomes.
