Daehwa Kim
Updates
• Sep 15: Our work on a painting robot is accepted to IROS Workshop
• Sep 4: Graph as Policy (GaP) is accepted to CoRL 2026
• July 7: Two papers are conditionally accepted to UIST 2026
• May 21: Check out CMU SCS news about SoundBubble
• May 11: Invited talk at UC Berkeley "Non-anthropocentric Machine Perception"
I am a fifth-year PhD student at HCI Institute in Carnegie Mellon University, advised by Chris Harrison in the Future Interfaces Group. My research explores the intersection of Human-Computer Interaction, Sensing, and Robotics. My work has been recognized with two Best Paper Honorable Mention awards at CHI. Formerly, I was a member of UC Berkeley BAIR AUTO Lab, working with Ken Goldberg. I interned at Apple AI for Robotics (2024, 2025) and Meta Reality Labs (2023).
Research Statement
I explore non-anthropocentric machine perceptions for human-centered future computing. Agentic perception has made incredible strides in recent years. Yet today's intelligent systems still rely largely on streams of data captured by sensors originally designed for media and signals humans consume: RGB cameras matched to the human-visible spectrum, microphones tuned to the human hearing range, and so on. Indeed, considerable robotics research today is focused on replicating the sensory capabilities of humans, adding modalities such as tactile touch sensing. I believe these efforts betray an anthropocentric blind spot and that we are imposing an artificial upperbound defined by the limits of our own senses. In doing so, we may be constraining our ability to unlock step-function improvements in machine perception. We have already seen tremendous gains as models have moved from unimodal to multimodal representations. I believe the next step is transmodal sensing — approaches that transcend human senses to power intelligent systems. Richer sensory inputs open two significant possibilities. First, they may make previously difficult scene-understanding and manipulation problems dramatically easier, lowering the barrier to achieving human-level intelligence. Second, they open the door to perceptual capabilities — and intelligence — that can surpass our own. Importantly, this may not even require machines to first replicate the full suite of human senses. My PhD research has already demonstrated the potential gains of this approach, with novel sensing approaches spanning electric field sensing, multi-device/user spatial perception, and spatial audio.
Research
Agent Behavior as a Driver for Novelty and Creativity in Human-Robot Co-Drawing
IROS 2026 Workshop on Creativity in Assembly (Accepted)
Paper | DOI | Video | Code | Project
Object-Agent Twins (OATs): Expressive Micro-Agent Avatars for Inanimate Everyday Objects
UIST 2026: ACM Symposium on User Interface Software and Technology. (To Appear)
Paper | DOI | Video | Code | Project
Co-opting Mobile Device Cellular RF Front Ends for HCI Sensing
UIST 2026: ACM Symposium on User Interface Software and Technology. (To Appear)
Paper | DOI | Video | Code | Project
GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks
CoRL 2026: Conference on Robot Learning (Accepted).
Soundbubble: Finger-Bound Virtual Microphone using Headset/Glasses Beamforming
CHI 2026: ACM Conference on Human Factors in Computing Systems.
PatternTrack: Multi-Device Tracking Using Infrared, Structured-Light Projections from Built-in LiDAR
CHI 2025: ACM Conference on Human Factors in Computing Systems.
ARMOR: Egocentric Perception for Humanoid Robot Collision Avoidance and Motion Planning
CASE 2025: IEEE International Conference on Automation Science and Engineering
EMOTION: Expressive Motion Sequence Generation for Humanoid Robots with In-Context Learning
RA-L 2025: IEEE Robotics and Automation Letters
Power-over-Skin: On-Body Devices Powered Using Intra-Body RF Energy
UIST 2024: ACM Symposium on User Interface Software and Technology.
SoundScroll: Robust Finger Slide Detection Using Friction Sound and Wrist-Worn Microphones
ISWC 2024: Proceedings of the 2024 ACM International Symposium on Wearable Computers.
WorldPoint: Finger Pointing as a Rapid and Natural Trigger for In-the-Wild Mobile Interactions
ISS 2023: ACM International Conference on Interactive Surfaces and Spaces.
Pantœnna: Mouth Pose Estimation for VR/AR Headsets Using Low-Profile Antenna and Impedance Characteristic Sensing
UIST 2023: ACM Symposium on User Interface Software and Technology.
OmniSense: Exploring Novel Input Sensing and Interaction Techniques on Mobile Device with Omni-Directional Camera
CHI 2023: ACM Conference on Human Factors in Computing Systems.
EtherPose: Continuous Hand Pose Tracking with Wrist-Worn Antenna Impedance Characteristic Sensing
UIST 2022: ACM Symposium on User Interface Software and Technology.
TriboTouch: Micro-Patterned Surfaces for Low Latency Touchscreens
CHI 2022: ACM Conference on Human Factors in Computing Systems.
Honorable Mention Award (Top 5%)
AtaTouch: Robust Finger Pinch Detection for a VR Controller Using RF Return Loss
CHI 2021: ACM Conference on Human Factors in Computing Systems.
Honorable Mention Award (Top 5%)
OddEyeCam: A Sensing Technique for Body-Centric Peephole Interaction Using WFoV RGB and NFoV Depth Cameras
UIST 2020: ACM Symposium on User Interface Software and Technology.
MagTouch: Robust Finger Identification for a Smartwatch Using a Magnet Ring and a Built-in Magnetometer
CHI 2020: ACM Conference on Human Factors in Computing Systems.
AWARDS & HONORS
Mar 2022
Honorable Mention Award (top 5%), ACM CHI 2022
Paper Title: "TriboTouch: Micro-Patterned Surfaces for Low Latency Touchscreens"
Mar 2021
Honorable Mention Award (top 5%), ACM CHI 2021
Paper Title: "AtaTouch: Robust Finger Pinch Detection for a VR Controller Using RF Return Loss"
Feb 2021
Outstanding M.S. Thesis Award, KAIST School of Computing
Thesis Title: "OddEyeCam: A Sensing Technique for Body-Centric Peephole Interaction Using WFoV RGB and NFoV Depth Cameras"
FUN PROJECTS
Shadow 3D scanner (2022)
AudioHero (2019)
VRone (2018)
System Light @ ETRI (2018)
Finger Joystick Interaction (2017)
TUIT Android Lecture (2016)
MEDIA ART
My age 22 - Travel to Europe with a Drone (2018)
Travel Film
Town Where Engineers Live (2015)
Web Drama
Designed by Daehwa