Research
Robust multi-modal perception for embodied agents
The through-line: multimodal perception → human-aware understanding → navigation and action → reliable embodied systems.
Field-robust perception
Traversability and semantic understanding for robots in unstructured outdoor environments, where sensor models drift and terrain breaks assumptions.
Vision-Language-Action models & learned manipulation
Embodiment transfer and fine-tuning generalist policies on small real-robot datasets collected through teleoperation.
Multi-modal & non-visual sensing
Acoustic perception for occluded and non-line-of-sight scenes, plus fusion of audio, LiDAR, and vision for scene understanding.
Research
OccluNet — acoustic detection of occluded objects
Neural audio method that detects both the occluder and the object hidden behind it using a 16-channel microphone array.
Problem
Cameras and LiDAR cannot see objects hidden behind a barrier. OccluNet uses sound to detect both the occluder and the hidden object under direct occlusion — without the relay-wall setup that prior non-line-of-sight methods require.
Method
- Emit a chirp (frequency sweep) into the scene and record it with a 16-channel microphone array.
- Extract room impulse responses via TSP deconvolution.
- Feed multi-channel spectrogram features into a dual-branch CNN (temporal + spectral) — the AudioIR encoder.
- Classify the occluded object and regress distance/direction from the acoustic features.
Results & status
- Journal paper "Neural Audio-based Detection of Occluding and Occluded Objects" submitted to Applied Acoustics (Elsevier), through reviewer revision rounds.
- [CONFIRM CURRENT STATUS: under review / accepted / published + DOI]
- [ADD KEY QUANTITATIVE RESULTS]
Research
Project Ace — robotic table tennis (Sony AI)
Contributor to a reinforcement-learning robotic table-tennis system that outplays elite human players, published in Nature.
Problem
Table tennis demands sub-100ms perception, prediction, and control against a human opponent — a benchmark for real-time embodied reinforcement learning.
Method
- Reinforcement-learning policy for stroke selection and execution on a physical robot.
- High-rate ball perception and trajectory prediction.
- Sim-to-real transfer and extensive evaluation against human players.
Results & status
- Published in Nature as "Outplaying elite table tennis players with an autonomous robot".
- [EXACT PAPER TITLE AS PUBLISHED, YEAR, AUTHOR POSITION, DOI]
- [YOUR SPECIFIC CONTRIBUTION — e.g. perception, simulation, hardware, evaluation]
Future directions
What I want to study next
Open questions I would pursue in a PhD.
- Can non-visual sensing (acoustics, contact, vibration) give embodied agents reliable state estimates where vision and LiDAR fail?
- How little real-robot data is enough to transfer a generalist VLA policy to a new embodiment?
- How should traversability risk be calibrated so a robot's confidence matches its real failure rate in the field?
- Can socially and acoustically aware navigation be learned jointly with geometric planning rather than bolted on?