Field robotics
Traversability benchmarking for field robots
A repeatable benchmark for comparing traversability and risk maps across algorithms and real field sites.
ROS 2PythonNumPyPyTorchCostmapsLiDARMMDet3DAWS S3
Problem
There was no objective way to compare traversability and risk maps across algorithms and sites, so filter-chain changes could not be judged on evidence.
Approach
- Bird's-eye-view ground-truth publishers for each site.
- Semantic-to-risk filter chains with a semantic label authority filter.
- Reachability ray-casting metrics over costmaps.
- Automated PDF reports comparing predicted vs. ground-truth risk maps.
Result
- Repeatable benchmark across multiple real field sites, including stairs and difficult terrain cases.
- Informed filter-chain and semantic-label-authority design.
- Investigated simulation-vs-real LiDAR behaviour: noise profiles, plane-fit scatter, bias, intensity, dropout, and sensor-model drift.
System breakdown
Hardware
- LiDAR-equipped field robots
- Multi-site customer datasets (described generically)
- GPU-backed benchmark runners
Firmware / embedded
- Sensor model validation against real LiDAR returns
- Intensity and dropout characterisation
Software
- ROS 2 costmap evaluation pipelines
- NumPy/PyTorch metric computation, MMDet3D semantics
- Templated benchmark configuration, reruns, long-timeout jobs
- Dataset migration and S3 transfer automation
Project demo video slot