Available summer 2027 · ~6 months
CV

Autonomous Thymio: localization that survives a blindfold

Year
2025
Context
EPFL · Basics of Mobile Robotics course
Team
Team of 4
My part
State estimation (EKF)
Links
Code (opens in a new tab)

TL;DR

Problem
Reach a goal on a 100 × 70 cm arena and avoid mapped and surprise obstacles, using one overhead camera that can be covered and odometry that drifts.
What we built
A camera + odometry navigation stack for a real Thymio. My module: the Extended Kalman Filter over [x, y, θ] and its noise identification.
Result
Validated live in three scenarios: a nominal run with a surprise obstacle, kidnapping, and camera blackout.
Stack
Python, FilterPy, OpenCV (ArUco), NumPy/SciPy, NetworkX, tdmclient

A small robot has to reach its goal with a camera that can be blocked and wheels that drift.

  • BLACKOUT ✓ · KIDNAP ✓
  • EKF
  • Sensor fusion
  • OpenCV
  • Real robot
Overhead view of the Thymio robot with an ArUco marker on a green arena with dark obstacles and a pink goal

The approach

  1. Vision (team). OpenCV thresholding finds the map, obstacles and goal; an ArUco marker on the robot gives its pose from the overhead camera.
  2. Planning (team). A visibility graph over the inflated obstacle corners and Dijkstra give the global path; proximity sensors handle surprise obstacles.
  3. Estimation (mine). An EKF over [x, y, θ] with a nonlinear differential-drive motion model and wheel speeds as input, corrected by the camera pose whenever it is available.
  4. Noise from data (mine). Camera covariance from repeated marker readings against ruler-measured positions; wheel noise from commanded against measured speed at 10 setpoints.
  5. Fallbacks. Without the camera, the filter runs on odometry alone. A jump of more than 20 cm between camera fixes is treated as a kidnapping and triggers a new plan.

What it achieved

  • Camera + odometry fusion. The filter runs inside the robot’s live control loop on a 100 × 70 cm arena.
  • Wheel speed calibrated from 10 setpoints: v [cm/s] = 0.0268 u + 0.4754, in Thymio motor units u.
  • Process noise from input uncertainty. Q = B Σᵤ Bᵀ + Q_model. I kept the nonlinear motion model after simulation showed a larger heading error with the linearized one.
  • 3 live scenarios passed: surprise obstacle, kidnapping (> 20 cm jump detected and replanned), and camera blackout.

Footage & figures

Kidnapping (video still)
Clip 01Kidnapping: the robot is picked up and moved, the jump is detected, and a new path is planned. Panels: internal map, raw camera, planned path. Team demo.Full size video: Kidnapping (MP4, opens in a new tab)
Camera covered (video still)
Clip 02Camera covered: the robot keeps estimating its position from wheel odometry alone. Team demo.Full size video: Camera covered (MP4, opens in a new tab)
Visibility graph around the inflated obstacles, with the shortest path and the robot's heading
Fig. 01Global planning: a visibility graph around the inflated obstacles and the shortest path to the goal. The EKF pose is the start of every plan.Team work · path-planning module by a teammateFull size image: Visibility graph around the inflated obstacles, with the shortest path and the robot's heading (opens in a new tab)

Notes

Team of 4 with one module each: global navigation and motion control, local navigation, vision, and state estimation (mine). The planning figure and the demo recordings show the whole team’s stack.