Real-Time Trajectory Matching for an Omnidirectional Robot
Designed a 3 wheeled omnidirectional robot capable of following the trajectory of a free floating balloon.
Robot Assembly
Three-wheeled omnidirectional robot with spring-loaded balloon trampoline.
Overview
This project consists of the design and development of a custom omnidirectional robot to serve as a modular research platform. The robotics lab at RIT lacked an omnidirectional platform.
Object Detection with YOLOv8
- Detection Model: YOLOv8 trained on balloon images for real-time detection.
- V2 Balloon Dataset: 75 annotated images processed into YOLO-compatible format.
- Deployment: Model deployed on an OAK-D camera with DepthAI pipelines for real-time detection and velocity estimation.
Control System Design
- Architecture: ROS-based system with modular nodes for kinematics, velocity updates, and PID control.
- PID Controllers: Used for wheel velocity control based on encoder feedback.
- Object Detection and Tracking:YOLOv8 employed with DepthAI pipeline to estimate 3D velocity of a floating balloon with a weighted average method.
Hardware Overview
- Omnidirectional Drive Base: Three-wheeled design using custom 3D-printed Swedish-90 wheels.
- Balloon Launcher: Spring-loaded trampoline mechanism for interaction testing.
- Power System:
- 7.4V LiPo battery.
- Buck converters to 5V rails.
- LDO to 3.3V rail.
Robot CAD
Wheel CAD
Two of these panels screwed together back to back make up a wheel with pins through the rollers.
Performance and Results
- YOLOv8 achieved a precision-recall score of 0.821 for balloon detection.
- Real-time tracking achieved 9 FPS on Jetson Nano with OAK-D.
- Hardware demonstrated issues with the Jetson Nano's pins not reading encoder values of the wheels, preventing smooth operation due to a lack of odometry. With more time this would be solved by transitioning to an external microcontroller for peripheral control.
View the Code
All source code, CAD models, and documentation for this project are available on GitHub.
View on GitHub