Skip to main content
Version: FILS English

DriveSight

A mobile robot project with a camera for visual tracking, based on a Raspberry Pi Pico 2W and ESP32-S3.

info

Author: Neagu Andrei-Cristian
GitHub Project Link: https://github.com/UPB-PMRust-Students/project-mohgTheOmen.git

Description

DriveSight is a manually controlled robotic platform equipped with real-time video streaming. It integrates a camera module for live feed monitoring and a wireless controller interface for direct motor control, providing precise, remote navigation and situational awareness.

Motivation

I started this project to challenge myself and explore how far I could push the capabilities of embedded hardware, combining real-time control, wireless communication, and camera streaming on constrained systems.

Architecture

Main Components:

  • Raspberry Pi Pico 2W (Motor Controller): Controls motors, ultrasonic sensors, and receives movement commands via UART.
  • ESP32-S3: Captures a live camera feed and sends control commands from a connected Bluetooth controller.
  • Raspberry Pi Pico 2W (Display Host): Hosts a Wi-Fi access point and receives data from the motor controller to display real-time sensor values or system status on a small screen.

Feedback Mechanisms:

  • Current sensors (INA219): Monitor motor current to help maintain consistent speed across varying surface conditions.
  • Ultrasonic sensors: Provide obstacle detection and distance feedback for navigation and collision avoidance.

User Interface:

  • Web-based interface: Hosted on the ESP32-S3 for viewing the live camera feed.
  • Bluetooth controller: Used for manual driving input.
  • On-board display: Connected to Pico no.2, shows live telemetry from the motor controller over Wi-Fi.

Connections Between Components:

  • UART: Used for communication between the ESP32-S3 and motor controller Pico.
  • Wi-Fi: Motor controller Pico connects to the display host Pico (AP mode) for data transmission.
  • I2C: Used by the motor controller Pico for reading data from INA219 current and ultrasonic sensors.
  • GPIO: Used to control the L298N motor driver from the motor controller Pico.

Architecture Block Diagram

Log

Week 21 - 27 April

  • Tested components individually.

Week 28 April - 4 May

  • Configured the ESP32-S3 to connect to a Wi-Fi network and stream the live camera feed. This involved setting up the esp_camera library for capturing frames and integrating it with the ESP-IDF networking stack.
  • Configured the second Raspberry Pi Pico to display telemetry data received via TCP from the central Pico.

Week 5 - 11 May

  • Tested UART communication between the ESP32-S3 and the motor controller Pico 2W to ensure reliable data exchange.
  • Developed the motor control functionality to process commands received from the Bluetooth controller.

Week 12 - 18 May

  • Integrated motor movement logic to dynamically adjust based on proximity feedback from the ultrasonic sensors.
  • Began designing the chassis for the robot, focusing on structural stability and accommodating all hardware components.

Week 19 - 25 May

  • Completed hardware assembly for the robot, including wiring and mounting all components.
  • Developed and integrated software for the distance sensors, enabling them to function as parking sensors with real-time feedback on the remote display.
  • Implemented motor control logic to respond to Bluetooth controller input, allowing for precise manual driving.
  • Most core features are functional; only finishing touches and minor bug fixes remain.

Week 26 - 30 May

  • Finalized integration tests for all hardware modules to ensure stable operation during the PM Fair.
  • Verified live camera streaming, Bluetooth controller input, and telemetry display on the remote unit.
  • Performed a full system demo run to check for any last-minute issues and confirmed all features are presentation-ready.
  • Recorded the demo video and assembled all components into the 3D-printed housings.

Project Demo

Project Demo Video Link

Hardware

  • Raspberry Pi Pico 2W: Main controller handling sensors and motor control.
  • ESP32-S3: Provides camera feed for object tracking and sends control commands.
  • Raspberry Pi Pico 2W (Display Unit): Hosts an access point and displays telemetry data.
  • ST7735s: Display the telemetry data.
  • L298N Motor Driver: Drives the two DC motors.
  • 2× DC Motors: Provide movement for the robot.
  • 2× INA219 Current Sensors: Monitor motor currents to detect stalls or resistance changes.
  • 3× HC-SR04 Ultrasonic Sensors: Detect obstacles in front of the robot.
  • Power Bank and Li-ion Batteries: Powers the system.
  • Wires, Breadboard, and Connectors: For interconnecting all the components.

Pictures

Overview:

Overview1

Overview2

ESP32-S3:

ESP32-S3

AP Pico 2W:

Remote

Schematics

ESP32-S3:

ESP32-S3

Central Pico 2W:

Central_Pico2W

AP Pico 2W:

Remote_Pico2W

Final Project

Main Unit

Central

Remote

Remote

Bill of Materials

DeviceUsagePrice
2x Raspberry Pi Pico 2WMain control board 39.66 lei
ESP32-S3Camera and Wi-Fi 56.71 lei
L298N Motor DriverControls motor power 10.99 lei
ST7735s DisplayDisplays sensor data 27.99 lei
2x JGA25-370 MotorsRobot movement 49.00 lei
2x INA219 Current SensorCurrent monitoring 29.99 lei
2x TLP281 OptocouplerSignal isolation 8.49 lei
3x HC-SR04 Ultrasonic SensorObstacle detection 14.99 lei
Li-Ion 3x Battery HolderPower source 20.13 lei
Li-Ion 1x 18650 Battery PowerbankPower source 19.93 lei
Li-Ion 2x 18650 Battery PowerbankPower source 30.35 lei
Wires, Connectors, Breadboard, ResistorsWiringAlready had them

Software

ESP32-S3 (C)

LibraryDescriptionUsage
ESP-IDFOfficial Espressif IoT Development FrameworkBase SDK for Wi-Fi, networking, etc.
esp_cameraCamera driver for OV3660Captures and streams MJPEG frames
Bluepad32Bluetooth game controller libraryReceives input from Bluetooth controller

Raspberry Pi Pico 2W (Rust)

LibraryDescriptionUsage
embassyAsync embedded runtime for embedded systemsTask scheduling and async infrastructure
embassy-executorAsynchronous task executorSchedules and runs tasks in a no_std async environment
embassy-rpRP2040-specific HAL for EmbassyGPIO, SPI, PWM, ADC, and other peripherals
embassy-timeTimer and delay handlingNon-blocking frame-rate control, delays, timeouts
embassy-syncSynchronization primitivesUsed for async channels and mutexes
embassy-gpioGPIO abstraction for EmbassyManages input/output pins for buttons, sensors, and motor control
embassy-futuresFutures and async utilities for EmbassyEnables combining and managing multiple async tasks and events
cyw43Wi-Fi driver for the CYW43 chipConnecting the Pico 2W to Wi-FI
cyw43-pioPIO-based driver for the CYW43 chipEnables Wi-Fi using the RP2350's PIO peripheral
embassy-netTCP/IP networking stackHosts server or client for sending/receiving data
embedded-halHardware abstraction layer for embedded systemsStandardized traits for peripherals (GPIO, I2C, SPI, etc.)
hcsr04-asyncAsync driver for ultrasonic distance sensor (if used)Obstacle detection
defmtLightweight logging framework for embedded systemsEnables efficient, structured logging
defmt-rttRTT (Real-Time Transfer) backend for defmtOutputs logs via RTT for real-time debugging
embedded-graphics2D graphics library for embedded systemsDrawing text, shapes, and UI elements on the display
mipidsiDisplay driver for MIPI-compatible SPI LCDsDriving the ST7735s display via SPI
randRandom number generationUsed internally by the TCP stack for port/sequence numbers
smoltcpEmbedded TCP/IP networking stackProvides low-level TCP/IP networking for the Pico 2W
static-cellSafe static memory cell abstractionManages static data for async tasks and peripherals
alloc-cortex-mHeap allocator for Cortex-M microcontrollersEnables dynamic memory allocation in embedded Rust
libmMath library for embedded systemsProvides floating-point math functions in no_std environments
heaplessFixed-capacity data structures without heapUsed for queues, buffers, and collections in constrained systems
panic-probeMinimal panic handler for embedded RustReports panics via RTT or serial for debugging
  1. Personal Project Repo