As part of a senior design collaboration with NASA's Astrobee team, I served as lead developer on an autonomous cargo management system for the International Space Station. Astrobee is NASA's free-flying robot designed to assist astronauts with routine tasks — this system extended its capabilities to independently identify, track, and manipulate cargo in a microgravity environment without astronaut intervention.
The project contributes to NASA's broader initiative to develop robotic assistants that reduce astronaut workload and improve operational efficiency — with the architecture designed to scale to future lunar habitats and deep-space missions. The completed prototype was showcased at Florida Atlantic University's Senior Design Showcase and received strong feedback from NASA collaborators.
Building for space is a fundamentally different engineering problem. The ISS environment introduces constraints that don't exist on Earth — microgravity means cargo doesn't stay put, lighting is inconsistent and extreme, and there's zero tolerance for system lockups when a robot is operating autonomously near astronauts and sensitive equipment.
The system needed to coordinate three distinct technical domains simultaneously — computer vision, embedded hardware control, and real-time networking — all with deterministic timing, fail-safe recovery, and power efficiency suitable for a resource-constrained space environment.
Vision Layer (Python / OpenCV)
├── Live camera feed ingestion
├── ArUco & AprilTag detection
├── 3D position estimation
└── Cargo ID + orientation output
│
│ TCP/IP socket
▼
State Machine Controller
├── Command queuing
├── Fail-safe & watchdog logic
└── System state management
│
▼
ESP32 Embedded Firmware (C++)
├── Electromagnetic actuation
├── Interrupt-driven response
├── Power state management
└── Fault loggingPython + OpenCV pipeline processes live camera feeds, detecting ArUco and AprilTag fiducial markers to identify cargo and compute 3D spatial position in real time — with occlusion handling and lighting compensation for the ISS visual environment.
Custom C++ firmware manages electromagnetic cargo capture and release with precise timing, interrupt-driven actuation, state-based power management (active/idle/sleep), and watchdog recovery for fault-tolerant autonomous operation.
Custom socket-based host-client architecture transmits position data, cargo IDs, and control commands between vision and hardware layers with minimal latency, command queuing, and error correction for reliable delivery.
Coordination layer manages system state, queues high-frequency commands, handles fail-safe logic, and provides comprehensive logging — ensuring deterministic behavior and graceful recovery in safety-critical operation.
Fiducial markers like ArUco give sub-centimeter positional accuracy at real-time frame rates without a GPU. For a cargo system where precise docking matters, the reliability far outweighs the constraint of requiring markers on cargo containers.
Latency was the primary constraint. A direct socket connection between the vision host and ESP32 adds microseconds, not milliseconds — critical when electromagnetic actuators need to fire within a narrow timing window after a position fix.
The ESP32 offers a dual-core processor, built-in WiFi for the TCP/IP layer, and real-time interrupt handling — all in a compact, low-power package. Its FreeRTOS support also made implementing the state machine and watchdog system straightforward.
Built in direct collaboration with NASA's Astrobee team through Florida Atlantic University's Senior Design program. Presented to NASA engineers at the FAU showcase.
The prototype demonstrated reliable autonomous cargo detection, tracking, and actuation under simulated ISS constraints. NASA collaborators highlighted its modular, scalable architecture as a strong foundation for future autonomous systems aboard the ISS and in next-generation space habitats. Presented at Florida Atlantic University's Senior Design Showcase to an audience of industry and NASA engineers.