Building the Decision-Making Brain for Humanoid Robots

The client is a robotics solutions company that develops physical robots for real-world tasks. At the core of its system is an AI decision-making module that translates natural human language into precise physical actions. It combines specialized AI agents, multimodal awareness, contextual memory, and layered safety guardrails to support intelligent, personalized, and reliable robotic interactions.

3-Month Phase 1 Pilot

Delivered a working MVP in three months, keeping pace with the vision, cloud, and hardware teams' wider development milestones. The system runs in both the Gazebo simulation environment and on physical robot hardware, with key security-patrol flows demonstrated end-to-end and approved by the client.

1 Unified Brain

Replaced separate conversational and task-execution logic across all robot roles with one centralized decision architecture, coordinating specialized AI agents for Q&A, physical actions, and patrol planning.

100% Interruptible Execution

Urgent safety events can preempt a task already in progress, preserve its state, and allow the robot to resume exactly where it left off afterward.

Multi-Modal Awareness

Spoken requests, vision events, camera, sensor, SLAM, and audio inputs are processed as first-class context, enabling the robot to plan and react based on its real-world environment.

Configuration-Driven Robot Roles

Each robot's persona and permitted skills are defined through configuration, allowing the client to introduce new roles without changing or redeploying the core AI system.

3 Layers of Safety Guardrails

Input, output, and action-level validation help ensure that user requests, AI-generated responses, and physical movements comply with defined safety policies and robot permissions.

Client information background
Anonymous client logo

About the client

A humanoid robotics company backed by a major technology group, developing intelligent humanoid robots for real-world applications. Founded in 2025 and based in Southeast Asia, the company combines robotics, AI, sensing, and automation to create safe, human-centric machines designed to operate in everyday environments. Its vision is to advance humanoid robotics from research into practical, scalable deployment across areas such as security, surveillance, assistance, and other real-world services.

The challenge

The client needed a single, intelligent decision layer that could translate natural language into precise physical actions while remaining safe, interruptible, and responsive to multiple input sources in real time. Unlike conversational AI, errors in a robotic system can have physical consequences, making reliability critical. The project also required coordination across vision, cloud, and hardware teams working with different interfaces and release schedules, all within an aggressive three-month MVP timeline.

From Language to Safe Physical Action
The AI system needed to interpret natural-language requests and convert them into precise, executable robot actions. This required coordinating conversational AI, motion execution, and patrol planning through a unified decision-making system rather than separate role-specific logic. Complex requests had to be broken down into manageable steps, with each action respecting the robot's permitted skills, user privileges, and physical safety constraints. The system also needed to validate generated plans and allow users to review or confirm actions before execution. For the initial security-patrol use case, the system had to translate spoken commands and detected anomalies into safe, actionable plans.
Handling Interruptions Without Losing Progress
Robotic tasks rarely run without interruption, particularly in security environments where unexpected events demand immediate attention. The AI system needed to respond to emergencies, such as detecting a person falling or a door left open, even while another task was underway. This required a reliable way to prioritize urgent requests, suspend ongoing activities, and resume them without losing progress or restarting execution. The system also needed to preserve task context, track individual execution steps, and monitor outcomes to ensure that interruptions did not compromise the accuracy or continuity of complex physical workflows.
Managing Multi-Modal Inputs, Memory, and Scalability
The AI system needed to process voice, camera, sensor, SLAM, and audio inputs while maintaining awareness of user preferences, conversation history, and robot-specific roles. Supporting personalized interactions required both short-term conversational memory and long-term knowledge retrieval. These capabilities also had to integrate with services maintained by separate vision, cloud, and hardware teams, each operating through different interfaces and development schedules. Beyond the initial security-patrol application, the client needed an extensible architecture that could accommodate new robot roles, specialized agents, and evolving integrations without requiring fundamental changes to the core decision-making system.

Why CodeLink?

The client needed a partner with proven expertise across AI agents, real-time systems, and physical hardware integration; a combination that few teams are equipped to handle end-to-end. CodeLink combined that technical depth with production experience in LLM-based systems, enabling the team to contribute from week one, avoid costly early mistakes, and design the AI system around real-world reliability while keeping pace with the wider robotics program.

The result

CodeLink delivered the working AI system pilot within three months. The system operates in both Gazebo simulation and physical robot environments, with core security-patrol workflows demonstrated end-to-end and approved by the client. Its unified architecture provides a foundation for increasingly autonomous robotic capabilities.

Unified AI Agents for Safe Physical Execution
CodeLink developed a centralized decision architecture that coordinates specialized Q&A, Motion, and Patrol Planning agents through a shared task router. A Physical ReAct Loop translates high-level requests into executable steps, monitors outcomes, and adjusts actions based on real-world feedback. Three layers of guardrails validate user inputs, AI-generated responses, and physical actions against defined safety policies. For tasks requiring additional oversight, a human-in-the-loop workflow allows users to review, modify, and approve plans before execution. Together, these capabilities enable the system to turn natural-language instructions into controlled physical actions while maintaining consistent behavior across robot roles.
Reliable Interruption, Resumption, and Task Monitoring
The AI system achieves 100% interruptible execution through priority-based task queues, asynchronous processing, and checkpointing. When an urgent event occurs, the system preserves the ongoing task's state, prioritizes the emergency, and resumes the original workflow exactly where it left off afterward. A dedicated Result Collector tracks subtask outcomes and transfers relevant information between execution steps, maintaining continuity across complex workflows. Task-monitoring mechanisms also record execution status, selected skills, priorities, and results, giving developers visibility into robotic behavior for debugging and refinement. This enables immediate emergency responses without sacrificing progress on previously assigned tasks.
Multi-Modal Intelligence with Extensible Architecture
The AI system combines real-time camera, sensor, SLAM, and audio inputs with personalized context through a centralized Context Builder. Short-term memory and long-term knowledge retrieval, supported by Mem0 and LightRAG, allow the robot to recall user preferences, access organizational knowledge, and maintain context across interactions. A dedicated memory update workflow further supports personalization over time. The modular architecture separates shared decision-making capabilities from robot-specific roles, with configuration-driven personas, permitted skills, and plug-and-play agents. This allows the client to expand beyond security patrols into applications such as tour guidance and household assistance without rebuilding the core AI system.

Engineering Excellence.
Built for Enterprise and Institutional Innovation.

Contact Us
background

CodeLink Newsletter

Stay up to date with the latest insights on software engineering and AI strategy from CodeLink.

CodeLink

We partner with enterprises and institutions to turn complex technology challenges into reliable digital systems. Through expert software engineering, AI implementation, and disciplined delivery, we help organizations modernize operations, strengthen platforms, and deliver measurable business outcomes.

Contact Us

(+84) 2839 333 143info@codelink.io
Book A Discovery Call
2026 © CodeLink Limited.
All right reserved.
Privacy Statement |