Robotic Action Planning Using Large Language Models - Silva, César Bastos Da; Ramírez, Juan Luis Barraza; Mastelari, Niederauer; Lotufo, Roberto; Pereira, Jayr; Rohmer, Eric (Anais do Simpósio Brasileiro de Robótica e Simpósio Latino Americano de Robótica (SBR/LARS); 2024: Anais do XXI Simpósio Latino Americano de Robótica, 2024)
This study explores the integration of artificial intelligence (AI) and large language models (LLMs) in robotics, focusing on task planning and execution. We implemented a Reasoning and Acting (ReAct) system within a simulated environment, utilizing a humanoid robot equipped with various tools for searching, locomotion, vision, manipulation, and communication. The robot operates based on natural language prompts and utilizes the LangChain framework to facilitate interaction with the LLM. We conducted experiments to evaluate the robot’s performance on tasks requiring short-term, medium-term, and long-term memory. Short-term memory tasks involved single-step actions, medium-term memory tasks needed the completion of two-step sequences, and long-term memory tasks involved three or more steps. The results demonstrated a high success rate for short-term tasks, while performance for medium and long-term tasks varied depending on the number of steps involved. Our findings highlight both the challenges and potential of using AI and LLMs in robotic task planning. The results demonstrate the promise of enhancing robotic capabilities to perform complex tasks through natural language instructions. The work is available on:  https://github.com/cesarbds/LLM_Planner