LLM Agents for Search via Reinforcement Learning with Trajectory-Level Self-Evaluation - Costa, Leandro Yamachita da; Oliveira e Souza Filho, João Baptista de (Anais do Encontro Nacional de Inteligência Artificial e Computacional (ENIAC); 2025: Anais do XXII Encontro Nacional de Inteligência Artificial e Computacional, 2025)
Agentic search enables language models to iteratively query and reason over retrieved information to answer complex questions. While reinforcement learning is used to train such agents, most approaches rely solely on final-answer accuracy as the reward signal, which can limit learning. We analyze a reward strategy combining final-answer correctness with trajectory-level feedback, guiding the agent to improve its search behavior. The agent is trained with QLoRA for efficient fine-tuning and performs self-evaluation by comparing answers to references and scoring search trajectories using predefined rubrics, without external supervision. Experiments on multi-hop QA tasks show gains of up to 5.7 points over standard reward setup.