Orchestration of IoT devices in complex scenarios with LLM-based human interaction

• Augusto de O. Perin UFBA https://orcid.org/0009-0004-9846-5347
• Marcus V. A. Pinto UFBA https://orcid.org/0009-0004-5955-7361
• Raimundo J. de A. Macêdo UFBA https://orcid.org/0000-0002-7672-2208
• Bruno P. Santos UFBA https://orcid.org/0000-0003-4501-2323
• Maycon L. M. Peixoto UFBA https://orcid.org/0000-0002-4851-5228
• Gustavo B. Figueiredo UFBA https://orcid.org/0000-0001-9756-378X
• Cássio V. S. Prazeres UFBA https://orcid.org/0000-0003-0197-0909

DOI: https://doi.org/10.5753/sbrc.2026.19352

Abstract

The use of Large Language Models (LLMs) in IoT device orchestration offers high semantic flexibility for command interpretation, yet their probabilistic nature imposes challenges to operational safety. The potential for hallucinations in physical actuation systems demands mitigation strategies that extend beyond the model’s native capabilities. To mitigate these risks, a hybrid architecture is proposed, combining the generative capabilities of LLMs with a deterministic validation layer focused on physical safety constraints. The methodology evaluated 1,250 unique smart agriculture scenarios, totaling 12,500 executions, achieving a global success rate of 74.34%. Results indicate that safety is not intrinsic to the model but depends on prompt engineering; technical instructions reduced the violation rate from 41.88% to 1.80%. Additionally, efficiency metrics highlighted the trade-off between correctness and operational cost. Finally, it was observed that the XML format outperforms JSON in context injection for secure orchestration.

References

Acharya, D. B., Kuppan, K., and Divya, B. (2025). Agentic ai: Autonomous intelligence for complex goals—a comprehensive survey. IEEE Access, 13:18912–18936.

Cui, H., Du, Y., Yang, Q., Shao, Y., and Liew, S. C. (2025). Llmind: Orchestrating ai and iot with llm for complex task execution. IEEE Communications Magazine, 63(4):214–220.

Friha, O., Amine Ferrag, M., Kantarci, B., Cakmak, B., Ozgun, A., and Ghoualmi-Zine, N. (2024). Llm-based edge intelligence: A comprehensive survey on architectures, applications, security and trustworthiness. IEEE Open Journal of the Communications Society, 5:5799–5856.

Geens, R., Shi, M., Symons, A., Fang, C., and Verhelst, M. (2024). Energy cost modelling for optimizing large language model inference on hardware accelerators. In 2024 IEEE 37th International System-on-Chip Conference (SOCC), pages 1–6.

He, F., Zhu, T., Ye, D., Liu, B., Zhou, W., and Yu, P. S. (2025). The emerged security and privacy of llm agent: A survey with case studies. ACM Comput. Surv., 58(6).

Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., de Las Casas, D., Hendricks, L. A., Welbl, J., Clark, A., Hennigan, T., Noland, E., Millican, K., van den Driessche, G., Damoc, B., Guy, A., Osindero, S., Simonyan, K., Elsen, E., Vinyals, O., Rae, J. W., and Sifre, L. (2022). Training compute-optimal large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Red Hook, NY, USA. Curran Associates Inc.

Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. (2020). Scaling laws for neural language models.

Lamnaour, M., Raiss, M., Mesmoudi, Y., El Khamlichi, Y., Tahiri, A., and Touhafi, A. (2024). A semantic-based middleware for supporting heterogeneity and context-awareness within iot applications. Journal of Communications, page 19–27.

Raiaan, M. A. K., Mukta, M. S. H., Fatema, K., Fahad, N. M., Sakib, S., Mim, M. M. J., Ahmad, J., Ali, M. E., and Azam, S. (2024). A review on large language models: Architectures, applications, taxonomies, open issues and challenges. IEEE Access, 12:26839–26874.

Rivkin, D., Hogan, F., Feriani, A., Konar, A., Sigal, A., Liu, X., and Dudek, G. (2025). Aiot smart home via autonomous llm agents. IEEE Internet of Things Journal, 12(3):2458–2472.

Singh, H., Das, R. J., Han, M., Nakov, P., and Laptev, I. (2025). Malmm: Multi-agent large language models for zero-shot robotic manipulation. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 20386–20393.

Vemprala, S. H., Bonatti, R., Bucker, A., and Kapoor, A. (2024). Chatgpt for robotics: Design principles and model abilities. IEEE Access, 12:55682–55696.

Vespoli, S., Mattera, G., Marchesano, M. G., Nele, L., and Guizzi, G. (2025). Adaptive manufacturing control with deep reinforcement learning for dynamic wip management in industry 4.0. Computers & Industrial Engineering, 202:110966.

Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., Zheng, R., Fan, X., Wang, X., Xiong, L., Zhou, Y., Wang, W., Jiang, C., Zou, Y., Liu, X., Yin, Z., Dou, S., Weng, R., Qin, W., Zheng, Y., Qiu, X., Huang, X., Zhang, Q., and Gui, T. (2025). The rise and potential of large language model based agents: a survey. Science China Information Sciences, 68(2).

Xiao, B., Kantarci, B., Kang, J., Niyato, D., and Guizani, M. (2025). Efficient prompting for llm-based generative internet of things. IEEE Internet of Things Journal, 12(1):778–791.

Zhang, M., Shen, X., Cao, J., Cui, Z., and Jiang, S. (2025a). Edgeshard: Efficient llm inference via collaborative edge computing. IEEE Internet of Things Journal, 12(10):13119–13131.

Zhang, X., Dong, X., Wang, Y., Zhang, D., and Cao, F. (2025b). A survey of multi-ai agent collaboration: Theories, technologies and applications. In Proceedings of the 2nd Guangdong-Hong Kong-Macao Greater Bay Area International Conference on Digital Economy and Artificial Intelligence, DEAI ’25, page 1875–1881, New York, NY, USA. Association for Computing Machinery.