Multimodal LLMs as Image-to-Tabular Translators for Ultra-Short-Term Photovoltaic Forecasting - Fontoura, Diego de Carvalho Neves da; Aguiar, Marilton Sanchotene de (Anais do Seminário Integrado de Software e Hardware (SEMISH), 2026)
Ultra-short-term photovoltaic forecasting is strongly affected by cloud dynamics, yet integrating visual information typically requires complex, dedicated computer vision architectures. This study proposes an alternative approach in which multimodal LLMs (OpenAI and Gemini) serve as image-to-tabular translators. The method converts directional sky-camera images into numeric variables, which are then fused with conventional predictors, such as inverter and meteorological data. Experimental evaluations across 5-, 10-, and 15-minute horizons demonstrate that LLM-enhanced configurations consistently outperform a strong tabular baseline. The results suggest that LLMs can serve as a modular and auditable intermediate layer between raw imagery and standard forecasting models. Limitations include modest performance gains and reliance on a single pilot installation, requiring future multi-site validation.