Representation Engineering in the Legal Domain: A Comparative Study of Linear and Non-Linear Probing for Initial Petition Classification
Resumo
This work compares linear and non-linear probing applied to the hidden states of the Jurema-7B model to classify Repetitive Demand Resolution Incident (IRDR) petitions. To mitigate textual length and class imbalance, summarization strategies and context injection via hybrid RAG were integrated. Results indicate that non-linear probing outperforms the linear approach, achieving a macro F1-score of 0.617, with statistically significant differences confirmed by McNemar’s test with Holm-Bonferroni correction in seven of the eight evaluated scenarios. We conclude that non-linear representation engineering is a viable alternative for automating procedural screening in the Brazilian judiciary.Referências
Akarsu, M., Karaman, R. K., and Mierbach, C. (2026). From BM25 to corrective RAG: Benchmarking retrieval strategies for text-and-table documents. arXiv preprint arXiv:2604.01733.
Aldabergen, A., Kynabay, B., and Kadyrov, S. (2026). Layer-wise probing of paralinguistic attributes in fine-tuned Whisper for Kazakh speech. Engineering, Technology & Applied Science Research, 16(2):33399–33404.
Almeida, V., S. Pires, R., A. Monteiro Neto, J., and Furtado, V. (2026). Using interpretability to uncover legal petition structures and optimize text classification. In Proceedings of the Twentieth International Conference on Artificial Intelligence and Law, ICAIL ’25, page 150–158, New York, NY, USA. Association for Computing Machinery.
Andrade, C. M., Magalhães, G., Asevedo, R., Paes, J., Oliveira, I., Pagano, A., Reis, Z., and Gonçalves, M. A. (2025). Estudo do impacto de dados sintéticos e paráfrases na mitigação do desbalanceamento em tarefas de classificação de textos em português com baixa amostragem. In Anais do XL Simpósio Brasileiro de Bancos de Dados, pages 837–843, Porto Alegre, RS, Brasil. SBC.
Araújo, G., Santana, E., and Lobato, F. (2026). Hybrid summarization for Brazilian judicial decisions. In Anais do XXII Simpósio Brasileiro de Sistemas de Informação, pages 910–929, Porto Alegre, RS, Brasil. SBC.
Azaria, A. and Mitchell, T. (2023). The internal state of an LLM knows when it’s lying. In Bouamor, H., Pino, J., and Bali, K., editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 967–976, Singapore. Association for Computational Linguistics.
Bartoszcze, L., Munshi, S., Sukidi, B., Yen, J., Yang, Z., Williams-King, D., Le, L., Asuzu, K., and Maple, C. (2025). Representation engineering for large-language models: Survey and research challenges. arXiv preprint arXiv:2502.17601.
Belinkov, Y. (2022). Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48(1):207–219.
Carvalho Neto, P. B. d. (2026). LegalBench-BR: A benchmark for evaluating large language models on Brazilian legal decision classification.
Chagas, M. A. d. N., Oliveira, A. F. d., Pinto Netto, M., Pontes, M. d. O. E. d., Alves, C. V. P., and Tizzo, L. G. L. (2025). A estrutura da petição inicial na escrita jurídica: uma abordagem propositiva. Revista de Ciências Jurídicas e Empresariais, 26(1):26–37.
Chen, J. and Lv, S. (2022). Long text truncation algorithm based on label embedding in text classification. Applied Sciences, 12(19).
Chi, J., Wang, K., Chen, Y., Lin, X., and Xu, Q. (2025). Activations as features: Probing LLMs for generalizable essay scoring representations. arXiv preprint arXiv:2512.19456.
Conselho Nacional de Justiça (CNJ) (2026). Painel de estatísticas do poder judiciário. Acesso em: 17 jun. 2026.
da Silva Junior, D., de Oliveira, D., and Paes, A. (2025). Evaluating text representations for unsupervised legal semantic textual similarity in Brazilian portuguese. Discover Data, 3.
do Carmo, F. A. (2024). Representações Embeddings Orientadas à Linguagem Jurídica Brasileira. Mestrado em engenharia da computação e sistemas, Universidade Estadual do Maranhão, São Luís/MA.
Garcia, M. H., Couturier, C., Diaz, D. M., Mallick, A., Kyrillidis, A., Sim, R., Ruhle, V., and Rajmohan, S. (2025). Exploring how LLMs capture and represent domain-specific knowledge. arXiv preprint arXiv:2504.16871.
Hakimi Parizi, A., Liu, Y., Nokku, P., Gholamian, S., and Emerson, D. (2023). A comparative study of prompting strategies for legal text classification. In Preoťiuc-Pietro, D., Goanta, C., Chalkidis, I., Barrett, L., Spanakis, G., and Aletras, N., editors, Proceedings of the Natural Legal Language Processing Workshop 2023, pages 258–265, Singapore. Association for Computational Linguistics.
Jayakumar, T., Farooqui, F., and Farooqui, L. (2023). Large language models are legal but they are not: Making the case for a powerful LegalLLM. In Preot,iuc-Pietro, D., Goanta, C., Chalkidis, I., Barrett, L., Spanakis, G., and Aletras, N., editors, Proceedings of the Natural Legal Language Processing Workshop 2023, pages 223–229, Singapore. Association for Computational Linguistics.
Jin, C. and Rinard, M. (2024). Latent causal probing: A formal perspective on probing with causal models of data. arXiv preprint arXiv:2407.13765.
Li, K., Hopkins, A. K., Bau, D., Viégas, F., Pfister, H., and Wattenberg, M. (2024). Emergent world representations: Exploring a sequence model trained on a synthetic task. arXiv preprint arXiv:2210.13382.
Mallen, A., Brumley, M., Kharchenko, J., and Belrose, N. (2024). Eliciting latent knowledge from quirky language models. arXiv preprint arXiv:2312.01037.
Park, K., Choe, Y. J., and Veitch, V. (2024). The linear representation hypothesis and the geometry of large language models. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org.
Peixoto, M., Silva, G., Figueredo, G., Silva, P., and Luz, E. J. S. (2026). Global vs. local sentence embeddings for Brazilian Portuguese: Revisiting monolingual models in the age of foundation models. In Souza, M., de Dios-Flores, I., Santos, D., Freitas, L., Souza, J. W. d. C., and Ribeiro, E., editors, Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1, pages 529–539, Salvador, Brazil. Association for Computational Linguistics.
Sawtell, M., Masterman, T., Besen, S., and Brown, J. (2024). Lightweight safety classification using pruned language models. arXiv preprint arXiv:2412.13435.
Skean, O., Arefin, M. R., Zhao, D., Patel, N., Naghiyev, J., LeCun, Y., and Shwartz-Ziv, R. (2025). Layer by layer: Uncovering hidden representations in language models. arXiv preprint arXiv:2502.02013.
Tupinambá, C. and Teixeira, S. T. (2022). O incidente de resolução de demandas repetitivas na justiça do trabalho. Revista do Tribunal do Trabalho da 2ª Região, 14(28):50–75.
Wilkens, R., Zilio, L., and Villavicencio, A. (2023). Assessing linguistic generalisation in language models: A dataset for Brazilian portuguese. arXiv preprint arXiv:2305.14070.
Wirth, R. and Hipp, J. (2000). CRISP-DM: Towards a standard process model for data mining. Proceedings of the 4th International Conference on the Practical Applications of Knowledge Discovery and Data Mining.
Zhao, H., He, Z., Tang, Y., Yang, F., Payani, A., Liu, D., and Du, M. (2026). Rep2Text: Decoding full text from a single LLM token representation. arXiv preprint arXiv:2511.06571.
Aldabergen, A., Kynabay, B., and Kadyrov, S. (2026). Layer-wise probing of paralinguistic attributes in fine-tuned Whisper for Kazakh speech. Engineering, Technology & Applied Science Research, 16(2):33399–33404.
Almeida, V., S. Pires, R., A. Monteiro Neto, J., and Furtado, V. (2026). Using interpretability to uncover legal petition structures and optimize text classification. In Proceedings of the Twentieth International Conference on Artificial Intelligence and Law, ICAIL ’25, page 150–158, New York, NY, USA. Association for Computing Machinery.
Andrade, C. M., Magalhães, G., Asevedo, R., Paes, J., Oliveira, I., Pagano, A., Reis, Z., and Gonçalves, M. A. (2025). Estudo do impacto de dados sintéticos e paráfrases na mitigação do desbalanceamento em tarefas de classificação de textos em português com baixa amostragem. In Anais do XL Simpósio Brasileiro de Bancos de Dados, pages 837–843, Porto Alegre, RS, Brasil. SBC.
Araújo, G., Santana, E., and Lobato, F. (2026). Hybrid summarization for Brazilian judicial decisions. In Anais do XXII Simpósio Brasileiro de Sistemas de Informação, pages 910–929, Porto Alegre, RS, Brasil. SBC.
Azaria, A. and Mitchell, T. (2023). The internal state of an LLM knows when it’s lying. In Bouamor, H., Pino, J., and Bali, K., editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 967–976, Singapore. Association for Computational Linguistics.
Bartoszcze, L., Munshi, S., Sukidi, B., Yen, J., Yang, Z., Williams-King, D., Le, L., Asuzu, K., and Maple, C. (2025). Representation engineering for large-language models: Survey and research challenges. arXiv preprint arXiv:2502.17601.
Belinkov, Y. (2022). Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48(1):207–219.
Carvalho Neto, P. B. d. (2026). LegalBench-BR: A benchmark for evaluating large language models on Brazilian legal decision classification.
Chagas, M. A. d. N., Oliveira, A. F. d., Pinto Netto, M., Pontes, M. d. O. E. d., Alves, C. V. P., and Tizzo, L. G. L. (2025). A estrutura da petição inicial na escrita jurídica: uma abordagem propositiva. Revista de Ciências Jurídicas e Empresariais, 26(1):26–37.
Chen, J. and Lv, S. (2022). Long text truncation algorithm based on label embedding in text classification. Applied Sciences, 12(19).
Chi, J., Wang, K., Chen, Y., Lin, X., and Xu, Q. (2025). Activations as features: Probing LLMs for generalizable essay scoring representations. arXiv preprint arXiv:2512.19456.
Conselho Nacional de Justiça (CNJ) (2026). Painel de estatísticas do poder judiciário. Acesso em: 17 jun. 2026.
da Silva Junior, D., de Oliveira, D., and Paes, A. (2025). Evaluating text representations for unsupervised legal semantic textual similarity in Brazilian portuguese. Discover Data, 3.
do Carmo, F. A. (2024). Representações Embeddings Orientadas à Linguagem Jurídica Brasileira. Mestrado em engenharia da computação e sistemas, Universidade Estadual do Maranhão, São Luís/MA.
Garcia, M. H., Couturier, C., Diaz, D. M., Mallick, A., Kyrillidis, A., Sim, R., Ruhle, V., and Rajmohan, S. (2025). Exploring how LLMs capture and represent domain-specific knowledge. arXiv preprint arXiv:2504.16871.
Hakimi Parizi, A., Liu, Y., Nokku, P., Gholamian, S., and Emerson, D. (2023). A comparative study of prompting strategies for legal text classification. In Preoťiuc-Pietro, D., Goanta, C., Chalkidis, I., Barrett, L., Spanakis, G., and Aletras, N., editors, Proceedings of the Natural Legal Language Processing Workshop 2023, pages 258–265, Singapore. Association for Computational Linguistics.
Jayakumar, T., Farooqui, F., and Farooqui, L. (2023). Large language models are legal but they are not: Making the case for a powerful LegalLLM. In Preot,iuc-Pietro, D., Goanta, C., Chalkidis, I., Barrett, L., Spanakis, G., and Aletras, N., editors, Proceedings of the Natural Legal Language Processing Workshop 2023, pages 223–229, Singapore. Association for Computational Linguistics.
Jin, C. and Rinard, M. (2024). Latent causal probing: A formal perspective on probing with causal models of data. arXiv preprint arXiv:2407.13765.
Li, K., Hopkins, A. K., Bau, D., Viégas, F., Pfister, H., and Wattenberg, M. (2024). Emergent world representations: Exploring a sequence model trained on a synthetic task. arXiv preprint arXiv:2210.13382.
Mallen, A., Brumley, M., Kharchenko, J., and Belrose, N. (2024). Eliciting latent knowledge from quirky language models. arXiv preprint arXiv:2312.01037.
Park, K., Choe, Y. J., and Veitch, V. (2024). The linear representation hypothesis and the geometry of large language models. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org.
Peixoto, M., Silva, G., Figueredo, G., Silva, P., and Luz, E. J. S. (2026). Global vs. local sentence embeddings for Brazilian Portuguese: Revisiting monolingual models in the age of foundation models. In Souza, M., de Dios-Flores, I., Santos, D., Freitas, L., Souza, J. W. d. C., and Ribeiro, E., editors, Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1, pages 529–539, Salvador, Brazil. Association for Computational Linguistics.
Sawtell, M., Masterman, T., Besen, S., and Brown, J. (2024). Lightweight safety classification using pruned language models. arXiv preprint arXiv:2412.13435.
Skean, O., Arefin, M. R., Zhao, D., Patel, N., Naghiyev, J., LeCun, Y., and Shwartz-Ziv, R. (2025). Layer by layer: Uncovering hidden representations in language models. arXiv preprint arXiv:2502.02013.
Tupinambá, C. and Teixeira, S. T. (2022). O incidente de resolução de demandas repetitivas na justiça do trabalho. Revista do Tribunal do Trabalho da 2ª Região, 14(28):50–75.
Wilkens, R., Zilio, L., and Villavicencio, A. (2023). Assessing linguistic generalisation in language models: A dataset for Brazilian portuguese. arXiv preprint arXiv:2305.14070.
Wirth, R. and Hipp, J. (2000). CRISP-DM: Towards a standard process model for data mining. Proceedings of the 4th International Conference on the Practical Applications of Knowledge Discovery and Data Mining.
Zhao, H., He, Z., Tang, Y., Yang, F., Payani, A., Liu, D., and Du, M. (2026). Rep2Text: Decoding full text from a single LLM token representation. arXiv preprint arXiv:2511.06571.
Publicado
19/10/2026
Como Citar
SOARES, Pedro Inácio C. de A.; JACOB JR., Antonio F. L.; SILVA, Gustavo S..
Representation Engineering in the Legal Domain: A Comparative Study of Linear and Non-Linear Probing for Initial Petition Classification. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 23. , 2026, Cuiabá/MT.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 3677-3688.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2026.31529.
