Surprise! How Do Science Fiction Stories Challenge Predictive AI Readers?
Resumo
Science fiction repeatedly violates expectation while preserving enough structure for strange worlds and technologies to become intelligible. We convert ten public-domain stories into token-level surprisal trajectories produced by a frozen GPT-2 predictive reader and compare each Human original with a conservative GPT-5.5 AI-polished descendant and an independently AI-written same-topic control. All three conditions receive identical whitespace canonicalization before tokenization. Under this common preprocessing, Human and AI-polished texts become nearly indistinguishable: mean surprisal differs by only 0.013 nats, their 99th-percentile surprisals do not differ detectably, and their multifractal scaling functions almost coincide. By contrast, AI-written controls retain much lower mean surprisal but a heavier low-probability tail, markedly narrower singularity spectra, strong compression relative to iterative amplitude-adjusted Fourier-transform surrogate nulls, and distinct entropy–complexity geometry. The results show that conservative AI polishing can preserve both local predictability and multiscale surprisal organization, whereas independent same-topic generation changes the organization of the predictive signal itself.
Referências
Bak, P., Tang, C., and Wiesenfeld, K. (1987). Self-organized criticality: An explanation of the 1/f noise. Physical Review Letters, 59(4):381–384.
Beggs, J. M. and Plenz, D. (2003). Neuronal avalanches in neocortical circuits. Journal of Neuroscience, 23(35):11167–11177.
Boyd, R. L., Blackburn, K. G., and Pennebaker, J. W. (2020). The narrative arc: Revealing core narrative structures through text analysis. Science Advances, 6(32):eaba2196.
Clark, E., Ross, A. S., Tan, C., Ji, Y., and Smith, N. A. (2018). Creative writing with a machine in the loop: Case studies on slogans and stories. In Proceedings of the 23rd International Conference on Intelligent User Interfaces, pages 329–340.
Frank, S. L., Otten, L. J., Galli, G., and Vigliocco, G. (2013). Word surprisal predicts N400 amplitude during reading. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 878–883.
Gehrmann, S., Strobelt, H., and Rush, A. M. (2019). GLTR: Statistical detection and visualization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 111–116.
Hale, J. (2001). A probabilistic Earley parser as a psycholinguistic model. In Second Meeting of the North American Chapter of the Association for Computational Linguistics. DOI: 10.3115/1073336.1073357.
Hesse, J. and Gross, T. (2014). Self-organized criticality as a fundamental property of neural systems. Frontiers in Systems Neuroscience, 8:166.
Ihlen, E. A. F. (2012). Introduction to multifractal detrended fluctuation analysis in Matlab. Frontiers in Physiology, 3:141.
Jockers, M. L. (2013). Macroanalysis: Digital Methods and Literary History. University of Illinois Press.
Kantelhardt, J. W., Zschiegner, S. A., Koscielny-Bunde, E., Havlin, S., Bunde, A., and Stanley, H. E. (2002). Multifractal detrended fluctuation analysis of nonstationary time series. Physica A, 316(1–4):87–114.
Kutas, M. and Federmeier, K. D. (2011). Thirty years and counting: Finding meaning in the N400 component of the event-related brain potential (ERP). Annual Review of Psychology, 62:621–647.
Kutas, M. and Hillyard, S. A. (1980). Reading senseless sentences: Brain potentials reflect semantic incongruity. Science, 207(4427):203–205.
Lee, M., Liang, P., and Yang, Q. (2022). CoAuthor: Designing a human–AI collaborative writing dataset for exploring language model capabilities. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, Article 388, pages 1–19.
Levy, R. (2008). Expectation-based syntactic comprehension. Cognition, 106(3):1126–1177.
Li, J., Jia, R., He, H., and Liang, P. (2018). Delete, retrieve, generate: A simple approach to sentiment and style transfer. In Proceedings of NAACL-HLT, Volume 1, pages 1865–1874.
López-Ruiz, R., Mancini, H. L., and Calbet, X. (1995). A statistical measure of complexity. Physics Letters A, 209(5–6):321–326.
Mao, X., Shang, P., Wang, J., and Ma, Y. (2018). Characterizing time series by extended complexity–entropy curves based on Tsallis, Rényi, and power spectral entropy. Chaos, 28(11):113106. DOI: 10.1063/1.5038758.
Michaelov, J. A., Bardolph, M. D., Van Petten, C. K., Bergen, B. K., and Coulson, S. (2024). Strong prediction: Language model surprisal explains multiple N400 effects. Neurobiology of Language, 5(1):107–135.
Mitchell, E., Lee, Y., Khazatsky, A., Manning, C. D., and Finn, C. (2023). DetectGPT: Zero-shot machine-generated text detection using probability curvature. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of PMLR, pages 24950–24962.
Peng, C.-K., Buldyrev, S. V., Havlin, S., Simons, M., Stanley, H. E., and Goldberger, A. L. (1994). Mosaic organization of DNA nucleotides. Physical Review E, 49(2):1685–1689.
Plenz, D., Ribeiro, T. L., Miller, S. R., Kells, P. A., Vakili, A., and Capek, E. L. (2021). Self-organized criticality in the brain. Frontiers in Physics, 9:639389.
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. (2019). Language models are unsupervised multitask learners. Technical report, OpenAI.
Reagan, A. J., Mitchell, L., Kiley, D., Danforth, C. M., and Dodds, P. S. (2016). The emotional arcs of stories are dominated by six basic shapes. EPJ Data Science, 5(1):31.
Rosso, O. A., Larrondo, H. A., Martin, M. T., Plastino, A., and Fuentes, M. A. (2007). Distinguishing noise from chaos. Physical Review Letters, 99(15):154102.
Salicchi, L. and Hsu, Y.-Y. (2025). Not every metric is equal: Cognitive models for predicting N400 and P600 components during reading comprehension. In Proceedings of the 31st International Conference on Computational Linguistics, pages 3648–3654.
Schreiber, T. and Schmitz, A. (2000). Surrogate time series. Physica D, 142(3–4):346–382.
Smith, N. J. and Levy, R. (2013). The effect of word predictability on reading time is logarithmic. Cognition, 128(3):302–319.
Tsallis, C. (1988). Possible generalization of Boltzmann–Gibbs statistics. Journal of Statistical Physics, 52(1–2):479–487.
Zhang, X. and Lapata, M. (2017). Sentence simplification with deep reinforcement learning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 584–594.
Beggs, J. M. and Plenz, D. (2003). Neuronal avalanches in neocortical circuits. Journal of Neuroscience, 23(35):11167–11177.
Boyd, R. L., Blackburn, K. G., and Pennebaker, J. W. (2020). The narrative arc: Revealing core narrative structures through text analysis. Science Advances, 6(32):eaba2196.
Clark, E., Ross, A. S., Tan, C., Ji, Y., and Smith, N. A. (2018). Creative writing with a machine in the loop: Case studies on slogans and stories. In Proceedings of the 23rd International Conference on Intelligent User Interfaces, pages 329–340.
Frank, S. L., Otten, L. J., Galli, G., and Vigliocco, G. (2013). Word surprisal predicts N400 amplitude during reading. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 878–883.
Gehrmann, S., Strobelt, H., and Rush, A. M. (2019). GLTR: Statistical detection and visualization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 111–116.
Hale, J. (2001). A probabilistic Earley parser as a psycholinguistic model. In Second Meeting of the North American Chapter of the Association for Computational Linguistics. DOI: 10.3115/1073336.1073357.
Hesse, J. and Gross, T. (2014). Self-organized criticality as a fundamental property of neural systems. Frontiers in Systems Neuroscience, 8:166.
Ihlen, E. A. F. (2012). Introduction to multifractal detrended fluctuation analysis in Matlab. Frontiers in Physiology, 3:141.
Jockers, M. L. (2013). Macroanalysis: Digital Methods and Literary History. University of Illinois Press.
Kantelhardt, J. W., Zschiegner, S. A., Koscielny-Bunde, E., Havlin, S., Bunde, A., and Stanley, H. E. (2002). Multifractal detrended fluctuation analysis of nonstationary time series. Physica A, 316(1–4):87–114.
Kutas, M. and Federmeier, K. D. (2011). Thirty years and counting: Finding meaning in the N400 component of the event-related brain potential (ERP). Annual Review of Psychology, 62:621–647.
Kutas, M. and Hillyard, S. A. (1980). Reading senseless sentences: Brain potentials reflect semantic incongruity. Science, 207(4427):203–205.
Lee, M., Liang, P., and Yang, Q. (2022). CoAuthor: Designing a human–AI collaborative writing dataset for exploring language model capabilities. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, Article 388, pages 1–19.
Levy, R. (2008). Expectation-based syntactic comprehension. Cognition, 106(3):1126–1177.
Li, J., Jia, R., He, H., and Liang, P. (2018). Delete, retrieve, generate: A simple approach to sentiment and style transfer. In Proceedings of NAACL-HLT, Volume 1, pages 1865–1874.
López-Ruiz, R., Mancini, H. L., and Calbet, X. (1995). A statistical measure of complexity. Physics Letters A, 209(5–6):321–326.
Mao, X., Shang, P., Wang, J., and Ma, Y. (2018). Characterizing time series by extended complexity–entropy curves based on Tsallis, Rényi, and power spectral entropy. Chaos, 28(11):113106. DOI: 10.1063/1.5038758.
Michaelov, J. A., Bardolph, M. D., Van Petten, C. K., Bergen, B. K., and Coulson, S. (2024). Strong prediction: Language model surprisal explains multiple N400 effects. Neurobiology of Language, 5(1):107–135.
Mitchell, E., Lee, Y., Khazatsky, A., Manning, C. D., and Finn, C. (2023). DetectGPT: Zero-shot machine-generated text detection using probability curvature. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of PMLR, pages 24950–24962.
Peng, C.-K., Buldyrev, S. V., Havlin, S., Simons, M., Stanley, H. E., and Goldberger, A. L. (1994). Mosaic organization of DNA nucleotides. Physical Review E, 49(2):1685–1689.
Plenz, D., Ribeiro, T. L., Miller, S. R., Kells, P. A., Vakili, A., and Capek, E. L. (2021). Self-organized criticality in the brain. Frontiers in Physics, 9:639389.
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. (2019). Language models are unsupervised multitask learners. Technical report, OpenAI.
Reagan, A. J., Mitchell, L., Kiley, D., Danforth, C. M., and Dodds, P. S. (2016). The emotional arcs of stories are dominated by six basic shapes. EPJ Data Science, 5(1):31.
Rosso, O. A., Larrondo, H. A., Martin, M. T., Plastino, A., and Fuentes, M. A. (2007). Distinguishing noise from chaos. Physical Review Letters, 99(15):154102.
Salicchi, L. and Hsu, Y.-Y. (2025). Not every metric is equal: Cognitive models for predicting N400 and P600 components during reading comprehension. In Proceedings of the 31st International Conference on Computational Linguistics, pages 3648–3654.
Schreiber, T. and Schmitz, A. (2000). Surrogate time series. Physica D, 142(3–4):346–382.
Smith, N. J. and Levy, R. (2013). The effect of word predictability on reading time is logarithmic. Cognition, 128(3):302–319.
Tsallis, C. (1988). Possible generalization of Boltzmann–Gibbs statistics. Journal of Statistical Physics, 52(1–2):479–487.
Zhang, X. and Lapata, M. (2017). Sentence simplification with deep reinforcement learning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 584–594.
Publicado
19/10/2026
Como Citar
FRIGORI, Rafael B.; COSTA, Ariadne A..
Surprise! How Do Science Fiction Stories Challenge Predictive AI Readers?. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 23. , 2026, Cuiabá/MT.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 289-300.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2026.29931.
