Language Models are the new Doom

• Caio Madeira PUCRS
• Maurício Cecílio Magnaguagno PUCRS
• Dalvan Griebler PUCRS

DOI: https://doi.org/10.5753/eramiars.2025.16777

Resumo

As more devices include a display, more probable they may run Doom. Various techniques are used to limit the processing required by the game. A parallel can be drawn with small language models, which are being ported to a wide variety of devices, including legacy ones. While less capable than their large-scale counterparts, these models retain the essence of natural language processing. In this work, we map parallels between Doom and language models, focusing on a TinyLlama model trained on the TinyStories dataset ported to the PlayStation Portable, with hyper-parameter value experimentation.

Referências

Eldan, R. and Li, Y. (2023). Tinystories: How small can language models be and still speak coherent english? arXiv preprint arXiv:2305.07759.

Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023). Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.

Zhang, P., Zeng, G., Wang, T., and Lu, W. (2024). Tinyllama: An open-source small language model. arXiv preprint arXiv:2401.02385.