Workshop with Tiberio Uricchio

2024-11-25

The workshop was held with the participation of Tiberio Uricchio (Associate Professor at the University of Pisa, Department of Information Engineering) and the team of Stylo, a semantic text editor developed as part of the Revue3.0 project. The discussion, held in the CRIHN's meeting rooms, focused on exploring potential experiments to integrate sparse autoencoders into Stylo. These technologies could be implemented, for example, to automate the suggestion of bibliographies, literature reviews, or disciplinary summaries within Stylo, while ensuring independence from prebuilt, proprietary LLMs.

Designed to regulate the nodes activated during the unsupervised learning of autoencoders, sparse autoencoders could allow manipulation of certain features of the input text which, in the case of Stylo, would correspond to the content inserted by article authors. Implementing this technology could also produce an output text enriched with references or summaries aligned with the textual features specified by the author, as well as provide a level of analysis on the semantic role of the nodes within the hidden layers of the autoencoders.

During this initial discussion, Uricchio briefly presented tools that could serve as a starting point for this type of experimentation, including Gemma Scope, a sparse autoencoder aimed at the interpretability of the Gemma AI model, SAELens, a codebase for training new sparse autoencoders, and Prism, which allows visualization and manipulation of the features of input texts.

Further experimentation and discussion workshops on potential AI technology integrations will be held in January 2025. These experiments should be accompanied by a theoretical reflection on the potential role of AI in humanities research. The workshop on Friday, November 29, 2024, with [Juan Luis Gastaldi](https://Juan Luis Gastaldi), aims to initiate a preliminary discussion dedicated to this type of theoretical reflection.