Workshop: Evaluating LLM-Written Abstracts with ChainForge

• Date: March 19th, 2026
• Time: 9h-10h15 (Montréal) / 14h-15h15 (Paris)
• Speaker: Gauransh Kumar (Université de Montréal)

Part of the new series of workshop-discussions, "Understanding and Evaluating Complex Automation Systems for Journals", this second workshop introduces ChainForge through a hands-on demonstration of prompt testing and evaluation for scientific abstract generation. G. Kumar will begin with a brief overview of ChainForge as a tool for comparing and analyzing LLM outputs, then show how a simple RAG pipeline can be used to generate abstracts from papers in PDF format. Finally, G. Kumar will demonstrate how these outputs can be evaluated against prompt constraints such as word count, grammatical quality, and relevance to the original abstract. The workshop aims to provide a practical example of how quantitative and qualitative evaluation can be combined in a clear and reproducible workflow.