Researchers from the Japan Advanced Institute of Science and Technology have developed a method to improve the accuracy of large language models (LLMs). The method, called Answer-Prefix Generation (ANSPRE), gives LLMs part of the answer in advance. “ANSPRE can improve the generation quality of LLMs, allowing them to output the exact answer phrase and produce reliable confidence scores,” says project lead Nguyen Le Minh.
Large language models have become increasingly popular, but they come with issues. They can be verbose and can generate answers that sound plausible but are inaccurate.
Enhancing LLMs with ANSPRE method
ANSPRE addresses these issues by giving the LLM a head-start with an answer prefix, prompting it to fill in the blanks. Nguyen explains with an example: “Consider the question, ‘What gambling game, requiring two coins to play, was popular in World War I?’ An answer prefix for this question could be, ‘The gambling game requiring two coins to play that was popular in World War I was ___.’ Since most LLMs are trained with causal language modeling, using the answer prefix would allow the LLM to generate the exact answer phrase in place of the blank.
Rather than creating the prefixes manually, ANSPRE uses a few-shot examples to generate a prefix for a given question. The system then utilizes an existing retriever to pull relevant content from a knowledge base, which is combined with the question and the answer prefix to form a detailed prompt for the target LLM.
The team has also developed an extended version called Self-Reflective Answer-Prefix Generation (SELF-ANSPRE), which further improves results by ranking responses based on confidence scores and the usefulness of the information retrieved from the knowledge base. Our method can lead to more concise and accurate question answering in critical fields like medical diagnosis, legal assistance, and education, and improve customer support,” says Prof. Nguyen. The team’s work was presented at the 27th European Conference on Artificial Intelligence and is available under open-access terms in the Frontiers in Artificial Intelligence and Applications.
Noah Nguyen is a multi-talented developer who brings a unique perspective to his craft. Initially a creative writing professor, he turned to Dev work for the ability to work remotely. He now lives in Seattle, spending time hiking and drinking craft beer with his fiancee.
