Large language models (LLMs) are distinguished by their immense capabilities in solving complex tasks, ranging from quantitative reasoning to natural language understanding. However, LLMs sometimes suffer from hallucinations, leading to the generation of statements that are plausible yet incorrect. This hinders the current large models’ utility in scientific discovery. Here, we present FunSearch (short for Function Search), an evolutionary procedure based on linking a pre-trained LLM with a systematic evaluator. We demonstrate the effectiveness of this approach in surpassing the best-known results in significant problems, pushing the boundaries of the current LLM-based methodology. By applying FunSearch to a central problem in extremal set theory – the hat set problem – we discovered new constructions of large hat sets that exceed those previously known, both in finite and infinite dimensions. This represents the first known discoveries made in open problems using LLMs. We showcase the generality of FunSearch by applying it to an algorithmic problem, which is internet box packing, where we find new techniques that enhance widely used foundations. Unlike most approaches in computational research, FunSearch seeks programs that describe how to solve a problem, rather than what the solution is. In addition to being an effective and scalable strategy, the discovered programs are more interpretable than raw solutions, enabling feedback loops between domain experts and FunSearch, and facilitating the deployment of such programs in practical real-world applications.
Introduction
Large language models (LLMs) are very powerful at solving complex tasks, including quantitative reasoning and natural language understanding. However, LLMs may sometimes suffer from hallucinations, leading to the generation of plausible yet incorrect statements. This hinders the utility of current large models in scientific discovery. In this study, we present FunSearch, an evolutionary procedure that employs a pairing technique between a pre-trained LLM and a systematic evaluator. We clarify the effectiveness of this approach in surpassing the best-known results in significant problems, pushing the boundaries of the current LLM-based methodology. We also apply FunSearch to a central problem in extremal set theory – the hat set problem – where we discover new constructions of large hat sets that surpass those previously known, both in finite and infinite dimensions. This is the first discovery made in known open problems using LLMs. We also showcase the generality of FunSearch by applying it to an algorithmic problem, which is internet box packing, where we find new techniques that improve widely used foundations. Instead of seeking solutions, FunSearch looks for programs that describe how to solve the problem. Additionally, being an effective and scalable strategy, the discovered programs are more interpretable than raw solutions, enabling feedback iterations between domain experts and FunSearch, and facilitating the deployment of such programs in practical real-world applications.
Conclusions
This study concludes that using large language models in program search can lead to new mathematical discoveries. FunSearch was applied to a central problem in extremal set theory and new constructions of large hat sets were discovered. FunSearch was also applied to an algorithmic problem and new techniques were found that improve widely used foundations. These discoveries can be utilized in real-world applications and enable interaction between domain experts and FunSearch. It is expected that this study will contribute to the advancement of the field and enhance scientific discovery.
Source:
Leave a Reply