Public demo

Toolformer: Language Models Can Teach Themselves to Use Tools

Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, Thomas Scialom (2023)

Toolformer shows how a language model can learn to call external tools by adding API-use examples to its own training data.

Why it matters

The paper is an early bridge from language models as text predictors to language models as tool-using systems. It helps explain why future research assistants need retrieval, calculators, citation tools, and document actions around the model.

Highlights

  • [method] Toolformer samples candidate API calls, keeps calls that improve language-model likelihood, and fine-tunes on the filtered examples. — why: this is the self-supervised recipe behind the paper.
  • [method] The model learns to use tools such as calculators, search, translation, and calendars through textual API call patterns. — why: it broadens LLM behavior beyond internal generation.
  • [claim] Tool use can improve performance without requiring large human-labeled tool-use datasets. — why: the data-generation loop is the main scalability argument.
  • [limitation] The model still decides when to call tools through learned text patterns, so reliability and verification remain open problems. — why: tool access does not automatically make outputs trustworthy.

Reader notes

  • Compare with chain-of-thought-prompting: CoT asks the model to reason in text; Toolformer asks the model to act through tools.
  • For SAV, the relevant tool is evidence retrieval: the assistant should know when a paragraph needs source support.
  • Good bridge into the Word add-in idea, where SAV becomes an evidence tool inside the writing environment.

Extracted evidence examples

  • OBJECTIVE: Toolformer enables language models to self-teach tool usage via API calls, overcoming limitations for broad task generality and functional accuracy without degrading performance.
    Therefore, we propose Toolformer, a model that learns to use tools in a novel way, which fulfills the following desiderata:
    Our approach for achieving these goals is based on the recent idea of using large LMs with incontext learning (Brown et al., 2020) to generate entire datasets from scratch (Schick and Schütze, 2021b;Honovich et al., 2022;Wang et al., 2022): Given just a handful of human-written examples of how an API can be used, we let a LM annotate a huge language modelin...
  • OBJECTIVE: The ability to ask external tools for help is investigated as model size is varied.
    We investigate how the ability to ask external tools for help affects performance as we vary the size of our LM.
  • RESEARCH_PROBLEM: Large language models exhibit inherent limitations, including poor arithmetic, factual recall, up-to-date information access, and low-resource language handling.
    However, all of these models have several inherent limitations that can at best be partially addressed by further scaling.
    These limitations include an inability to access up-to-date information on recent events (Komeili et al., 2022) and the related tendency to hallucinate facts (Maynez et al., 2020;Ji et al., 2022), difficulties in understanding low-resource languages (Lin et al., 2021), a lack of mathematical skills to perform precise calculations (Patel et al., 2021) and an...
  • RESEARCH_PROBLEM: Filtering API calls likely to aid token prediction, by including input/output or just input, improves MT tool efficiency in data generation.
    Intuitively, an API call is helpful to M if providing it with both the input and the output of this call makes it easier for the model to predict future tokens, compared to not receiving the API call at all, or receiving only its input.
    This filtering mechanism allows us to generate data more efficiently by focusing our API call generations in places where the MT tool is likely to be helpful.
  • RESEARCH_PROBLEM: The paper investigates the influence of model size on the model's ability to learn tool usage.
    Finally, we investigate how the ability to learn using tools is affected by model size (Section 4.4).
  • RESEARCH_GAP: Self-supervised learning for language model tool use overcomes limitations of annotation-heavy or task-specific methods, enabling broader applicability and addressing LM weaknesses.
    However, existing approaches either rely on large amounts of human annotations (Komeili et al., 2022;Thoppilan et al., 2022) or limit tool use to task-specific settings only (e.g., Gao et al., 2022;Parisi et al., 2022), hindering a more widespread adoption of tool use in LMs.
    • The use of tools should be learned in a self-supervised way without requiring large amounts of human annotations.
  • RESEARCH_GAP: The question answering tool is disabled for Toolformer to avoid making the tasks trivial, especially since the QA system was finetuned on Natural Questions.
    For Toolformer, we disable the question answering tool as 7 An exception to this is if the model's prediction contains an equation (e.g., "The correct answer is 5+3=8"), in which case we consider the first number after the "=" sign to be its prediction.
    this would make solving the tasks trivial, especially given that the underlying QA system was finetuned on Natural Questions.
  • RESEARCH_GAP: Existing approaches to tool use provide information regardless of its usefulness, unlike Toolformer which learns to request only necessary information.
    approaches, additional information is always provided, regardless of whether it is helpful or not.
    In contrast, Toolformer learns for itself to explicitly asks for the right information.