Working with APIs
Learning Objectives
- You know of some of the challenges and solutions related to working with APIs using large language models.
Much of contemporary software development relies on building on existing libraries and applications. The libraries and applications are used through their application programming interfaces (APIs). For example, the Python requests library provides a set of functions that can be used to make HTTP requests. The Requests API outlines the functions that the library provides to the user.
The endpoints that are used through the requests library are also APIs. For example, the Simple Joke API at https://simple-joke-api.deno.dev/random provides random jokes in a JSON format — the API at https://simple-joke-api.deno.dev/random could be used by writing a program that follows the API of the requests.
Applications are typically composed of layers of APIs, where the API of one layer is used by the layer above it. For example, the Simple Joke API uses a file system API to read jokes from the file system, and provides an HTTP API that others can use to retrieve jokes from the API.
The problem with APIs is that they are often poorly documented. Even if documentation exists, it might be out of date. This is especially true for APIs that are developed by small teams or individuals, who may not have the time or resources to maintain the documentation. Similarly, many APIs and their (lacking) documentation are not included in the training data of large language models, which means that creating code for using them might take effort.
APIs and their documentation are not always included in the training data of large language models.
When working with APIs, at least in the case of smaller HTTP APIs, one can provide examples of inputs and outputs to the LLM, and ask it to generate the functionality for using the API. This is similar to the approach used for source code generation, which we discussed earlier.
The example below outlines a prompt where the user asks the LLM to generate a Python wrapper for the Simple Joke API.
The API endpoint at https://simple-joke-api.deno.dev/random provides random jokes that are formatted as follows.
{"setup":"Why do programmers write code in the dark?","punchline":"Because light attracts bugs."}
Please implement a Python wrapper for the API that uses the requests library.
Certainly! Here's a simple Python wrapper for the given joke API using the requests library:
import requests
class SimpleJokeAPI:
BASE_URL = "https://simple-joke-api.deno.dev"
@classmethod
def get_random_joke(cls):
"""Retrieve a random joke."""
response = requests.get(f"{cls.BASE_URL}/random")
if response.status_code == 200:
joke_data = response.json()
return joke_data
else:
response.raise_for_status()
# Usage example
if __name__ == "__main__":
joke = SimpleJokeAPI.get_random_joke()
print(joke['setup'])
print(joke['punchline'])
...
The generated code is a good starting point, although it does not include error handling functionality. Like before, we could continue prompting the model for additional improvements to the code.
A more generic approach would be to first fine-tune the large language model for providing API-related responses in a specific style from prompts formatted in a specific way. Once the large language model would be fine-tuned to produce specific responses, the model could be given information about APIs that the model does not have in its data in the specific format that the model was fine-tuned with. This way, the model could more accurately respond to prompts also when the APIs that are a part of the prompt are not known to the model.
This is what, for example, the Gorilla project at Berkeley University has been tackling. For additional details, see “Gorilla: Large Language Model Connected with Massive APIs”.
The Gorilla project also provides a leaderboard highlighting the performance of large language models in calling functions.
At the same time, there are also APIs that some large language models are already aware of without any additional fine-tuning or without providing additional context in the prompt. As an example from above, when asking large language models for code that uses the requests library, the model was already aware of the the library and its APIs, and could provide us working code.