Interacting with Large Language Models Programmatically

Chatting with a Large Language Model

---

Here, we briefly look into building a chatbot using a large language model through an API with the OpenAI Python library. If you have an OpenAI API Key, you can use the OpenAI API. If not, you can use one of the models on HuggingFace.

Stateless conversation

Let’s first create a chatbot that does not remember the past conversation. In this case, a simple function for calling the API is a sufficient starting point. The below code outlines prompting a large language model with the message “Hi there!”.

from openai import OpenAI

client = OpenAI(
  # client initialization
)

MODEL = "mistralai/Mistral-7B-Instruct-v0.3"

def prompt(message):
  response = client.chat.completions.create(
    model=MODEL,
    messages=[
      {"role": "user", "content": message},
    ]
  )

  return response.choices[0].message.content


print(prompt("Hi there!"))

The function could be called until the user types in a specific string such as “exit”. This could be done, for example, with a while-loop as follows, creating a simple chatbot that does not have information on the past conversation.

from openai import OpenAI

client = OpenAI(
  # client initialization
)

MODEL = "mistralai/Mistral-7B-Instruct-v0.3"

def prompt(message):
  response = client.chat.completions.create(
    model=MODEL,
    messages=[
      {"role": "user", "content": message},
    ]
  )

  return response.choices[0].message.content

def main():
  print("Chatbot: Type 'exit' to end the chat.")
  while True:
    user_input = input("You: ")
    print(f"You: {user_input}")
    if user_input.lower() == "exit":
      print("Goodbye!")
      break

    response = prompt(user_input)
    print(f"Chatbot: {response}")

if __name__ == "__main__":
  main()

When run, the program would continue asking for input from the user until the user types in the string “exit”.

The key problem with the above example is that the system is stateless and it does not remember the past conversation. This means that the system does not have the context from the prior conversation and it responds only based on the latest message.

Stateful conversation

To address the problem, we need to make the system stateful. This means that the system needs to have the past conversation as a part of the prompt. To achieve this, we would change the prompt function to take in a list of messages instead of the last message, and adjust the main function to record the conversation.

First, the prompt function would be adjusted to take in a list of messages. The below code outlines how to do this.

def prompt(messages):
  response = client.chat.completions.create(
    model=MODEL,
    messages=messages
  )

  return response.choices[0].message.content

Next, the main function would be adjusted to record the conversation. The below code outlines how to do this.

def main():
  messages = []
  print("Chatbot: Type 'exit' to end the chat.")
  while True:
    user_input = input("You: ")
    print(f"You: {user_input}")
    if user_input.lower() == "exit":
      print("Goodbye!")
      break

    messages.append({"role": "user", "content": user_input})
    response = prompt(messages)
    print(f"Chatbot: {response}")

    messages.append({"role": "assistant", "content": response})

One possible discussion with the chatbot could be as follows. In the following, we first ask for a joke, which the model responds to. Then, we use “Tell another.” as the prompt, which the model responds to with another joke. Finally, we use “exit” to end the conversation.

Chatbot: Type 'exit' to end the chat.
You: Tell me a joke.
Chatbot: Of course! Here's a little classic one for you:

Why don't scientists trust atoms?

Because they make up everything!

Hope that puts a smile on your face! If you need any more, just let me know! :)
You: Tell another.
Chatbot: All right! Here's another one:

Why was the math book sad?

Because it had too many problems.

Now, for something a little more cerebral:

A mathematician, a computer scientist, and a physicist walk into a bar...

The bartender asks them, "What'll it be, gents?"

The mathematician says: "Give me a problem to solve."

You: exit
Goodbye!

The reason why the model was able to respond with a joke when prompted with “Tell another.” is that the model was given the prior past conversation as a context. Due to this, it generated a response based on the entire conversation, interpreting the prompt “Tell another.” in the context of the broader discussion.

Loading Exercise...