LLM#

A Large Language Model (LLM) is a type of Artificial Intelligence that is designed to understand and generate human language. These models are trained on vast amounts of text data and use complex algorithms, particularly deep learning techniques, to learn the patterns and structures of language.

LLM diagram

Example tasks that can be done with an LLM.#

There are multiple kinds of LLMs, so choosing one may become a complicated task if we are not familiar with them. Moreover, the NLP field moves at a very fast pace, so LLMs become obsolete quite fast (which is actually good because it means something better has been created out there!).

Some important things to consider when selecting an LLM:

  • Proprietary (e.g., OpenAI) vs Open-Source (HuggingFace is the biggest repository): Most proprietary LLMs are good and reliable, although it is rare to find good free options.

  • Locally deployed vs through API: You can use an LLM deployed on your premises, but depending on its size it may be cheaper to pay for an API (The HuggingFace Inference API offers some LLMs for free)

  • Latency: the time the LLM spends to generate the answers.

  • Size: Smaller LLMs (e.g., 2 billion parameters) tend to be more limited than bigger ones (e.g., 70 billion parameters), although they may be enough if you want to deploy locally and for specific tasks.

How to use#

Let’s see how to seamlessly integrate an LLM into our bot. You can also check the LLM bot for a complete example.

We are going to create an LLMOpenAI:

from besser.bot.nlp.llm.llm_openai_api import LLMOpenAI

bot = Bot('example_bot')

gpt = LLMOpenAI(bot=bot, name='gpt-4o-mini')

This LLM can be used within any bot state (in both the body and the fallback body):

def answer_body(session: Session):
    answer = gpt.predict(session.message) # Predicts the output for the given input (the user message)
    session.reply(answer)

There are plenty of possibilities to take advantage of LLMs in a chatbot. The previous is a very simple use case, but we can do more advanced tasks through prompt engineering.

Available LLMs#

BBF comes with LLM wrappers that provide the necessary methods to use them. All LLM wrappers must implement the LLM class, which comes with the following methods to be implemented:

  • initialize(): Initialize the LLM.

  • predict(): Generate the output for a given input.

  • chat(): Simulate a conversation. The LLM receives previous messages to be able to continue with a conversation. Necessary to get chat history from the database. Not mandatory to implement.

  • intent_classification(): Predict the intent of a given message (it allows the LLM Intent Classifier to use this LLM). Not mandatory to implement.

These are the currently available LLM wrappers in BBF:

API References#