Introduction

One of the new skills data scientists and other tech professionals are learning is how to build a chatbot. The rise of large language models have increased the effectiveness of chatbots and thus many industries are adopting them for their employees and their customers to use. There are endless use cases for large language models. Some of the popular use cases people are developing are:

Simple chatbots with no added functionality
Chatbots which allow users to query/summarise documents like PDFs, CSVs or even Graph databases
Chatbots which can work with images and video
and more

I was interested in the first two and started working on them last year (it’s taken a while to get this finished).

In a nutshell, instead of just providing a question to a chatbot, a technique called retrieval augmented generation (RAG) enables you to provide context + question. The ability to provide context means that you are able to ask questions about your own personal data. This leads to two interesting use-cases:

Question-answering with your own data
- E.g. you are a recruiter and have received a CV from an applicant. You could ask questions like:
  
  What work experience does this candidate have?
  
  Does this applicant have experience with Python?
  
  What contact details has this applicant provided?
Content summarisation
- Depending on how you set up your application, RAG can be used to summarise long documents by retrieving relevant information from the source and generating a concise summary. This is particularly useful for quickly extracting key insights from lengthy articles or reports.

How do we build a chatbot like this?

Firstly, let’s run through the necessary tooling for this. We need 3 main tools:

A large language model - provided by HuggingFace
A framework for interacting with large language models - provided by LangChain
A front end for our application - provided by Streamlit

and if you want a chatbot capable of querying documents, you’ll need

A vector database for storing vectorised chunks of your document - provided by FAISS