In this blog post, I will guide you through the process of creating a unique RAG (Retrieval Augmented Generation) chatbot. Unlike typical chatbots, this one is specifically designed for handling queries related to very specific topics or articles. By utilising the RAG technique, our chatbot can generate responses to complex queries that standard chatbots might find challenging.
Let’s narrow our focus to a topic that typically falls outside the capabilities of a generic chatbot. We are delving into a niche area, such as providing horoscope predictions for individuals born under the Sagittarius sign in the year 2024. Imagine having a chatbot that can answer questions from Sagittarius individuals about what the year 2024 holds for them. While this may seem highly specific, it serves as an example of the type of specialised chatbot we aim to create. With our target set on a fortune-telling chatbot for Sagittarius folks, let’s dive into the tools and techniques we’ll employ to create this bot.
What is LLM and RAG
I believe anyone involved in the tech world has come across the term LLM. With the rise of generative AI, “LLM” has turned into a key term for many developers, particularly those who are interested in or are currently working in the AI field. But what exactly is LLM?
LLM
Large Language Models (LLMs) form a specific category within the broader field of Natural Language Processing (NLP). These models specialise in generating text by analysing and processing vast datasets. Their notable strength lies in their capacity to comprehend and generate language in a broad and versatile manner. LLMs use something called the transformer model. The transformer model is a neural network that learns context and semantic meaning in sequential data like text.
A well-known example of a chatbot using LLM technology is ChatGPT, which incorporates the GPT-3.5 and GPT-4 models.
As for this blog, we will be using a model from MistralAI called Mixtral8x7b. This model is capable of matching or surpassing the performance of Llama 70B and GPT-3.5 and it is available for free use.
Problems with Generic LLM
When it comes to Large Language Models (LLMs), there are two possible scenarios involving topics that they may be less knowledgeable about.
Firstly, the model may straightforwardly admit that it lacks information on a particular subject because it hasn’t been trained on that specific data.
Secondly, there’s the potential for what’s known as “hallucination”, where the model generates responses that are inaccurate or misleading due to its lack of specialised knowledge. This is because generic LLMs are not trained with detailed information in certain areas, such as specific legal rules or medical data, which typically fall outside the scope of a general-purpose LLM’s training data.
