With all the attention that these LLMs and ChatGPT have been receiving, a new principle emerges “Garbage in, garbage out.” When using GPT-4 or GPT-3.5, or any other large language model, this concept applies to two places [Prompt Engineering](https://en.wikipedia.org/wiki/Prompt_engineering#:~:text=Prompt%20engineering%20(also%20known%20as,of%20it%20being%20explicitly%20given.) and Fine-tuning. We should also consider when these models were trained to verify how old or new the data they provide us is.

Most businesses today sit within corporate data sources – inside a firewall or outside and not in a public domain internet. If we can leverage LLMs on this data, then new possibilities emerge.

This article will discuss building a chatbot with a custom knowledge base in 30 minutes to have your AI answer questions about your data.

Pre-Requisites for Creating a Knowledge Bot

We must consider how much data we have to create a custom knowledge base. In this blog, we will create an assistant using PubNub documentation. There is a lot of documentation, so we should consider using a production service such as Vectara, Pinecone, or Weaviate to manage our vector embeddings. You can use LangChain to create a local Vector Database for smaller amounts of data. LangChain also supports mapping your custom embeddings to your LLM models through a semantic or similarity search.

For this blog, we will go through setting up a Vector Database on Vectara, as it provides a simple drag-and-drop solution for all of your company data. The best way to interact with Vectara is to utilize PubNub Functions, which provides a serverless JavaScript container that runs whenever a pre-defined event occurs. You will have customizability over when the function runs and gets fired, being able to adjust how your AI Knowledge Bot operates.

Vectara will use your dataset to index your data into multiple embeddings. When you pass a text input or user input Vectara will run a Semantic Search on your data and summarize the results it has found, providing an answer to your question using your custom data. This will give you the relevant information needed. Vectara supports multiple file types such as TXT, HTML, PDF, and Word Files. Gather all the documents you want to upload and add them to an individual folder.

How to set up Vectara

Set up Vectara is as follows:

Sign up or log in to Vectara
Once you are on the dashboard, click Create corpus
Once you give the Corpus a name and a description under the Data Ingestion header drag and drop the files you want your LLM to know about into the Upload Files section. After your files are uploaded, you can check your corpus ID at the top of the webpage as you will need it for the request we are about the write.
Click on your email in the top right corner and save your Customer ID for later
Select API Keys and create an API key for your corpus by selecting your corpus in the drop-down menu. Save your API key for later.

High-Level Architecture

The architecture will be structured as follows:

The chat application will use PubNub to send and receive messages
A PubNub Function will listen to these messages on a specific channel
A PubNub signal will be fired to let the user know when the AI is thinking and when it is done.
The message will then be forwarded to Vectara using the Vectara Rest API
The PubNub Function will then parse 1 of many results out of the response from Vectara
The response will then be published on a channel associated with your chatbot

Configuring your PubNub Function

Navigate to the admin dashboard
Select functions on the left-hand menu and click on the appropriate key set you would like to use
Select + Create Module and enter a module name and description
Select the module you just created and click + Create a Function
Give the Function a name, such as Vectara Query and select After Publish or Fire in the drop-down menu. This function will fire after the message has been published to the relevant channel, in this case, docs-pubnub-ai
Set the channel name to docs-pubnub-ai
Click on My Secrets and create a secret called VECTARA_API_KEY and CUSTOMER_ID

Here is the code for querying the Vectara Database from a PubNub Function

The code snippet for the PubNub Function is defined as follows:

const Promise = require('promise');

//
// API Key For OpenAI
// **Add your Vectara API Key and Customer ID to MY SECRETS (Left Panel)**
//
let VECTARA_API_KEY = null;
let CUSTOMER_ID = null;
let channel = "docs-pubnub-ai-response";
function getVectaraKey() {
   // Use cached key
   if (VECTARA_API_KEY && CUSTOMER_ID) {
       return new Promise(resolve => resolve(VECTARA_API_KEY));
   }
   // Fetch key from vault
   return vault.get("VECTARA_API_KEY").then(apikey => {
       return vault.get("CUSTOMER_ID").then(customer_id => {
            VECTARA_API_KEY = apikey;
            CUSTOMER_ID = customer_id;
           return new Promise(resolve => resolve(VECTARA_API_KEY));
       });
   });
}

//
// Import Modules
//
const xhr = require('xhr');
const vault = require('vault');
const pubnub = require('pubnub');

//
// Main
//
export default (request) => {
   // Input data for Vectara
   let message = request.message.text;
   let signalMessage = { message: { type: "typing_on" }, channel: channel };
   return pubnub.signal(signalMessage).then((_) => {
       return getVectaraKey().then(_ => {
           return vectara(message).then(response => {
               return pubnub.publish({
                   channel: channel,
                   message: {
                       id: "PubNubAI",
                       type: "default",
                       text: response
                   },
               }).then((publishResponse) => {
                   console.log(publishResponse);
                   message = { message: { type: "typing_off" }, channel: channel };
                   return pubnub.signal(message).then((signalOffResponse) => {
                       console.log(signalOffResponse);
                       return request.ok();
                   });
               }).catch((err) => {
                   console.error(err);
               });
           });
       });
   });
};

//
// API Call to Vectara asking the AI a question
//
async function vectara(question) {
   console.log("SENDING REQUEST TO VECTARA");
   const url = "https://api.vectara.io/v1/query";
   const body = JSON.stringify({
       "query": [
           {
           "query": question,
           "start": 0,
           "numResults": 10,
           "contextConfig": {
               "charsBefore": 0,
               "charsAfter": 0,
               "sentencesBefore": 2,
               "sentencesAfter": 2,
               "startTag": "<b>",
               "endTag": "</b>"
           },
           "corpusKey": [
               {
               "customerId": CUSTOMER_ID,
               "corpusId": 1,
               "semantics": "DEFAULT",
               "dim": [
                   {
                   "name": "string",
                   "weight": 0
                   }
               ],
               "lexicalInterpolationConfig": {
                   "lambda": 0.025
               }
               }
           ],
           "rerankingConfig": {
               "rerankerId": 272725717
           },
           "summary": [
               {
               "summarizerPromptName": "",
               "maxSummarizedResults": 5,
               "responseLang": "eng",
               "summarizerPromptId": 1,
               "debug": false,
               "responseChars": 0
               }
           ]
           }
       ]
   });

   const http_options = {
       'method': 'POST',
       'timeout': 20000,
       'headers': {
           'Content-Type': 'application/json',
           'customer-id': CUSTOMER_ID,
           'x-api-key': VECTARA_API_KEY
        },
       'body': body
   };

   xhr.timeout = 100000;

   // Send API Request to OpenAI GPT Model
   const response = await xhr.fetch(url, http_options).then(res => res.json());

   // Parse the response and get the summary
   const summary = response.responseSet[0].summary[0].text;

   return summary;
}

Connecting the PubNub Function to your UI

To connect the PubNub function to a UI following code defined above using one of the many SDKs that PubNub provides. Publish/subscribe to the channel pubnub-docs-ai and wait for the Vectara query to finish running after utilizing the PubNub Function above. Connecting a Typing Indicator to listen for PubNub signals on the channel pubnub-docs-ai will allow the user to see when the PubNub Knowledge bot is thinking, adding a smoother end-user experience.

The code for connecting the PubNub function in React:

const pubnub = new PubNub({
   publishKey: process.env.NEXT_PUBLIC_PUBLISH_KEY!,
   subscribeKey: process.env.NEXT_PUBLIC_SUBSCRIBE_KEY!,
   userId: "You",
 });

 const publishChannel = "docs-pubnub-ai";

 const sendMessage = (message: string) => {
   if (message) {
     pubnub.publish({
         channel: publishChannel, message: {
           text: message
       }});
   }
 };

In Summary

Using PubNub Functions along with any Vector Database or Vector Store is a very quick and production-ready way to create your own AI Knowledge Bot. Not only could you utilize Vectara this way but also Pinecone, Weaviate or any other production Vector Database. With PubNub Functions it is easy to host and control how the message is being sent and when it is sent to enhance your Vector Databases functionality.

Sign up for our admin dashboard to start configuring your PubNub keyset. Also, check the number of tutorials and blogs we have for your specific use case.

How can PubNub help you?

This article was originally published on PubNub.com

Our platform helps developers build, deliver, and manage real-time interactivity for web apps, mobile apps, and IoT devices.

The foundation of our platform is the industry's largest and most scalable real-time edge messaging network. With over 15 points-of-presence worldwide supporting 800 million monthly active users, and 99.999% reliability, you'll never have to worry about outages, concurrency limits, or any latency issues caused by traffic spikes.

Experience PubNub

Check out Live Tour to understand the essential concepts behind every PubNub-powered app in less than 5 minutes

Get Setup

Get Started

The PubNub docs will get you up and running, regardless of your use case or SDK

Build a LLM Chatbot with a Custom Knowledge Base