This is a submission for the Open Source AI Challenge with pgai and Ollama

What I Built

This is a conversational RAG app where all the RAG pipelines are entirely built in PostgreSQL procedures using PL/pgSQL!

The idea behind this app stems from my master's thesis work. I have to do systematic literature review and doing it manually is boring. So, I created this small app so I can just upload the full text paper and chat with it, create summaries, highlights, and key results. Massively streamlining the process of systematic literature review.

Of course, this app would work with any kind of data, we just need to change the system prompt a bit!😏

Key Features:

• Summarize research papers (journal articles, conference papers, etc.)
• Create highlights/key insights
• Automatic processing using pgai Vectorizer
• Chat with independent paper
• Save multiple chat sessions

Initially I want to fully use Ollama, but pgai Vectorizer currently do not support Ollama, so I opted to use Open AI.

Demo

Demo video here

Tools Used

• TimescaleDB as the main database to store the documents and its embeddings
• pgai to access Open AI services in database
• pgvector to store document embeddings
• pgvectorscale to create indexes on the embeddings
• pgai Vectorizer to automatically create embeddings from the uploaded papers

Prize Categories

Vectorizer Vibe, All the Extensions

Tech Stack

• PostgreSQL (TimescaleDB)
• Minio
• Remix

So little tech stack for a RAG app😊 We can make it smaller by storing blobs in Postgres but I don't like that idea.

Conversational RAG in PL/pgSQL

In this SQL script I implemented two Postgres function to build the conversational RAG pipeline. This is the heart and soul of this app.

I got the idea from this LangChain tutorial.

CREATE FUNCTION contextualize_question(p_session_id VARCHAR(36), p_query TEXT) RETURNS TEXT

CREATE PROCEDURE chat_with_paper(p_session_id VARCHAR(36), p_chat_content TEXT)

I never thought I would be writing LLM chain/pipeline using SQL instead of Haystack, LangChain, or LlamaIndex, but here we are!

It's crazy what pgai could bring in the future for LLM in databases.

Final Thoughts

This has been an interesting journey because the idea of running LLM directly in database is really weird at first. But after learning it for the last 2 days, I found it really interesting and could possibly revolutionize data mining pipelines for non-AI engineers. I imagine data analysts and researchers could easily get insights from database systems without major changes to existing systems.

One of my favorite experiences in this project is I learned how to write Postgres procedures and functions using PL/pgSQL. It was a really interesting journey especially to write LLM apps that used to be written using LangChain, Haystack, or LammaIndex now I implemented it using pure PL/pgSQL to build a conversational RAG.