Last week, I had the pleasure of attending the Mindstone.ai meetup in London, and it was an incredible experience! 🎉 The event featured three insightful talks, each packed with valuable information and forward-thinking perspectives.

Erik Schmiegelow, the CEO of Hivemind Technologies AG, delivered an eye-opening session on the practical applications of large language models (LLMs) in matching and ranking.

Here are some key takeaways:

Matching and Ranking with LLMs

Erik delved into the mechanics of how LLMs can be leveraged to enhance the accuracy and efficiency of matching algorithms. This has significant implications for search engines, recommendation systems, and more.

Practical Applications

Real-world use cases were discussed, showcasing how LLMs are being used to solve complex problems in various industries, from e-commerce to information retrieval.

Implementation Strategies

Erik shared best practices for implementing these models in production environments, including tips on optimizing performance and maintaining scalability.

Key Points from the Slides:

RAG Architecture

• Retrieval Augmented Generation (RAG) separates the model from the relevant internal data to tackle issues.
• Step 1: Data is chunked using an embedding model and stored in a vector store.
• Step 2: The model extracts relevant data from the vector store to create answers.

Typical LLM Tasks

• Classification: Assign a set of classes to the input.
• Summarisation: Summarise long-form input.
• Entity Extraction: Extract attributes from unstructured input.

Programmatic Approach with LangChain

• LangChain framework facilitates chaining LLM components such as prompt templates, agents, query endpoints, and inference endpoints.
• LangChain Agents: Enable LLMs to execute tasks like calculations, database queries, and web lookups.

Common Approach

• Use general-purpose LLMs (ChatGPT, Claude) with fine-tuned prompts.
• Benefit: Minimal operational effort required.

Why it’s not Efficient

• Cost: Inference endpoints charge based on tokens.
• Latency: Large models can slow performance.
• Other Concerns: Privacy, IP access, and compliance issues.

My Notes from the Talk:

• Data Privacy: LLMs can recognize which data is private and what is not, addressing some data privacy concerns.
• LangChain: Known as a pragmatic approach to thinking about large LLMs, allowing the creation of a chain of execution. LangChain's "agents" can handle complex questions and enhance LLM capabilities.
• Hallucinations: LLMs can’t reason and try to create something that is "most likely" true rather than actually true, akin to some management consultants (Erik’s joke).
• Composable Architecture: Allows the use of smaller models for specific tasks.
• CV Matching App: Helps process resumes, identifies candidate qualities, generates job offers, and solves problems that rule-based systems can't.
• Audience Questions:
  • Bias Guarding: Models can be protected from bias through normalization processes.
  • Open Source: Many open-source projects and communities, particularly around green software, are good entry points to LLMs.
  • Fighting Bias: LLMs can work on non-specific keywords, basing their work on meaning rather than specific words.
  • Cost-Effective LLMs: Small, specialized LLMs often provide better quality for reasonable costs.

Erik's talk provided a deep dive into the transformative power of LLMs in modern technology landscapes. It was a fantastic learning experience, and I left with a lot of actionable insights. 💡

A huge thank you to Joshua Wohle from Mindstone.ai and Barry Cranford from RecWorks (the most community-supportive recruiting agency in the UK) for organizing and sponsoring this fantastic event.

Please find some of the photos attached.

P.S. The article was created with a ton of my notes, photos and some help from ChatGPT.