Generative AI and LLM Insights: August 2024

While many teams have been building LLM applications for over a year now, there is still much to learn about RAG and all types of hallucinations. Check out our roundup of the top generative AI and LLM articles for August 2024.

What We’ve Learned From A Year of Building with LLMs

Building production-ready apps using LLMs remains deceptively difficult. Thankfully a group of AI builders have compiled their year of learnings into an in-depth guide: https://applied-llms.org/

Searching for Best Practices in RAG

RAG's complexity can be daunting for any team. Here are some actionable ways to streamline your RAG workflows and optimize performance: https://arxiv.org/abs/2407.01219

🏆 Rerank for Relevance: Use monoT5 or TILDEv2 based on your efficiency requirements

📈 Efficient Embedding: Choose embedding models like LLM-Embedder for better retrieval performance

🛠 Implement Query Classification: Automate the decision-making process to determine if retrieval is necessary

🏎 Select Appropriate Retrieval Methods: Depending on your performance vs. efficiency needs, choose between Hybrid with HyDE or Hybrid

🍪 Optimize Chunking: Use methods like Small2Big and sliding windows for effective chunking

Survey of Hallucinations in Multimodal Models

Multimodal models are gaining popularity across industries, but they are just as prone to hallucinations as LLMs. Learn the different types of hallucinations across modalities, what causes them, and how to mitigate them: https://www.rungalileo.io/blog/survey-of-hallucinations-in-multimodal-models

Extrinsic Hallucinations in LLMs

Extrinsic hallucinations occur when the model fabricates information not supported by its pre-training dataset. Learn some frameworks for detecting and evaluating these hallucinations: https://lilianweng.github.io/posts/2024-07-07-hallucination/

Solving Challenges in GenAI Evaluation – Cost, Latency, and Accuracy

Whether you're using an LLM-as-a-judge or doing human eval vibe checks, there are issues with either approach to GenAI evaluation. Learn the leading ways to evaluate your generative AI initiatives, what to look out for, and how to do it right: https://www.rungalileo.io/blog/solving-challenges-in-genai-evaluation-cost-latency-and-accuracy

Building the Future: A Deep Dive Into the Generative AI App Infrastructure Stack

The GenAI infra stack is constantly evolving and growing. Sapphire Ventures attempts to codeify all the moving pieces powering the AI revolution: https://sapphireventures.com/blog/building-the-future-a-deep-dive-into-the-generative-ai-app-infrastructure-stack/