LLM Hallucination Index RAG SpecialModel Insights
open-mistral-nemo-2407
Details
Developer
Mistral
License
Apache 2.0
Model parameters
12B
Supported context length
128k
Price for prompt token
$0.15/Million tokens
Price for response token
$0.15/Million tokens
Model Performance Across Task-Types
Chainpoll Score
Short Context
0.88
Medium Context
1
Model Insights Across Task-Types
Digging deeper, here’s a look how open-mistral-nemo-2407 performed across specific datasets
Short Context RAG
Medium Context RAG
This heatmap indicates the model's success in recalling information at different locations in the context. Green signifies success, while red indicates failure.
Long Context RAG
This heatmap indicates the model's success in recalling information at different locations in the context. Green signifies success, while red indicates failure.
Performance Summary
| Tasks | Dataset | Context adherence | Avg response length |
|---|---|---|---|
| Short context RAG | Drop | 0.84 | 199.7 |
| Hotpot | 0.85 | 109.1 | |
| MS Marco | 0.91 | 379.5 | |
| ConvFinQA | 0.94 | 269.4 | |
| Medium context RAG | Medium context RAG | 1.00 | NA |
| Long context RAG | Long context RAG | NaN | NA |