Model Insights

claude-3-haiku-20240307

Details

Developer

Anthropic

License

NA (private model)

Model parameters

NA (private model)

Supported context length

200k

Price for prompt token

$0.25/Million tokens

Price for response token

$1.25/Million tokens

Model Performance Across Task-Types

Chainpoll Score

Short Context

0.92

Medium Context

0.96

Long Context

0.7

Model Insights Across Task-Types

Digging deeper, here’s a look how claude-3-haiku-20240307 performed across specific datasets

Short Context RAG

Medium Context RAG

This heatmap indicates the model's success in recalling information at different locations in the context. Green signifies success, while red indicates failure.

Performance Summary

Tasks	Dataset	Context adherence	Avg response length
Short context RAG	Drop	0.93	389.6
	Hotpot	0.87	455.7
	MS Marco	0.93	1043.4
	ConvFinQA	0.97	443.1
Medium context RAG	Medium context RAG	0.96	NA
Long context RAG	Long context RAG	0.70	NA