Benchmark5d ago
Artificial Analysis Benchmarks DeepSeek Search API: $0.75/1K Searches Delivers +36 Point Score Lift
Independent AI evaluation platform Artificial Analysis (@ArtificialAnlys) officially benchmarked DeepSeek's web search API (DeepSeek Search). Augmenting the base model with DeepSeek Search boosted its Search Index score from 33 to 69—an impressive +36 point leap (ranking 17th among 27 tested search APIs). DeepSeek Search established a market-leading price-performance ratio, priced at just $0.75 per 1,000 searches (~$0.032 per completed task), making it the most cost-effective paid search API evaluated. It recorded 70% accuracy on AA-Omniscience and 63% on BrowseComp, highlighting an optimal balance between factual grounding and operational economics.
Benchmark5d ago
Artificial Analysis Releases HealthBench & MLCR-AA: Exposing Severe Verbosity Bias in Medical AI Rankings
Independent AI evaluation platform Artificial Analysis officially released HealthBench Professional alongside the length-normalized MLCR-AA index (@ArtificialAnlys). The benchmark revealed a pervasive industry blindspot: LLM-as-a-Judge evaluations suffer from acute 'verbosity bias', disproportionately rewarding longer generations regardless of medical conciseness. In raw HealthBench scoring where Anthropic outputs were 4–6x longer, Claude Opus 5.5 led at 71.4%. However, once length bias was scientifically normalized via MLCR-AA, GPT-6 Astra (max) tied Claude Sonnet 5.5 for #1, while Claude Opus 5.5 plummeted from #1 down to #6, setting a new rigorous standard for medical model evaluation.
Benchmark8d ago
Artificial Analysis launches HealthBench and HealthBench Professional leaderboards: GPT-6 Astra leads clinician conversations at 66.0%, while Claude Opus 5.5 tops the raw score but falls to sixth after the length penalty
On October 19 independent evaluator Artificial Analysis launched two health-conversation leaderboards built on OpenAI's HealthBench (5,000 realistic health conversations) and HealthBench Professional (525 clinician conversations), graded against physician-written rubrics and reported as both raw and OpenAI-style length-adjusted scores. On Professional, GPT-6 Astra (max) leads at 66.0% length-adjusted, ahead of GPT-6.1 Sol (65.1%) and Claude Sonnet 5.5 (63.0%); Claude Opus 5.5 has the top raw score (71.4%) but drops to sixth (59.9%) because its ~5,900-character answers are penalized. On HealthBench, Claude Sonnet 5.5 leads at 62.1%, with DeepSeek V4.1 Flash close behind at 60.9% for about 1/16 of the cost per task.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.