Agent Search Bench 🔎
Der unabhängige Search-Provider-Benchmark, der sich statistisch weigert, ein Ranking zu behaupten, das er nicht halten kann. Live-Läufe, geblindet, Bradley-Terry pro Intent, Bootstrap-CIs, Replay-Fixtures mit Hash. Leere Zellen, wo die Evidenz fehlt.
„The benchmark that admits when it doesn't know."
Provider-Leaderboard pro Intent-Klasse
ranked (θ + Rang)insufficient — Gate hält nichtnicht getestet (keine Credits/Calls)
| Intent | brave | exa | firecrawl | keenable | linkup | parallel | serpbase | serper | you.com |
|---|
| academic | #2 θ 0.611 | #1 θ 0.6649 | #7 θ -0.1311 | #5 θ -0.0429 | #9 θ -1.5799 | #4 θ 0.15 | #8 θ -0.644 | #6 θ -0.1089 | #3 θ 0.3595 |
|---|
| community_forum | #2 θ 0.529 | #9 θ -0.9704 | #5 θ 0.1316 | #8 θ -0.8133 | #7 θ -0.4894 | #6 θ -0.0581 | insufficient ? | #4 θ 0.2144 | #3 θ 0.5226 |
|---|
| current_news | #2 θ 0.4112 | #5 θ -0.0113 | #4 θ 0.3139 | #8 θ -0.7069 | #9 θ -0.9317 | #6 θ -0.0424 | #1 θ 0.4456 | #3 θ 0.406 | #7 θ -0.1831 |
|---|
| github_code | #1 θ 0.5806 | #4 θ 0.4593 | #8 θ -0.7204 | #5 θ -0.016 | #7 θ -0.4821 | #3 θ 0.4609 | #9 θ -0.8398 | #6 θ -0.4058 | #2 θ 0.5751 |
|---|
| local_entity | insufficient ? | #8 θ -0.4852 | #7 θ -0.4158 | #9 θ -0.557 | #6 θ -0.3301 | #5 θ -0.0166 | #3 θ 0.2865 | #4 θ 0.2605 | #2 θ 0.4372 |
|---|
| official_docs | #2 θ 0.8657 | #4 θ 0.0123 | #7 θ -0.45 | #9 θ -0.7907 | #6 θ -0.4341 | #5 θ -0.2579 | #8 θ -0.6304 | #3 θ 0.1688 | #1 θ 1.0536 |
|---|
| product_research | #3 θ 0.434 | #9 θ -1.6831 | #4 θ 0.4332 | #1 θ 0.8297 | #6 θ -0.2581 | #7 θ -0.3323 | #5 θ -0.143 | #8 θ -0.4676 | #2 θ 0.5314 |
|---|
| semantic_discovery | #1 θ 1.0559 | #5 θ -0.3373 | #9 θ -0.7005 | #4 θ 0.0605 | #7 θ -0.5983 | #2 θ 0.7111 | #6 θ -0.3952 | #8 θ -0.6481 | #3 θ 0.4517 |
|---|
| technical_troubleshooting | #1 θ 1.2496 | #7 θ -0.6176 | #4 θ -0.2558 | #8 θ -0.7456 | #9 θ -0.8251 | #2 θ 0.7524 | #6 θ -0.4578 | #5 θ -0.4275 | #3 θ 0.7319 |
|---|
Speed · Verfügbarkeit · Kosten
brave
Speed1221ms p95 1467
Verfügbar95.8% (207/216)
Kosten$5.0/1k
exa
Speed695ms p95 989
Verfügbar100.0% (216/216)
Kosten$7.0/1k
firecrawl
Speed2139ms p95 5037
Verfügbar99.5% (215/216)
Kosten$6.4/1k
keenable
Speed441ms p95 802
Verfügbar100.0% (216/216)
Kostenn/a
linkup
Speed1430ms p95 1949
Verfügbar100.0% (216/216)
Kosten$5.0/1k
parallel
Speed1836ms p95 3237
Verfügbar100.0% (216/216)
Kosten$5.0/1k
serpbase
Speed9561ms p95 12909
Verfügbar94.4% (204/216)
Kosten$0.5/1k
serper
Speed1187ms p95 1841
Verfügbar100.0% (216/216)
Kosten$1.0/1k
you.com
Speed882ms p95 1031
Verfügbar100.0% (216/216)
Kosten$5.0/1k
Methodik. Qualität = Bradley-Terry-θ pro Intent-Klasse aus geblindeten, paarweisen Judge-Urteilen → Rang + 95%-CI. Ein Ranking erscheint NUR, wenn es die Gates hält (K1 ≥ genug informative Fälle · K2 CI-Untergrenze · K3 Stabilität · Panel-Veto); sonst insufficient_evidence — als First-Class-Ergebnis, nie eine erfundene Zahl. Fehlende Calls fließen NUR in Verfügbarkeit, nie in Qualität/Latenz. θ nur innerhalb einer Korpus-Version vergleichbar.
Dieser Snapshot: Run asb-v0.1-9provider-20260724 · Korpus unknown · Judge codexhttp:gpt-5.6-luna, grokhttp:grok-4.5 · 18/216 abstained.
Determinismus: dieselbe Eingabe → byte-identische site_data.json. Reproduzierbar via Replay-Fixtures.