Agents / sentry-triage
sentry-triage
1.5.0last run 10 min ago
Runs · 24 h
40
280 over 7 days
Failed · 24 h
10.0%
p50 duration · 24 h
15.0s
Mean cost · 24 h
$0.052
Is it getting worse?
Agent version1.5.0(80 runs)vs1.4.0(480 runs)8 regressions
| Metric | A · 1.4.0 | B · 1.5.0 | Change | Verdict |
|---|---|---|---|---|
| Failed runs | 2.5% | 12.5% | +10.0 pts | critical |
| Failed tool calls | 2.0% | 4.4% | +2.3 pts | stable |
| Mean duration | 11.4s | 15.5s | +36% | regression |
| Mean cost | $0.035 | $0.056 | +58% | critical |
| Steps per run | 9.10 | 10.91 | +20% | regression |
| Tokens per run | 11.2k | 15.6k | +39% | regression |
| Repeated tool calls | 0 | 0.76 | +0.76 | critical |
| Runs on a new tool path Most common new path: sentry.getIssue → gitlab.searchCode ×2 → gitlab.readFile → jira.createIssue → slack.postMessage (23 runs) | 0.0% | 58.8% | +58.8 pts | critical |
| Score · correct | 0.95 | 0.68 | −0.27 | critical |
A metric is flagged only when the change is both statistically significant (one-sided test, p < 0.01, Holm-corrected for the number of metrics) and large enough to matter (+15% on a mean, +3 pts on a rate, −0.05 on a score), with at least 20 runs on each side.
Last 14 days · dashed lines mark the first run of each version
stepsrepeated tool calls
p50p95
Tool paths
Agent version 1.5.0· 80 runs
| sentry.getIssue → gitlab.searchCode → gitlab.readFile → jira.createIssue → slack.postMessage | 41% |
| sentry.getIssue → gitlab.searchCode ×2 → gitlab.readFile → jira.createIssue → slack.postMessage | 29% |
| sentry.getIssue → gitlab.searchCode ×3 → gitlab.readFile → jira.createIssue → slack.postMessage | 18% |
| sentry.getIssue → gitlab.searchCode ×2 → gitlab.readFile → jira.createIssue → gitlab.createMergeRequest → slack.postMessage | 8% |
| sentry.getIssue → gitlab.searchCode ×3 → gitlab.readFile → jira.createIssue → gitlab.createMergeRequest → slack.postMessage | 3% |
Agent version 1.4.0· 480 runs
| sentry.getIssue → gitlab.searchCode → gitlab.readFile → jira.createIssue → slack.postMessage | 100% |
Agent versions
Recent runs
Runs with repeated tool calls →| Run | Version | Status | Started | Duration | Steps | Cost | Flags |
|---|---|---|---|---|---|---|---|
| run_b590779e | 1.5.0 app 3.2.0 · prompt v8 | ok | 10 min ago | 11.5s | 9 | ≈$0.043 | |
| run_ad4ce345 | 1.5.0 app 3.2.0 · prompt v8 | ok | 1 h ago | 18.9s | 13 | ≈$0.081 | repeat ×2 |
| run_ae257847 | 1.5.0 app 3.2.0 · prompt v8 | ok | 2 h ago | 14.9s | 11 | ≈$0.065 | repeat ×1 |
| run_6f21d623 | 1.5.0 app 3.2.0 · prompt v8 | ok | 2 h ago | 14.4s | 11 | ≈$0.0075 | repeat ×1correct 0.00 |
| run_34f63b8a | 1.5.0 app 3.2.0 · prompt v8 | ok | 3 h ago | 15.1s | 11 | ≈$0.0083 | repeat ×1correct 0.00 |
| run_4ca1c1c2 | 1.5.0 app 3.2.0 · prompt v8 | ok | 3 h ago | 15.7s | 11 | ≈$0.065 | repeat ×1correct 0.00 |
| run_1f44248d | 1.5.0 app 3.2.0 · prompt v8 | ok | 4 h ago | 17.4s | 11 | ≈$0.064 | repeat ×1 |
| run_604782f9 | 1.5.0 app 3.2.0 · prompt v8 | ok | 5 h ago | 11.5s | 9 | ≈$0.048 | correct 0.00 |
| run_2a751ac7 | 1.5.0 app 3.2.0 · prompt v8 | ok | 5 h ago | 12.6s | 9 | ≈$0.0057 | |
| run_6d9634c4 | 1.5.0 app 3.2.0 · prompt v8 | ok | 6 h ago | 16.4s | 11 | ≈$0.067 | repeat ×1correct 0.00 |
| run_9b3a9bed | 1.5.0 app 3.2.0 · prompt v8 | ok | 6 h ago | 14.5s | 11 | ≈$0.064 | repeat ×1 |
| run_ebad7bdf | 1.5.0 app 3.2.0 · prompt v8 | ok | 7 h ago | 13.3s | 10 | ≈$0.0056 | retry ×11 tool error |