Agents / sentry-triage
sentry-triage
1.5.0last run 10 min ago
Runs · 24 h
40
280 over 7 days
Failed · 24 h
10.0%
p50 duration · 24 h
15.0s
Mean cost · 24 h
$0.052
Is it getting worse?
App version3.2.0(200 runs)vs3.1.0(360 runs)6 regressions
| Metric | A · 3.1.0 | B · 3.2.0 | Change | Verdict |
|---|---|---|---|---|
| Failed runs | 2.2% | 7.0% | +4.8 pts | regression |
| Failed tool calls | 2.0% | 3.2% | +1.2 pts | stable |
| Mean duration | 11.4s | 13.0s | +15% | stable |
| Mean cost | $0.035 | $0.044 | +25% | regression |
| Steps per run | 9.10 | 9.84 | +8% | stable |
| Tokens per run | 11.2k | 13.0k | +16% | regression |
| Repeated tool calls | 0 | 0.30 | +0.30 | critical |
| Runs on a new tool path Most common new path: sentry.getIssue → gitlab.searchCode ×2 → gitlab.readFile → jira.createIssue → slack.postMessage (23 runs) | 0.0% | 23.5% | +23.5 pts | regression |
| Score · correct | 0.95 | 0.84 | −0.11 | regression |
A metric is flagged only when the change is both statistically significant (one-sided test, p < 0.01, Holm-corrected for the number of metrics) and large enough to matter (+15% on a mean, +3 pts on a rate, −0.05 on a score), with at least 20 runs on each side.
Last 14 days · dashed lines mark the first run of each version
stepsrepeated tool calls
p50p95
Tool paths
App version 3.2.0· 200 runs
| sentry.getIssue → gitlab.searchCode → gitlab.readFile → jira.createIssue → slack.postMessage | 77% |
| sentry.getIssue → gitlab.searchCode ×2 → gitlab.readFile → jira.createIssue → slack.postMessage | 12% |
| sentry.getIssue → gitlab.searchCode ×3 → gitlab.readFile → jira.createIssue → slack.postMessage | 7% |
| sentry.getIssue → gitlab.searchCode ×2 → gitlab.readFile → jira.createIssue → gitlab.createMergeRequest → slack.postMessage | 3% |
| sentry.getIssue → gitlab.searchCode ×3 → gitlab.readFile → jira.createIssue → gitlab.createMergeRequest → slack.postMessage | 1% |
App version 3.1.0· 360 runs
| sentry.getIssue → gitlab.searchCode → gitlab.readFile → jira.createIssue → slack.postMessage | 100% |
App versions
Recent runs
Runs with repeated tool calls →| Run | Version | Status | Started | Duration | Steps | Cost | Flags |
|---|---|---|---|---|---|---|---|
| run_b590779e | 1.5.0 app 3.2.0 · prompt v8 | ok | 10 min ago | 11.5s | 9 | ≈$0.043 | |
| run_ad4ce345 | 1.5.0 app 3.2.0 · prompt v8 | ok | 1 h ago | 18.9s | 13 | ≈$0.081 | repeat ×2 |
| run_ae257847 | 1.5.0 app 3.2.0 · prompt v8 | ok | 2 h ago | 14.9s | 11 | ≈$0.065 | repeat ×1 |
| run_6f21d623 | 1.5.0 app 3.2.0 · prompt v8 | ok | 2 h ago | 14.4s | 11 | ≈$0.0075 | repeat ×1correct 0.00 |
| run_34f63b8a | 1.5.0 app 3.2.0 · prompt v8 | ok | 3 h ago | 15.1s | 11 | ≈$0.0083 | repeat ×1correct 0.00 |
| run_4ca1c1c2 | 1.5.0 app 3.2.0 · prompt v8 | ok | 3 h ago | 15.7s | 11 | ≈$0.065 | repeat ×1correct 0.00 |
| run_1f44248d | 1.5.0 app 3.2.0 · prompt v8 | ok | 4 h ago | 17.4s | 11 | ≈$0.064 | repeat ×1 |
| run_604782f9 | 1.5.0 app 3.2.0 · prompt v8 | ok | 5 h ago | 11.5s | 9 | ≈$0.048 | correct 0.00 |
| run_2a751ac7 | 1.5.0 app 3.2.0 · prompt v8 | ok | 5 h ago | 12.6s | 9 | ≈$0.0057 | |
| run_6d9634c4 | 1.5.0 app 3.2.0 · prompt v8 | ok | 6 h ago | 16.4s | 11 | ≈$0.067 | repeat ×1correct 0.00 |
| run_9b3a9bed | 1.5.0 app 3.2.0 · prompt v8 | ok | 6 h ago | 14.5s | 11 | ≈$0.064 | repeat ×1 |
| run_ebad7bdf | 1.5.0 app 3.2.0 · prompt v8 | ok | 7 h ago | 13.3s | 10 | ≈$0.0056 | retry ×11 tool error |