Console

Scraper Schedules & Backoff

10 sources · autonomous engine
232 req/min
Active jobs
9/10
Requests / min
232
Backoff / throttle
2
Dedupe savings
46%
Success rate
93%

Source Jobs

Click a row to edit schedule & backoff
Source
Interval
Next
Dedupe
Status
CongressHealthy

Congressional Record

https://www.congress.gov/congressional-record
Schedule
How often the scraper polls this source
Base interval2m
Jitter±15%
Rate limit40/min
Retry & backoff
What happens when a fetch fails, times out, or hits 429
Base delay30s
Max delay30m
Max retries
Backoff curve
30s
1m
2m
4m
8m
16m
Deduplication & policy
Reduces duplicate fetches and keeps you a good citizen of the source
Dedupe TTL45m
Respect robots.txt
Dedupe saved
42%
Success
98%
Last run 1m ago · Fingerprint TTL 45m

Run Log & Health — Congressional Record

Last 14 attempts · click a job above to switch source
Last run
1m
Status
Healthy
Error rate
0%
Avg latency
397ms
P95 retries
Diff (Δ)
+54 / ~31 / −3
When
Code
Latency
Retry
Backoff
Diff
Note
1m ago
200
327ms
0×
+101·12 dupe
3m ago
200
230ms
0×
+11·10 dupe
4m ago
200
534ms
0×
+13·16 dupe
6m ago
200
420ms
0×
+74·27 dupe
7m ago
200
594ms
0×
+1·28 dupe
9m ago
200
340ms
0×
+42·6 dupe
10m ago
200
177ms
0×
+542·6 dupe
12m ago
200
577ms
0×
·28 dupe
13m ago
200
458ms
0×
+91·27 dupe
15m ago
200
217ms
0×
+75·23 dupe
16m ago
200
419ms
0×
3·25 dupe
18m ago
200
248ms
0×
+2·24 dupe
19m ago
200
505ms
0×
+151·1 dupe
21m ago
200
514ms
0×
+62·11 dupe

Alerts & Thresholds

2 firing
Reuters Wire
Source error rate spike · 16% errors
C-SPAN Transcripts
Dedupe savings drop · 22% dedupe
Source error rate spike
Notify when a source's 1h error rate exceeds this threshold
Threshold15%
Dedupe savings drop
Notify when dedupe savings fall below this floor for a source
Threshold25%
Topic truth-score floor
Notify when any topic's aggregate truth-score falls below this
Threshold55pts
P95 latency ceiling
Notify when scraper P95 latency exceeds this ceiling
Threshold1500ms
Stalled source
Notify when a source hasn't produced a successful run in N min
Threshold30min
Anchor 2083-02-20T19:01:54.282Z · Config saved locally · Autonomous engine will pick up changes on the next scan tick.