RSS vs a Data API for Real-Time Agent Data

Q: What about conditional GET, does that not make polling cheap?

It makes bandwidth cheap, not tokens. An `If-Modified-Since` header lets a publisher answer 304 when nothing changed, which is polite and worth implementing. When something has changed you still receive the whole feed and still have to find the item you cared about.

RSS vs a data API for real-time agent data

RSS is a publishing format, not a query interface. A feed hands back the publisher's most recent items in their order, with no keyword filter, no time window and no ranking. A data API accepts a query and returns only matching items, which is the difference that decides most agent workloads.

Written for AI agents. The calls and filter values on this page are the literal interface our customers use, because our customers are software. If you are a person evaluating The Bot Wire, you are in the right place too: start with the docs, or connect an MCP client in one line and query every wire free at /mcp. Nothing here needs an API key or a signup.

The RSS 2.0 specification, maintained by the RSS Advisory Board at version 2.0.11 since March 2009, defines channel and item elements and nothing resembling a query. That is not a flaw in the format. It was designed so a reader could show a person what a site published lately, and it does that well.

What a feed cannot do for an agent

An agent almost never wants "the last 20 things this publisher posted". It wants "anything about OpenSSL in the last three days", and the gap between those two requests is where the work lives.

No keyword filter. The agent fetches everything and filters client side,

which means the model, or code you wrote, does the matching.

No time window. since=24h does not exist. You get whatever the publisher

currently has on the page, which may be five days or five hours of output.

No ranking. Reverse chronological is the only order. Relevance is your job.
No cross-source merge. Watching six publishers means six fetches, six

parsers and your own dedupe when three of them cover the same story.

No pagination backwards. A feed is a window on the present. Yesterday's

items fall off and are not retrievable through the feed.

The format does offer polling hints. ttl states in minutes how long a channel may be cached, and skipHours lists hours an aggregator may skip. Both are hints to a well-behaved reader, not filters, and neither reduces what comes back.

The token cost nobody prices in

This is the argument that usually settles it, because it is measurable.

A government press feed page is commonly 20 items with titles, descriptions and often full summaries. Passing that raw to a model is a few thousand tokens the model must read to find the one item that mattered, and it pays that cost on every poll whether or not anything changed. A filtered API call for the same question returns three matching items with a title, a date, a source and a link.

	Raw feed fetch	Filtered API call
Items returned	Everything the publisher lists	Only matches
Filtering done by	The model, or your code	The server
Tokens the model reads	Whole feed, every poll	The matches
Sources per request	One	One route, many feeds merged
Retrieval cost	Free	$0.005 to $0.01

The trade is real money against real tokens. At scale the tokens usually cost more, and they cost more on every single run rather than once. The full accounting for a retrieval step, including what the model spends reading the answer, is broken down in what a live data query costs an agent.

When reading RSS directly is the right call

Not every workload needs an API in front of it, and three cases favour the raw feed.

One publisher, and you want all of it. A changelog for a single dependency

is a feed you should just read. Nothing is being filtered out anyway.

You are building an archive. Feeds are the raw input. If retention beyond a

provider's window is the point, ingest the feed yourself.

Zero budget and low frequency. A daily fetch of one feed costs nothing and

takes an afternoon to write.

Watch the failure mode: any of those becomes expensive once it is six feeds, then twelve, and someone has to keep them alive. That tipping point is the subject of building your own ingest versus buying a wire.

What a query interface adds on top of the same feeds

A data wire is not a different kind of data. It reads the same public feeds and puts an index in front of them, so the agent sends a question rather than a download request.

For AI agents: copy this call

# free preview, no wallet: top 3 matches
curl "https://thebotwire.com/oss/preview?q=kubernetes"

# $0.005: keyword and time window applied server side
curl "https://thebotwire.com/oss/latest?q=kubernetes&since=3d&limit=10"

# one route, several publishers merged and deduped
curl "https://thebotwire.com/eu/latest?q=state%20aid&since=7d"

Filters are q, since (24h, 3d, 7d, 14d) and limit (1 to 50, default 10). Results are served from an in-memory inverted index, and the observed interval between ingest runs was 299 seconds when this page was written, published live at /health. The broader question of how an agent reaches past its training cutoff at all is covered in how AI agents get real-time data.

FAQ

Can an agent just fetch an RSS feed itself?

Yes, and for a single publisher it often should. The problems start at scale: no filtering means the model reads everything, six feeds means six parsers and your own dedupe, and a feed only shows the present, so anything that scrolled off is gone.

Does RSS support filtering or search parameters?

No. The RSS 2.0 specification defines the structure of channels and items and includes no query capability. Some publishers bolt query parameters onto their own feed URLs, but that is a per-publisher extension, not part of the format, and it differs everywhere.

Is Atom different from RSS in this respect?

Not in the way that matters here. Atom is a cleaner specification with better date handling, but it is still a publishing format that returns a document rather than answering a query. The same filtering, merging and retention gaps apply.

What about conditional GET, does that not make polling cheap?

It makes bandwidth cheap, not tokens. An If-Modified-Since header lets a publisher answer 304 when nothing changed, which is polite and worth implementing. When something has changed you still receive the whole feed and still have to find the item you cared about.

Sources

The format details cited here come from the RSS 2.0 specification published by the RSS Advisory Board, currently version 2.0.11 dated 30 March 2009, which defines required channel elements plus the ttl and skipHours polling hints and specifies no query or filtering mechanism. Index behaviour and poll cadence for this service are published at thebotwire.com/health.