> For the complete documentation index, see [llms.txt](https://fleet.hackmap.win/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://fleet.hackmap.win/concepts/abstract-apis.md).

# Abstract APIs

Fleet ships a set of *abstract API contracts* — one per generic product category — so concrete plugins from different providers return the same shape. A consumer that takes `SerpResult` from a Google plugin should read the same fields when you swap in Bing or DuckDuckGo.

## The catalog today

| package                                   | shape                               | covers                                                                                                                                         | mode       |
| ----------------------------------------- | ----------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ---------- |
| [`fleet-serp`](#fleet-serp)               | `SerpQuery → SerpResult`            | Google · Bing · DuckDuckGo · Yandex · Baidu · Naver · Brave · Kagi · Ecosia                                                                    | batch      |
| [`fleet-social`](#fleet-social)           | `SocialQuery → SocialResult`        | TikTok · Instagram · YouTube · X · LinkedIn · Reddit · Facebook · Pinterest · Threads · Bluesky · Mastodon · Snapchat                          | batch      |
| [`fleet-place`](#fleet-place)             | `PlaceQuery → PlaceResult`          | Google/Apple Maps · Booking · Airbnb · Hotels.com · Expedia · Yelp · TripAdvisor · OpenTable · Zillow · Redfin · Realtor                       | batch      |
| [`fleet-marketplace`](#fleet-marketplace) | `ProductQuery → ProductResult`      | Amazon · eBay · Walmart · Shopify · AliExpress · Etsy · Mercado Libre · Rakuten · Target · Best Buy · Home Depot                               | batch      |
| [`fleet-jobs`](#fleet-jobs)               | `JobQuery → JobResult`              | LinkedIn · Indeed · Greenhouse · Lever · Glassdoor · Workable · ZipRecruiter · Remotive · WWR · Ashby · SmartRecruiters · BambooHR · AngelList | batch      |
| [`fleet-news`](#fleet-news)               | `NewsArticle` stream                | RSS · Atom · JSON Feed · Google News · Hacker News · Reddit · sitemap-news                                                                     | continuous |
| [`fleet-content`](#fleet-content)         | `ContentRequest → ExtractedContent` | any URL — readability / DOM-to-markdown / screenshot / PDF / raw HTML                                                                          | batch      |

Two cross-cutting helpers underneath them:

| package                           | purpose                                                                                                     |
| --------------------------------- | ----------------------------------------------------------------------------------------------------------- |
| [`fleet-headers`](#fleet-headers) | typed `HeaderProfile` (UA + Sec-CH-UA + Accept-\*) and a built-in catalog of chrome/firefox/safari profiles |
| [`fleet-detect`](#fleet-detect)   | classify which anti-bot system fronts a response so a plugin can route to the right solver                  |

## Why abstract, not concrete

Concrete plugins (`fleet-serp-google`) live in their own packages because:

* Their dependencies differ wildly — a Google scraper needs a browser + `fleet-cloudflare`, a Brave Search API client needs `httpx` and an API key.
* Their anti-bot stories differ — anti-bot tooling that works for one provider hurts another (residential proxy on `google.com` is critical; on `serpapi.com` it's wasted spend).
* Their licensing differs — some providers are free to scrape, some have ToS that forbid it, some have paid APIs that the framework should be able to wrap.

Pushing the common shape into the abstract package lets all the concrete plugins compose with each other in a typed way. A multi-engine SERP front-end reads `SerpAutomation.Output` from whichever subclasses are installed and merges them without per-provider parsing.

## Anatomy of an abstract package

```
packages/fleet-<x>/
  fleet_<x>/
    __init__.py          # public re-exports
    contracts.py         # Pydantic schemas (TaskPayload, Output, enums)
    automation.py        # Abstract BatchAutomation or ContinuousAutomation
  tests/test_contracts.py
  pyproject.toml
  README.md
```

The abstract `Automation` subclass:

* Sets `Config`, `TaskPayload`, `Output` as `ClassVar`s.
* Does **not** use `@register(...)`. It's an inheritance target, not a runnable plugin.
* Raises `NotImplementedError` from `run_one` / `run_slot` so subclassing without overriding fails on the first task, not silently.

## Anatomy of a concrete plugin

```python
from fleet_serp import SerpAutomation, SerpQuery, SerpResult, SerpResultRow, SerpResultType
from fleet.core.automation import register

@register("serp-google")              # this is the registry name
class GoogleSerp(SerpAutomation):     # inherit the abstract base
    async def run_one(self, task, ctx):
        q: SerpQuery = task.payload
        # ... fetch, parse, build the model
        result = SerpResult(query=q, rows=[
            SerpResultRow(rank=1, type=SerpResultType.ORGANIC, title=..., url=...),
        ])
        await ctx.emit(result)
```

A `BatchAutomation` is required to declare `TaskPayload` — the abstract base already does, so concrete subclasses don't have to repeat themselves.

## Shippable, even without concrete impls

Every abstract package:

* Ships **only schemas + abstract base**. No mock implementations, no canned responses.
* Has its own test suite that validates the schemas (required fields, value bounds, cross-field validators).
* Is independently `pip install`-able. A consumer who only wants the Pydantic types (e.g. to build a UI that submits SERP queries to an external solver) can use the package without ever running a fleet worker.

That's why the packages are flagged as shippable today even though no `fleet-serp-google` exists yet — the contract is what travels with downstream code, the concrete plugins are private to whoever runs the fleet.

## When to add a new abstract package

A new generic API surface earns its own abstract package when **two or more concrete providers would implement it**. Single-provider work goes directly in a concrete `fleet-<provider>` package without abstracting prematurely.

The current 7 product packages were picked from a sweep of Apify Store's category filter; together they cover the \~30 highest-traffic actor families.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://fleet.hackmap.win/concepts/abstract-apis.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.