---
seo_title: Distill (Phase 1): Extract Structured Data | EnConvert
meta_desc: Private beta, Phase 1: schema-driven structured extraction from any URL. A free CSS pass runs first and an LLM fallback second, returning your JSON shape.
keywords: extract structured data from website api, web scraping json schema api, llm web scraping api, css selector extraction api, firecrawl extract alternative, scrape product data api, website to json api, structured web data extraction
---

# Extract Structured Data from Website API

<div class="alert alert-warning">
<strong>Private beta.</strong> Distill is callable today with your normal API key, on every plan including Founding, and it draws on your monthly ops allowance like any other call. It is not announced or generally available: the request and response shapes can change without notice, and there is no stability or support commitment, so do not build anything load-bearing on it yet. The roadmap is on <a href="/docs/coming-soon">Coming Soon</a>, and every release is announced in <a href="/changelog">the changelog</a>.
</div>

`POST /v2/distill` is an API to extract structured data from websites: it
pulls fields out of one or more URLs to match a schema you supply, either
a JSON-Schema object or a flat `{field: description}` map. It runs a
two-pass engine: a free CSS pass first (Crawl4AI's
`JsonCssExtractionStrategy` driven by your selectors), then a capped LLM
pass for the fields the CSS pass left empty. The response `data` is
guaranteed to come back in the exact shape
you asked for. It will be EnConvert's answer to Firecrawl `/extract`.

Here is the smallest useful call. Send one URL and a flat
`{field: description}` schema, and get the extracted fields back:

```bash
curl -X POST https://api.enconvert.com/v2/distill \
  -H "X-API-Key: sk_your_private_key" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://example.com/product/widget"],
    "schema": {
      "name": "the product name",
      "price": "the listed price",
      "in_stock": "whether it is in stock"
    }
  }'
```

The response carries one result per URL, the extracted `data`, and which
tier produced it:

```json
{
    "operation_id": "dst_3f9a2c1b8e7d4a6f90b1c2d3e4f5a6b7",
    "total": 1,
    "completed": 1,
    "failed": 0,
    "results": [
        {
            "url": "https://example.com/product/widget",
            "url_final": "https://example.com/product/widget",
            "status": "completed",
            "data": {
                "name": "Widget Pro",
                "price": "$49.00",
                "in_stock": "yes"
            },
            "extraction_tier": "llm",
            "fields_from_css": 0,
            "fields_from_llm": 3,
            "render_quality": 0.91,
            "tokens": {"input": 4120, "output": 38},
            "cost_cents": 0.45,
            "warnings": []
        }
    ],
    "total_cost_cents": 0.45,
    "warnings": []
}
```

---

## Endpoints

| Method | Path | Purpose |
|--------|------|---------|
| `POST` | `/v2/distill` | Distill an explicit URL list, or discover a site's URLs first and distill each, against one schema. |

**Content-Type:** `application/json`.

Unlike [perceive](/docs/endpoints/perceive.md), distill is a single synchronous
endpoint: there is no separate GET re-fetch or async batch path. Every
URL renders sequentially through the shared headless Chrome singleton and
the full result set comes back in one response.

---

## Authentication

Authenticate with a private key in the `X-API-Key` header for
server-to-server calls. This is the path the examples below use.

```http
X-API-Key: sk_your_private_key
```

Public keys with a JWT bearer token also work, using the same flow as
every other endpoint: generate a token with your `pk_` key, then send it
as `Authorization: Bearer <token>`. The full flow, including domain
locking and token refresh, is in [the authentication guide](/docs/authentication.md).

Each API key carries an allowed-endpoints allowlist. If `/v2/distill` is
not on the key's list, the request is rejected with `403`.

---

## How distill works

One request distills a list of URLs against one schema. The flow is the
same for every URL:

1. **Resolve the URL list.** With `urls`, the list is exactly what you
   sent (deduplicated, order preserved). With `discover_from`, distill
   runs [discover](/docs/coming-soon/discover.md) on the seed URL first (parsing the
   sitemap, crawling, or both), then distills the discovered URLs up to
   `max_pages`.
2. **Render.** Each URL renders once in headless Chrome through the same
   capture pipeline that powers [perceive](/docs/endpoints/perceive.md). The render
   is screened for SSRF and, if `respect_robots=true`, checked against the
   site's `robots.txt`. No artifacts are uploaded to storage, because
   distill needs only the rendered DOM.
3. **Pass 1: CSS (free).** If you supplied a `css_schema`, the CSS
   extractor runs over the rendered HTML and fills every selector-
   addressable field at zero LLM cost. This pass is time-bounded to 10
   seconds; on timeout the URL falls through to the LLM pass with a
   warning.
4. **Pass 2: LLM (capped, only when needed).** Distill collects the
   schema fields the CSS pass left missing or empty and escalates *only
   those fields* to an LLM, under hard per-call and per-period
   budget caps. If your plan has no LLM tier, the page was flagged as
   blocked, or a budget cap is hit, the LLM pass is skipped and the
   missing fields come back as `null` with a warning.
5. **Normalize.** The merged result is reshaped to exactly your schema's
   keys: missing scalars become `null`, missing arrays become `[]`, and
   any extra keys are dropped. The shape guarantee holds regardless of
   what CSS or the LLM produced.

One op is billed per URL, and only after that URL completes. Render
failures (SSRF rejection, robots block, a crashed render) produce a
`failed` result row and cost no ops.

---

## Request parameters

You must provide **exactly one** of `urls` or `discover_from`, plus a
`schema`. Sending both, or neither, is a `422`.

### Source: explicit URLs

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `urls` | `string[]` | -- | Explicit URLs to distill. Each must start with `http://` or `https://` and be at most 2,048 characters. Max 50 URLs per request (`MAX_DISTILL_URLS`). Mutually exclusive with `discover_from`. |

### Source: discover then distill

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `discover_from` | `object` | -- | Discover a site's URLs first, then distill each. Mutually exclusive with `urls`. Requires the `discover_enabled` plan flag (otherwise `402`). |
| `discover_from.url` | `string` | -- | Seed URL. Must start with `http://` or `https://`. Max 2,048 characters. |
| `discover_from.mode` | `string` | `hybrid` | `sitemap`, `crawl`, or `hybrid`. Same modes as [the discover endpoint](/docs/coming-soon/discover.md). |
| `discover_from.max_pages` | `integer` | `10` | Cap on URLs discovered and distilled, from 1 to 50. Each is a full render, so it is bounded by `MAX_DISTILL_URLS`. |

### Schema or prompt

Provide **either** a `schema` (the output shape you want) **or** a
`prompt` (a plain-language description of what to extract). Exactly one is
required; if you send both, `schema` wins.

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `schema` | `object` | `null` | The output shape, sent under the JSON key `schema`. Either a JSON-Schema object (`{"type": "object", "properties": {...}}`) or a forgiving flat `{field: description}` map. Max 200 top-level properties. The response `data` is guaranteed to match this shape. A structurally invalid schema is a `422`. |
| `prompt` | `string` | `null` | A natural-language description of what to extract. When given without a `schema`, distill synthesizes the extraction schema from it (single model) and then runs the normal two-pass engine. Max 2,000 characters. |

The schema is the contract. If you send a JSON-Schema object, distill
reads its `properties`; if you send a flat map, each key names a field and
each value is the description handed to the LLM. Either way, `data` comes
back with exactly the schema's top-level keys.

**Prompt-only mode.** If you send a `prompt` instead of a `schema`,
distill first synthesizes a schema of fields from your prompt, then
extracts against it. The synthesized fields are echoed on the response as
`synthesized_schema`. Prompt-only mode uses the LLM extraction tier and
requires a plan that includes it; without one, distill returns a clear
warning instead of guessing.

### CSS schema (optional)

Supply a `css_schema` to answer fields for free before any LLM call.
Without it, every field falls straight to the LLM pass.

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `css_schema.baseSelector` | `string` | -- | CSS selector for the repeating container; one record is extracted per match. 1–1,024 characters. Required when `css_schema` is present. |
| `css_schema.fields` | `CssField[]` | -- | 1–128 field definitions read out of each container. Required. |
| `css_schema.name` | `string` | `"distill"` | Optional label for the schema. Max 128 characters. |
| `css_schema.target_field` | `string` | inferred | Which top-level output property the CSS records fill: an array property gets the full record list, a scalar/object property gets the first record. When omitted, distill infers it if the schema has exactly one array property. Max 128 characters. |

Each entry in `fields` is a `CssField`:

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `name` | `string` | -- | Output key for this field. 1–128 characters. Required. |
| `type` | `string` | -- | One of `text`, `attribute`, `html`, `regex`, `nested`, `list`, `nested_list`. Required. |
| `selector` | `string` | `null` | CSS sub-selector. Optional for leaf types (`text`/`attribute`/`html`/`regex`); required for `nested`/`list`/`nested_list`. Max 1,024 characters. |
| `attribute` | `string` | `null` | Attribute name to read. Required when `type` is `attribute`. Max 128 characters. |
| `pattern` | `string` | `null` | Regex pattern. Required when `type` is `regex`. Compiled at the edge; a pattern with nested re-quantified groups (a ReDoS shape like `(a+)+`) is rejected with `422`. Max 1,024 characters. |
| `default` | any | `null` | Value when the selector matches nothing. |
| `transform` | `string` | `null` | One of `lowercase`, `uppercase`, `strip`. |
| `fields` | `CssField[]` | `null` | Child fields, for `nested`/`list`/`nested_list`. Max 64 children; total nesting depth max 5. |

Worth flagging: the `computed` field type from Crawl4AI is deliberately
not accepted. Its expression form runs `eval` on caller input, and its
callable form cannot cross a JSON boundary, so distill enumerates only
the seven safe types above.

### Render knobs

Distill renders only the DOM, so it exposes a small subset of the
[perceive render options](/docs/endpoints/perceive.md).

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `wait_for` | `string` | `null` | Wait after navigation for a CSS selector or a JS expression (`"js:window.dataReady === true"`). Max 1,024 characters. |
| `wait_timeout_ms` | `integer` | `30000` | How long `wait_for` may wait, in milliseconds. 0–60,000. |
| `headers` | `object` | `null` | Custom request headers for the render. |
| `cookies` | `array` | `null` | Cookies to inject before navigation. |
| `respect_robots` | `boolean` | `false` | When `true`, a URL disallowed by the site's `robots.txt` is rejected and that URL's result is marked `failed`. |

---

## Response

`POST /v2/distill` returns a `DistillResponse`:

| Field | Type | Description |
|-------|------|-------------|
| `operation_id` | `string` | Opaque ID (`dst_...`). Quote it to support. |
| `total` | `integer` | Number of URLs processed (rows in `results`). |
| `completed` | `integer` | URLs that rendered and produced a `data` object. |
| `failed` | `integer` | URLs whose render was rejected or crashed. |
| `results` | `object[]` | One `DistillItemResult` per URL, described below. |
| `total_cost_cents` | `number` | Sum of per-URL LLM cost across the request, in cents. |
| `synthesized_schema` | `object` | Present only in prompt-only mode: the schema that was synthesized from your `prompt` and used for extraction. |
| `warnings` | `string[]` | Request-level notes (e.g. ops allowance exhausted mid-list, discover crawl faults). |

Each entry in `results` is a `DistillItemResult`:

| Field | Type | Description |
|-------|------|-------------|
| `url` | `string` | The URL you sent (or discovered). |
| `url_final` | `string` | The URL after redirects. Omitted on a `failed` row. |
| `status` | `string` | `completed` or `failed`. |
| `data` | `object` | The extracted data, normalized to exactly your schema's keys. `null` on a `failed` row. |
| `extraction_tier` | `string` | `css` (CSS only), `llm` (LLM only), `mixed` (both contributed), or `none` (nothing found). |
| `fields_from_css` | `integer` | Count of fields the CSS pass filled. |
| `fields_from_llm` | `integer` | Count of fields the LLM pass filled. |
| `render_quality` | `number` | 0.0–1.0. Low scores flag anti-bot challenges or login walls. |
| `tokens` | `object` | `{input, output}` LLM tokens used. Zero unless the LLM pass ran. |
| `cost_cents` | `number` | LLM cost in cents for this URL. Zero unless the LLM pass ran. |
| `error` | `string` | Set only when `status` is `failed`. A generic message, because internal render detail stays server-side. |
| `warnings` | `string[]` | Per-URL notes: a CSS timeout, a skipped LLM pass, a budget cap hit. |

To be straight about it: the response carries no signed download URLs and
no stored artifacts. Distill returns the structured `data` inline and
nothing else. If you also want the page's Markdown, HTML, a screenshot,
or a PDF, that is what [perceive](/docs/endpoints/perceive.md) is for.

---

## The two-pass cost model

The CSS pass is free. The LLM pass costs money, so distill fires it as
narrowly as possible and caps it from several directions.

**It escalates only the missing fields.** After the CSS pass, distill
computes which schema fields are still empty: a scalar that came back
`null`/`""`, an array that came back empty, or an array whose items lack a
declared sub-field. Only those field names go into a reduced schema for
the LLM call, which keeps the prompt and the cost minimal.

**It skips the LLM pass entirely when** any of these hold, returning the
CSS-only result with a warning instead of overspending:

- Your plan has no LLM tier (`llm_extraction_enabled` plus a non-`none`
  `agent_model_tier`).
- The page's `render_quality` flagged it as blocked by anti-bot
  protection.
- The per-request LLM budget for this call is reached, or the per-period
  budget cap is reached.

**The budget caps are layered:**

| Cap | Value | Scope |
|-----|-------|-------|
| Per call | $0.05 (`PER_REQUEST_CAP_CENTS`) | One LLM call's worst-case projected cost. Over it → skipped before any network I/O. |
| Per request | $0.50 (`_REQUEST_LLM_BUDGET_CENTS`) | Total LLM spend across all URLs in one `/v2/distill` call. Remaining URLs return CSS-only. |
| Per request escalations | 50 (`_MAX_LLM_ESCALATIONS`) | At most one LLM call per URL, hard-capped. |
| Per period | Your monthly AI-credit balance: $5 / $15 / $40 granted per month on Indie / Studio / Production, unused credits roll over | `ch_usage_periods.llm_cost_cents` against the period's granted credits (`usage.reserve_llm_budget`). |

> **Note.** The per-period budget is reserved atomically before the call
> and settled down to the real cost after, so concurrent calls cannot
> collectively overshoot the cap. A project with no active usage-period
> row fails closed, because spend EnConvert cannot account for is spend it
> does not make. When a cap is hit, the affected fields come back as `null`
> with a warning; the request still succeeds.

When the LLM pass does run, `extraction_tier` reports `llm` or `mixed`,
and `tokens` and `cost_cents` report what it cost. When it does not run,
both are zero.

---

## Mapping CSS records to your schema

The common case is a listing page: a repeating row, and an output schema
with one array property to hold the rows. Give distill a `css_schema`
whose `baseSelector` matches the row and whose `fields` read the columns,
and it fills the array for free:

```bash
curl -X POST https://api.enconvert.com/v2/distill \
  -H "X-API-Key: sk_your_private_key" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://example.com/products"],
    "schema": {
      "type": "object",
      "properties": {
        "products": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "name": {"type": "string"},
              "price": {"type": "string"},
              "sku": {"type": "string"}
            }
          }
        }
      }
    },
    "css_schema": {
      "baseSelector": ".product-card",
      "target_field": "products",
      "fields": [
        {"name": "name", "type": "text", "selector": ".title"},
        {"name": "price", "type": "text", "selector": ".price"},
        {"name": "sku", "type": "attribute",
         "selector": ".product-card", "attribute": "data-sku"}
      ]
    }
  }'
```

How records land on the schema:

- `target_field` set to an **array** property → that property gets the
  full record list.
- `target_field` set to a **scalar/object** property → it gets the first
  record.
- `target_field` omitted, schema has **exactly one** array property →
  distill infers it and fills that property.
- Otherwise → the first record is treated as a single flat object and its
  matching keys are lifted to the top level.

If CSS fills the array but some items miss a declared sub-field (say
`sku` is absent on half the cards), distill escalates `products` to the
LLM pass to fill the gaps. That is the two-pass differentiator over a
plain scraper: structured where it can be, model-backed where it has to be.

---

## Code examples

### curl: flat schema, LLM-only

```bash
curl -X POST https://api.enconvert.com/v2/distill \
  -H "X-API-Key: sk_your_private_key" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://example.com/article"],
    "schema": {
      "headline": "the article headline",
      "author": "the author name",
      "published": "the publish date"
    }
  }'
```

### curl: discover then distill

```bash
curl -X POST https://api.enconvert.com/v2/distill \
  -H "X-API-Key: sk_your_private_key" \
  -H "Content-Type: application/json" \
  -d '{
    "discover_from": {
      "url": "https://example.com/blog",
      "mode": "sitemap",
      "max_pages": 25
    },
    "schema": {
      "title": "the post title",
      "summary": "a one-line summary"
    }
  }'
```

### Python

```python
import requests

response = requests.post(
    "https://api.enconvert.com/v2/distill",
    headers={"X-API-Key": "sk_your_private_key"},
    json={
        "urls": ["https://example.com/product/widget"],
        "schema": {
            "name": "the product name",
            "price": "the listed price",
            "in_stock": "whether it is in stock",
        },
    },
)
response.raise_for_status()
result = response.json()

for item in result["results"]:
    if item["status"] == "completed":
        print(item["url"], "->", item["data"])
    else:
        print(item["url"], "FAILED:", item["error"])

print("total cost (cents):", result["total_cost_cents"])
```

### Node.js

```javascript
const res = await fetch("https://api.enconvert.com/v2/distill", {
    method: "POST",
    headers: {
        "Content-Type": "application/json",
        "X-API-Key": "sk_your_private_key"
    },
    body: JSON.stringify({
        urls: ["https://example.com/product/widget"],
        schema: {
            name: "the product name",
            price: "the listed price",
            in_stock: "whether it is in stock"
        }
    })
});

const result = await res.json();

for (const item of result.results) {
    if (item.status === "completed") {
        console.log(item.url, "->", item.data);
    } else {
        console.log(item.url, "FAILED:", item.error);
    }
}

console.log("total cost (cents):", result.total_cost_cents);
```

---

## Error responses

| Status | Condition |
|--------|-----------|
| `400 Bad Request` | A URL (or the `discover_from` seed) resolves to a private, loopback, or link-local address, has no hostname, or otherwise fails the SSRF screen. Raised per URL during render. |
| `401 Unauthorized` | Missing or invalid API key / JWT token. |
| `402 Payment Required` | Distill is not on your current plan, your monthly ops allowance is exhausted, or `discover_from` was sent without the `discover_enabled` plan flag. |
| `403 Forbidden` | `/v2/distill` is not in the API key's allowed endpoints. |
| `422 Unprocessable Entity` | No `schema`, both or neither of `urls`/`discover_from`, a structurally invalid schema, over 200 schema properties, an invalid `CssField` (missing `attribute`/`pattern`/`fields` for its type, an uncompilable or ReDoS-prone regex), or CSS field nesting deeper than 5. |
| `500 Internal Server Error` | The orchestration failed unexpectedly. The message includes the `operation_id` to quote to support. |

A few status notes worth keeping straight: a single URL whose render is
rejected for SSRF surfaces a `400` only when it is the seed of a
`discover_from` request; for an explicit `urls` list, a per-URL render
rejection becomes a `failed` result row rather than failing the whole
request. An LLM budget cap is never an error, because it degrades to a
CSS-only result with a warning. The full status-code reference is in
[the error-codes guide](/docs/reference/errors.md).

---

## Limits

| Limit | Value |
|-------|-------|
| URLs per request (`urls`) | 50 (`MAX_DISTILL_URLS`) |
| `discover_from.max_pages` | 1–50 |
| URL length | 2,048 characters |
| Schema top-level properties | 200 (`MAX_SCHEMA_PROPERTIES`) |
| `css_schema.fields` | 1–128 |
| `CssField.fields` children | 64 |
| CSS field nesting depth | 5 (`MAX_CSS_FIELD_DEPTH`) |
| `wait_for` length | 1,024 characters |
| `wait_timeout_ms` | 0–60,000 ms |
| CSS pass timeout | 10 seconds per URL |
| LLM per-call cap | $0.05 |
| LLM per-request cap | $0.50 |
| LLM escalations per request | 50 |
| LLM per-period cap | Monthly AI-credit balance ($5 / $15 / $40 by tier; unused credits roll over) |
| Monthly ops (shared across every endpoint, 1 per completed URL) | 500 / 3,000 / 15,000 / 50,000 by tier; see [pricing](/pricing.md) |

---

## Frequently asked questions

### How do I extract structured data from a website with a REST API?

Send `POST /v2/distill` with `urls` (up to 50 per request) and a `schema`, either a JSON-Schema object or a flat `{field: description}` map. The response `data` comes back normalized to exactly your schema's top-level keys.

### Can I scrape a website into a JSON schema without writing CSS selectors?

Yes. `css_schema` is optional, and without it every field falls straight to the LLM pass. Supplying a `css_schema` fills selector-addressable fields for free and escalates only the fields the CSS pass left empty.

### How much does the LLM extraction pass cost, and how is it capped?

The budget caps are layered: $0.05 per LLM call, $0.50 per `/v2/distill` request, at most 50 escalations per request, and per period your monthly AI-credit balance ($5 / $15 / $40 granted on Indie / Studio / Production; unused credits roll over). LLM extraction spends credits, not ops. Hitting a cap never fails the request: affected fields come back `null` with a warning.

### Why are some fields null in my distill response?

Missing scalars are normalized to `null` (and missing arrays to `[]`) to preserve the shape guarantee. The LLM pass is skipped (with a warning) when your plan has no LLM tier, the page's `render_quality` flagged it as blocked, or a budget cap was reached.

### Can I crawl a whole site and extract the same schema from every page?

Yes. Send `discover_from` with a seed `url`, a `mode` (`sitemap`, `crawl`, or `hybrid`), and `max_pages` (1-50) instead of `urls`. It requires the `discover_enabled` plan flag; without it the request is rejected with `402`.
