Extract Structured Data from Website API#

POST /v2/distill is an API to extract structured data from websites: it pulls fields out of one or more URLs to match a schema you supply, either a JSON-Schema object or a flat {field: description} map. It runs a two-pass engine: a free CSS pass first (Crawl4AI's JsonCssExtractionStrategy driven by your selectors), then a capped LLM pass for the fields the CSS pass left empty. The response data is guaranteed to come back in the exact shape you asked for. It will be EnConvert's answer to Firecrawl /extract.

Here is the smallest useful call. Send one URL and a flat {field: description} schema, and get the extracted fields back:

curl -X POST https://api.enconvert.com/v2/distill \
  -H "X-API-Key: sk_your_private_key" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://example.com/product/widget"],
    "schema": {
      "name": "the product name",
      "price": "the listed price",
      "in_stock": "whether it is in stock"
    }
  }'

The response carries one result per URL, the extracted data, and which tier produced it:

{
    "operation_id": "dst_3f9a2c1b8e7d4a6f90b1c2d3e4f5a6b7",
    "total": 1,
    "completed": 1,
    "failed": 0,
    "results": [
        {
            "url": "https://example.com/product/widget",
            "url_final": "https://example.com/product/widget",
            "status": "completed",
            "data": {
                "name": "Widget Pro",
                "price": "$49.00",
                "in_stock": "yes"
            },
            "extraction_tier": "llm",
            "fields_from_css": 0,
            "fields_from_llm": 3,
            "render_quality": 0.91,
            "tokens": {"input": 4120, "output": 38},
            "cost_cents": 0.45,
            "warnings": []
        }
    ],
    "total_cost_cents": 0.45,
    "warnings": []
}

Endpoints#

Content-Type: application/json.

Unlike perceive, distill is a single synchronous endpoint: there is no separate GET re-fetch or async batch path. Every URL renders sequentially through the shared headless Chrome singleton and the full result set comes back in one response.

Authentication#

Authenticate with a private key in the X-API-Key header for server-to-server calls. This is the path the examples below use.

X-API-Key: sk_your_private_key

Public keys with a JWT bearer token also work, using the same flow as every other endpoint: generate a token with your pk_ key, then send it as Authorization: Bearer <token>. The full flow, including domain locking and token refresh, is in the authentication guide.

Each API key carries an allowed-endpoints allowlist. If /v2/distill is not on the key's list, the request is rejected with 403.

How distill works#

One request distills a list of URLs against one schema. The flow is the same for every URL:

1. Resolve the URL list. With urls, the list is exactly what you sent (deduplicated, order preserved). With discover_from, distill runs discover on the seed URL first (parsing the sitemap, crawling, or both), then distills the discovered URLs up to max_pages.
2. Render. Each URL renders once in headless Chrome through the same capture pipeline that powers perceive. The render is screened for SSRF and, if respect_robots=true, checked against the site's robots.txt. No artifacts are uploaded to storage, because distill needs only the rendered DOM.
3. Pass 1: CSS (free). If you supplied a css_schema, the CSS extractor runs over the rendered HTML and fills every selector- addressable field at zero LLM cost. This pass is time-bounded to 10 seconds; on timeout the URL falls through to the LLM pass with a warning.
4. Pass 2: LLM (capped, only when needed). Distill collects the schema fields the CSS pass left missing or empty and escalates only those fields to an LLM, under hard per-call and per-period budget caps. If your plan has no LLM tier, the page was flagged as blocked, or a budget cap is hit, the LLM pass is skipped and the missing fields come back as null with a warning.
5. Normalize. The merged result is reshaped to exactly your schema's keys: missing scalars become null, missing arrays become [], and any extra keys are dropped. The shape guarantee holds regardless of what CSS or the LLM produced.

One op is billed per URL, and only after that URL completes. Render failures (SSRF rejection, robots block, a crashed render) produce a failed result row and cost no ops.

Request parameters#

You must provide exactly one of urls or discover_from, plus a schema. Sending both, or neither, is a 422.

Source: explicit URLs#

Source: discover then distill#

Schema or prompt#

Provide either a schema (the output shape you want) or a prompt (a plain-language description of what to extract). Exactly one is required; if you send both, schema wins.

The schema is the contract. If you send a JSON-Schema object, distill reads its properties; if you send a flat map, each key names a field and each value is the description handed to the LLM. Either way, data comes back with exactly the schema's top-level keys.

Prompt-only mode. If you send a prompt instead of a schema, distill first synthesizes a schema of fields from your prompt, then extracts against it. The synthesized fields are echoed on the response as synthesized_schema. Prompt-only mode uses the LLM extraction tier and requires a plan that includes it; without one, distill returns a clear warning instead of guessing.

CSS schema (optional)#

Supply a css_schema to answer fields for free before any LLM call. Without it, every field falls straight to the LLM pass.

Each entry in fields is a CssField:

Worth flagging: the computed field type from Crawl4AI is deliberately not accepted. Its expression form runs eval on caller input, and its callable form cannot cross a JSON boundary, so distill enumerates only the seven safe types above.

Render knobs#

Distill renders only the DOM, so it exposes a small subset of the perceive render options.

Response#

POST /v2/distill returns a DistillResponse:

Each entry in results is a DistillItemResult:

To be straight about it: the response carries no signed download URLs and no stored artifacts. Distill returns the structured data inline and nothing else. If you also want the page's Markdown, HTML, a screenshot, or a PDF, that is what perceive is for.

The two-pass cost model#

The CSS pass is free. The LLM pass costs money, so distill fires it as narrowly as possible and caps it from several directions.

It escalates only the missing fields. After the CSS pass, distill computes which schema fields are still empty: a scalar that came back null/"", an array that came back empty, or an array whose items lack a declared sub-field. Only those field names go into a reduced schema for the LLM call, which keeps the prompt and the cost minimal.

It skips the LLM pass entirely when any of these hold, returning the CSS-only result with a warning instead of overspending:

• Your plan has no LLM tier (llm_extraction_enabled plus a non-none agent_model_tier).
• The page's render_quality flagged it as blocked by anti-bot protection.
• The per-request LLM budget for this call is reached, or the per-period budget cap is reached.

The budget caps are layered:

Note. The per-period budget is reserved atomically before the call and settled down to the real cost after, so concurrent calls cannot collectively overshoot the cap. A project with no active usage-period row fails closed, because spend EnConvert cannot account for is spend it does not make. When a cap is hit, the affected fields come back as null with a warning; the request still succeeds.

When the LLM pass does run, extraction_tier reports llm or mixed, and tokens and cost_cents report what it cost. When it does not run, both are zero.

Mapping CSS records to your schema#

The common case is a listing page: a repeating row, and an output schema with one array property to hold the rows. Give distill a css_schema whose baseSelector matches the row and whose fields read the columns, and it fills the array for free:

curl -X POST https://api.enconvert.com/v2/distill \
  -H "X-API-Key: sk_your_private_key" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://example.com/products"],
    "schema": {
      "type": "object",
      "properties": {
        "products": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "name": {"type": "string"},
              "price": {"type": "string"},
              "sku": {"type": "string"}
            }
          }
        }
      }
    },
    "css_schema": {
      "baseSelector": ".product-card",
      "target_field": "products",
      "fields": [
        {"name": "name", "type": "text", "selector": ".title"},
        {"name": "price", "type": "text", "selector": ".price"},
        {"name": "sku", "type": "attribute",
         "selector": ".product-card", "attribute": "data-sku"}
      ]
    }
  }'

How records land on the schema:

• target_field set to an array property → that property gets the full record list.
• target_field set to a scalar/object property → it gets the first record.
• target_field omitted, schema has exactly one array property → distill infers it and fills that property.
• Otherwise → the first record is treated as a single flat object and its matching keys are lifted to the top level.

If CSS fills the array but some items miss a declared sub-field (say sku is absent on half the cards), distill escalates products to the LLM pass to fill the gaps. That is the two-pass differentiator over a plain scraper: structured where it can be, model-backed where it has to be.

Code examples#

curl: flat schema, LLM-only#

curl -X POST https://api.enconvert.com/v2/distill \
  -H "X-API-Key: sk_your_private_key" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://example.com/article"],
    "schema": {
      "headline": "the article headline",
      "author": "the author name",
      "published": "the publish date"
    }
  }'

curl: discover then distill#

curl -X POST https://api.enconvert.com/v2/distill \
  -H "X-API-Key: sk_your_private_key" \
  -H "Content-Type: application/json" \
  -d '{
    "discover_from": {
      "url": "https://example.com/blog",
      "mode": "sitemap",
      "max_pages": 25
    },
    "schema": {
      "title": "the post title",
      "summary": "a one-line summary"
    }
  }'

Python#

import requests

response = requests.post(
    "https://api.enconvert.com/v2/distill",
    headers={"X-API-Key": "sk_your_private_key"},
    json={
        "urls": ["https://example.com/product/widget"],
        "schema": {
            "name": "the product name",
            "price": "the listed price",
            "in_stock": "whether it is in stock",
        },
    },
)
response.raise_for_status()
result = response.json()

for item in result["results"]:
    if item["status"] == "completed":
        print(item["url"], "->", item["data"])
    else:
        print(item["url"], "FAILED:", item["error"])

print("total cost (cents):", result["total_cost_cents"])

Node.js#

const res = await fetch("https://api.enconvert.com/v2/distill", {
    method: "POST",
    headers: {
        "Content-Type": "application/json",
        "X-API-Key": "sk_your_private_key"
    },
    body: JSON.stringify({
        urls: ["https://example.com/product/widget"],
        schema: {
            name: "the product name",
            price: "the listed price",
            in_stock: "whether it is in stock"
        }
    })
});

const result = await res.json();

for (const item of result.results) {
    if (item.status === "completed") {
        console.log(item.url, "->", item.data);
    } else {
        console.log(item.url, "FAILED:", item.error);
    }
}

console.log("total cost (cents):", result.total_cost_cents);

Error responses#

A few status notes worth keeping straight: a single URL whose render is rejected for SSRF surfaces a 400 only when it is the seed of a discover_from request; for an explicit urls list, a per-URL render rejection becomes a failed result row rather than failing the whole request. An LLM budget cap is never an error, because it degrades to a CSS-only result with a warning. The full status-code reference is in the error-codes guide.

Limits#

Frequently asked questions#

How do I extract structured data from a website with a REST API?#

Send POST /v2/distill with urls (up to 50 per request) and a schema, either a JSON-Schema object or a flat {field: description} map. The response data comes back normalized to exactly your schema's top-level keys.

Can I scrape a website into a JSON schema without writing CSS selectors?#

Yes. css_schema is optional, and without it every field falls straight to the LLM pass. Supplying a css_schema fills selector-addressable fields for free and escalates only the fields the CSS pass left empty.

How much does the LLM extraction pass cost, and how is it capped?#

The budget caps are layered: $0.05 per LLM call, $0.50 per /v2/distill request, at most 50 escalations per request, and per period your monthly AI-credit balance ($5 / $15 / $40 granted on Indie / Studio / Production; unused credits roll over). LLM extraction spends credits, not ops. Hitting a cap never fails the request: affected fields come back null with a warning.

Why are some fields null in my distill response?#

Missing scalars are normalized to null (and missing arrays to []) to preserve the shape guarantee. The LLM pass is skipped (with a warning) when your plan has no LLM tier, the page's render_quality flagged it as blocked, or a budget cap was reached.

Can I crawl a whole site and extract the same schema from every page?#

Yes. Send discover_from with a seed url, a mode (sitemap, crawl, or hybrid), and max_pages (1-50) instead of urls. It requires the discover_enabled plan flag; without it the request is rejected with 402.