---
seo_title: Web Scraping API for Markdown, Screenshots & PDF | EnConvert
meta_desc: POST /v2/perceive renders a JavaScript page in headless Chrome and returns Markdown, screenshots, PDF, and structured data from one web scraping API call.
keywords: web scraping api markdown screenshot, render javascript page to markdown api, read web page for llm api, url to markdown api, website screenshot api, html to markdown api, extract structured data from web page api, batch web scraping api
---

# Web Scraping API for Markdown, Screenshots, and Structured Data

`POST /v2/perceive` is EnConvert's web scraping API: it renders a URL
once in a real headless browser (JavaScript executed, lazy content
loaded) and hands back every output you ask for from that single
render: clean Markdown (main content only by default, site chrome
stripped), cleaned or raw HTML, a screenshot, a PDF, the link and image
inventory, and structured data (page metadata, JSON-LD, headings,
tables). File outputs come back as short-lived pre-signed download URLs,
the structured block inline, and batches over 10 URLs run asynchronously
behind a polled `job_id`. One request replaces a whole stack of separate
calls: url-to-markdown, url-to-screenshot, url-to-pdf, plus your own
scraping.

Here is the smallest useful call. Send a URL, get clean Markdown and the
page's structured metadata back:

```bash
curl -X POST https://api.enconvert.com/v2/perceive \
  -H "X-API-Key: sk_your_private_key" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/pricing",
    "outputs": ["markdown", "structured"]
  }'
```

The response carries a pre-signed download URL for the Markdown file and
the structured block inline:

```json
{
    "operation_id": "per_3f9a2c1b8e7d4a6f90b1c2d3e4f5a6b7",
    "status": "completed",
    "url": "https://example.com/pricing",
    "url_final": "https://example.com/pricing",
    "content_hash": "9f2b8c1a...d4e5",
    "render_quality": 0.93,
    "cache_hit": false,
    "outputs": {
        "markdown": {
            "url": "https://spaces.example.com/...signed...",
            "object_key": "env/files/4127/v2-perceive/per_3f9a..._markdown.md",
            "size_bytes": 8421,
            "content_type": "text/markdown; charset=utf-8",
            "expires_in": 900
        }
    },
    "structured": {
        "metadata": {
            "title": "Pricing",
            "description": "Simple, usage-based pricing."
        },
        "structured_data": [
            {"@type": "Product", "name": "Studio", "offers": {"price": "99"}}
        ]
    },
    "extraction_tier": "heuristic",
    "tokens": {"input": 0, "output": 0},
    "cost_cents": 0.0,
    "duration_ms": 6230,
    "warnings": []
}
```

---

## Endpoints

| Method | Path | Purpose |
|--------|------|---------|
| `POST` | `/v2/perceive` | Perceive a single URL and return the outputs you requested. |
| `GET` | `/v2/perceive/{operation_id}` | Re-fetch a past operation with freshly signed download URLs. |
| `POST` | `/v2/perceive/batch` | Perceive up to 1,000 URLs that share one set of options. |
| `GET` | `/v2/perceive/batch/{job_id}` | Poll the status and per-URL results of a batch. |
| `DELETE` | `/v2/perceive/batch/{job_id}` | Cancel a running batch. |

**Content-Type:** `application/json` on every `POST`.

---

## Authentication

Authenticate with a private key in the `X-API-Key` header for
server-to-server calls. This is the path the examples below use.

```http
X-API-Key: sk_your_private_key
```

Public keys with a JWT bearer token also work, using the same flow as
every other endpoint: generate a token with your `pk_` key, then send it
as `Authorization: Bearer <token>`. The full flow, including domain
locking and token refresh, is in [the authentication guide](/docs/authentication.md).

Each API key carries an allowed-endpoints allowlist. If `/v2/perceive`
is not on the key's list, the request is rejected with `403`.

---

## How perceive works

One request triggers one browser render through a shared headless Chrome
singleton, then materialises every output from that render. You never
pay for the same page twice in a single call.

1. **Render.** The page is fetched through an automatic multi-engine
   fallback: a fast real-browser TLS fingerprint first, escalating to
   headless Chrome when the page is blocked or needs JavaScript, and once
   more to a stealth-hardened render when a page still looks blocked by
   anti-bot protection, so more real-world pages come back with usable
   content. In the browser, cookie banners are dismissed, the page is
   scrolled to trigger lazy-loaded content, sticky headers are handled,
   and images are given time to load. This is the same capture pipeline that
   powers [the url-to-pdf endpoint](/docs/endpoints/convert/web-pages/url-to-pdf.md).
2. **Materialise.** From the rendered DOM, perceive builds whatever you
   listed in `outputs`: Markdown, cleaned/raw HTML, links, images, a
   screenshot, a PDF. The DOM is normalised first so the Markdown
   reflects what a reader sees, code fences keep their language, card
   links keep their structure, and interface furniture is removed under
   `only_main_content`. See [Markdown quality](#markdown-quality).
3. **Extract.** If you requested `structured` output, perceive runs a
   heuristic pass for page metadata, JSON-LD, headings, and tables. If
   you also send a `schema` and your plan carries the LLM tier, an
   LLM fills the schema when the heuristic pass comes up short.
4. **Score.** A render-quality score (0.0-1.0) tells a real render from
   a failed one. Scores below 0.40 mean a failed render: an anti-bot
   page, a login wall, an HTTP error page, a soft 404, or an empty
   shell. Consult `deductions` for the reason and `status_code` for the
   upstream status.

Binary and text outputs (Markdown, HTML, screenshots, PDFs, the link and
image JSON) are uploaded to storage and returned as **pre-signed URLs**
that expire after 15 minutes. The `structured` block is returned inline
in the JSON. Re-fetch any operation with `GET /v2/perceive/{operation_id}`
to get a fresh set of signed URLs.

---

## Request parameters

Validation is strict: a request key the schema does not know is rejected
with `422` naming the offending field. Unknown keys are never silently
ignored. Every `422` body also carries a top-level `errors` array of
human-readable messages alongside the machine-readable `detail` list.

### Core

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `url` | `string` | -- | The page to perceive. Must start with `http://` or `https://`. Max 2,048 characters. Required. |
| `outputs` | `string[]` | `["markdown", "structured"]` | Which outputs to produce. See [Outputs](#outputs). |
| `extract` | `string[]` | `[]` | Which structured fields to pull when `structured` is in `outputs`. See [Structured extraction](#structured-extraction). |
| `schema` | `object` | `null` | A JSON schema describing the fields you want extracted. Triggers the LLM extraction tier on plans that include it. |
| `only_main_content` | `boolean` | `true` | Strip site chrome (navigation, header, footer, sidebars, cookie banners, hidden nodes) plus interface furniture (buttons, tab strips, "Was this page helpful?" widgets, screen-reader-only labels, breadcrumbs) from the `markdown` output and the `main_content` extract, behind a fidelity guard: if stripping would remove too much real content, the full page is returned instead and a warning is added. Image URLs are rendered as their alt text (the full image list stays available via `outputs: ["images"]`). Set `false` for the full page, nothing stripped. See [Markdown quality](#markdown-quality). |
| `truncate_data_arrays` | `boolean` | unset | Collapse long runs of numeric literals (raw embedding vectors, tensor dumps printed in notebook output cells) to a leading sample plus a count, e.g. `... [truncated 1520 of 1536 values]`. Unset follows `only_main_content`: on when the page is being curated, off when you asked for the page as-is. Set `true` or `false` to control it explicitly. |
| `allow_degraded` | `boolean` | `false` | Return the render even when it is an anti-bot challenge or block page carrying no page content. By default such a render fails with `502` instead of delivering the interstitial's text as if it were the page. |
| `direct_download` | `boolean` | `false` | Return the artifact bytes directly as the HTTP response body instead of a JSON envelope. Requires exactly one artifact-producing output. Single-URL requests only, because the batch endpoint rejects it with `422`. See [Direct download](#direct-download). |
| `cache_mode` | `string` | `"enabled"` | `enabled`, `bypass`, or `refresh`. See [Caching](#caching). |

### Outputs

`outputs` accepts any combination of these names:

| Output | Returned as | What you get |
|--------|-------------|--------------|
| `markdown` | signed URL | Clean Markdown of the page. With `only_main_content` (default `true`) site chrome such as navigation, header, footer, sidebars, cookie banners, and hidden nodes is stripped behind a fidelity guard, and image URLs are rendered as their alt text. Code blocks keep their language on the fence (` ```python `) in both modes. Set `only_main_content: false` for the full page. See [Markdown quality](#markdown-quality). |
| `html_cleaned` | signed URL | The rendered HTML with scripts, styles, and boilerplate stripped. |
| `html_raw` | signed URL | The full rendered HTML exactly as the browser produced it. |
| `screenshot` | signed URL | A viewport PNG at the requested (or default) viewport size. |
| `screenshot_full_page` | signed URL | A full-page PNG capturing the entire scroll height. |
| `pdf` | signed URL | A PDF of the page. Accepts the full `pdf_options` surface (see below). |
| `links` | signed URL | A JSON array of every link found, with absolute URLs and anchor text. |
| `images` | signed URL | A JSON array of every image, with absolute `src` and `alt` text. |
| `structured` | inline JSON | Structured data extracted from the page (the `structured` response field). |

### Markdown quality

Before the page is converted, the rendered DOM is normalised so that
the Markdown reflects what a reader sees rather than how the page was
built. This runs on every render, so the result does not depend on
which extraction strategy wins for a given page.

Always applied, in both `only_main_content` modes:

- **Code fences keep their language.** The language is read from
  whichever convention the site uses (`class="language-python"`,
  `data-lang`, a bare `language` attribute, or a highlighter wrapper)
  and normalised, so ` ```python ` comes through instead of a bare
  fence.
- **Card links stay readable.** A link wrapping a heading and a
  description becomes a linked title followed by its description,
  instead of one run-together link such as
  `[DatabaseSupabase provides a full Postgres database...]`. The
  destination URL is preserved.
- **Headings stay on one line.** A heading whose text sits inside a
  nested element no longer emits a bare `##` with the text stranded
  below it.
- **Adjacent elements no longer concatenate.** Layouts that space
  their items with CSS rather than whitespace produced `YesNo` and
  `EvaluationDeploymentProduction`; those now read as separate words.
- **Invisible characters are removed**: zero-width spaces used as
  anchor labels, soft hyphens, and Private Use Area glyphs from icon
  fonts, which arrive as unprintable tokens.
- **Empty elements are dropped**: icon-only `<i>` elements that
  rendered as stray `__`, and links whose label is empty.

Additionally, with `only_main_content: true`:

- **Interface controls are removed**: buttons, tab strips, keyboard
  shortcut hints, "Copy page" / "On this page" actions, and
  "Was this page helpful? Yes/No" rating widgets. A control that
  carries real content (an FAQ question, a clickable card body) is
  kept.
- **Screen-reader-only text is removed**: skip links, and the
  "Section titled ..." labels many documentation themes attach to
  every heading.
- **Site-declared non-content is honoured**: blocks marked
  `data-nosnippet`, `data-pagefind-ignore`, or `data-noindex`, unless
  they contain headings or code.
- **Duplicate blocks are collapsed**: responsive designs that ship a
  desktop and a mobile copy of the same bar, and carousels that
  pre-render every frame, appear once.
- **Breadcrumbs and eyebrow labels above the page title are dropped.**

Deferred content is deliberately *kept*: an inactive tab panel inside
the content region holds a real code sample (the Python example in one
tab, the JavaScript one in another), so both reach the Markdown rather
than only whichever tab happened to be selected at render time.

### Rendering and waiting

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `viewport` | `object` | `1920 x 1080` | `{"width": <int>, "height": <int>}`. Width 320–3840, height 240–2160. |
| `mobile` | `boolean` | `false` | Render at a mobile viewport (390 x 844) unless `viewport` is set explicitly. |
| `wait_for` | `string` | `null` | Wait after navigation for a CSS selector (`".price"` or `"css:.price"`) or a JS expression (`"js:window.dataReady === true"`). |
| `wait_timeout_ms` | `integer` | `30000` | How long `wait_for` may wait, in milliseconds. 0–60,000. A timeout degrades to a warning; the page is captured as-is. |
| `js_code` | `string` | `null` | JavaScript to run on the page after navigation. Max 20,000 characters. An error becomes a warning, not a failure. |
| `block_resources` | `string[]` | `[]` | Resource types to abort before they load. Any of `image`, `media`, `font`, `stylesheet`, `script`, `xhr`, `fetch`, `websocket`, `manifest`, `other`. Useful for faster, text-only renders. |
| `respect_robots` | `boolean` | `false` | When `true`, a URL disallowed by the site's `robots.txt` is rejected with `403`. |
| `pdf_options` | `object` | `null` | Page format, margins, headers, footers, scale, and orientation for `pdf` output. Same object as [url-to-pdf](/docs/endpoints/convert/web-pages/url-to-pdf.md). With no `pdf_options`, perceive produces a single continuous page, byte-identical to V1 url-to-pdf. |

### Authenticated and custom requests

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `auth` | `object` | `null` | HTTP Basic Auth for the target page: `{"username": "...", "password": "..."}`. |
| `cookies` | `array` | `null` | Cookies to inject before navigation. Max 50. Each needs `name`, `value`, and either `domain` or `url`. |
| `headers` | `object` | `null` | Custom request headers. Max 20. Blocked names: `host`, `content-length`, `transfer-encoding`, `connection`, `upgrade`, `te`, `trailer`. |

<div class="alert alert-warning">
<strong>Reserved, not yet live.</strong> <code>proxy_url</code> (Production+),
<code>geolocation</code>, and <code>action_chain</code> are accepted by the
request schema but return <code>422</code> today. They arrive in a later
release; sending them now tells you exactly which knob is not ready
instead of silently ignoring it.
</div>

---

## Structured extraction

When `structured` is in `outputs`, the `extract` list controls which
fields perceive pulls. Ask for nothing and it defaults to `metadata` and
`structured_data`.

| `extract` value | Field in `structured` | Status |
|-----------------|-----------------------|--------|
| `metadata` | `metadata` | Live |
| `structured_data` | `structured_data` (JSON-LD) | Live |
| `headings` | `headings` | Live |
| `tables` | `tables` | Live |
| `main_content` | `main_content` (text, capped at 50,000 chars) | Live |
| `all` | expands to every live field above | Live |
| `prices` | -- | Not yet live: returns a warning, omitted |
| `contacts` | -- | Not yet live: returns a warning, omitted |
| `technologies` | -- | Not yet live: returns a warning, omitted |

To be straight about it: `prices`, `contacts`, and `technologies` are
reserved names. Request one today and it does not error. The name lands
in the `warnings` array and is dropped from `structured`.

### Schema-driven extraction

Send a `schema` to pull specific fields into `structured.extracted`:

```json
{
    "url": "https://example.com/product/widget",
    "outputs": ["markdown", "structured"],
    "schema": {
        "type": "object",
        "properties": {
            "name": {"type": "string"},
            "price": {"type": "number"},
            "in_stock": {"type": "boolean"}
        }
    }
}
```

The LLM extraction tier fills the schema, and it
fires only when **all** of these hold: you sent a `schema`, your plan
carries the LLM tier (Indie and up), the page was not scored as
blocked, and the heuristic pass left schema fields empty. When it runs,
`extraction_tier` is `"llm"`, and `tokens` and `cost_cents` report what
that extraction cost; otherwise `extraction_tier` is `"heuristic"` and
both are zero.

> **Note.** Schema extraction is hard-capped to protect your bill: a
> single extraction is capped per request, and project spend draws on
> your monthly AI-credit balance ($5 / $15 / $40 per month on Indie /
> Studio / Production; unused credits roll over). LLM extraction spends
> credits, not ops. If a cap is hit or the balance is exhausted, perceive
> returns the heuristic result with a note in `warnings` rather than
> overspending. On a plan without the LLM tier, you get heuristic
> `structured` data only.

---

## Response

Both `POST /v2/perceive` and `GET /v2/perceive/{operation_id}` return the
same object.

| Field | Type | Description |
|-------|------|-------------|
| `operation_id` | `string` | Opaque ID (`per_...`). Use it with the GET endpoint and quote it to support. |
| `status` | `string` | `queued`, `processing`, `completed`, or `failed`. |
| `url` | `string` | The URL you sent. |
| `url_final` | `string` | The URL after redirects. |
| `content_hash` | `string` | SHA-256 of the rendered page. Drives the 1-hour cache. |
| `render_quality` | `number` | 0.0-1.0. Scores below 0.40 mean a failed render: an anti-bot page, a login wall, an HTTP error page, a soft 404, or an empty shell. Consult `deductions` for the reason and `status_code` for the upstream status. |
| `status_code` | `integer` | HTTP status of the final main-document response (e.g. `200`, `404`). `null` when unknown. |
| `deductions` | `object` | Named render-quality deductions that fired, e.g. `{"http_error": 0.7}`, `{"soft_404": 0.65}`, `{"login_wall": 0.65}`. Empty on a clean render. |
| `options_echo` | `object` | Echo of the request options the server honoured. Secrets are redacted to booleans (`auth_provided`, `cookies_provided`, `headers_provided`, `js_code_provided`, `schema_provided`, `pdf_options_provided`). The plain options (`outputs`, `only_main_content`, `truncate_data_arrays`, `allow_degraded`, `extract`, `cache_mode`, `mobile`, `respect_robots`, `direct_download`, `wait_for`, `wait_timeout_ms`, `viewport`, `block_resources`) are echoed as honoured. `truncate_data_arrays` is echoed as the **resolved** boolean, so leaving it unset still tells you which way it went. |
| `cache_hit` | `boolean` | `true` when the result came from the cache instead of a fresh render. |
| `outputs` | `object` | Map of output name to `{url, object_key, size_bytes, content_type, expires_in}`. Signed URLs expire in 900 seconds. |
| `structured` | `object` | Inline structured data, present when `structured` was requested. |
| `extraction_tier` | `string` | `heuristic`, `css`, or `llm`. |
| `tokens` | `object` | `{input, output}` LLM tokens used. Zero unless the LLM tier ran. |
| `cost_cents` | `number` | LLM cost in cents for this operation. Zero unless the LLM tier ran. |
| `duration_ms` | `integer` | End-to-end render time. |
| `error` | `string` | Set only when `status` is `failed`. |
| `warnings` | `string[]` | Non-fatal notes: a `wait_for` timeout, a skipped extract, a blocked-page flag, an `only_main_content` fallback to the full page, a note that long numeric data arrays were truncated. |

---

## Retrieve an operation

Signed URLs expire after 15 minutes. To download an output later, re-fetch
the operation and perceive re-signs every URL from the stored object keys.
No re-render happens, so this does not consume ops.

```bash
curl https://api.enconvert.com/v2/perceive/per_3f9a2c1b8e7d4a6f90b1c2d3e4f5a6b7 \
  -H "X-API-Key: sk_your_private_key"
```

An unknown operation ID, or one that belongs to a different project,
returns `404`. Existence is never leaked across projects.

---

## Direct download

By default every file output comes back as a pre-signed URL you fetch in
a second request. Set `direct_download: true` on the POST to skip the
envelope: the HTTP response body **is** the artifact bytes, with no JSON,
no signed URL, and no second fetch. The request must produce exactly one
artifact output (`outputs: ["markdown"]`, `outputs: ["pdf"]`, …),
otherwise it is rejected with `400`. The metadata that would have been
in the JSON rides in response headers instead: `Content-Disposition`,
`X-Operation-Id`, `X-Object-Key`, `X-Cache-Hit`, `X-Render-Quality`,
`X-Source-Status-Code`, `X-Content-Hash`, and `X-Warnings-Count`.

```bash
curl -X POST https://api.enconvert.com/v2/perceive \
  -H "X-API-Key: sk_your_private_key" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/blog/post",
    "outputs": ["markdown"],
    "direct_download": true
  }' \
  -o post.md
```

The GET endpoints stream stored artifacts the same way:

- `GET /v2/perceive/{operation_id}?direct_download=true&output=markdown`
  streams one artifact from a past operation. `output` is required when
  the operation produced more than one artifact. An artifact past your
  plan's retention window answers `410`.
- `GET /v2/perceive/batch/{job_id}?direct_download=true` streams the
  batch ZIP for `output_mode: "zip"` batches whose archive is ready, and
  answers `400` otherwise.

`direct_download` is for single URLs only: `POST /v2/perceive/batch`
rejects it with `422`. Set `output_mode` to `"zip"` and download the
archive instead. See [Batch perception](#batch-perception).

---

## Batch perception

`POST /v2/perceive/batch` perceives a list of URLs that share one
`options` block. Each URL is rendered through the same pipeline as a
single call and produces its own operation row.

```bash
curl -X POST https://api.enconvert.com/v2/perceive/batch \
  -H "X-API-Key: sk_your_private_key" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": [
      "https://example.com/a",
      "https://example.com/b",
      "https://example.com/c"
    ],
    "options": {"outputs": ["markdown"]},
    "output_mode": "manifest"
  }'
```

Batches of 10 URLs or fewer run inline and answer `200` with every result
populated. Larger batches answer `202` with a `job_id`; the URLs are
drained one at a time and you poll for results:

```bash
curl https://api.enconvert.com/v2/perceive/batch/{job_id} \
  -H "X-API-Key: sk_your_private_key"
```

The batch response reports aggregate progress and carries one full
perceive result per URL once rendered:

```json
{
    "job_id": "bat_8c1a...",
    "status": "partial",
    "output_mode": "manifest",
    "total": 3,
    "completed": 2,
    "failed": 1,
    "pending": 0,
    "items": [
        {"operation_id": "per_...", "status": "completed", "url": "https://example.com/a"},
        {"operation_id": "per_...", "status": "completed", "url": "https://example.com/b"},
        {"operation_id": "per_...", "status": "failed", "url": "https://example.com/c"}
    ]
}
```

`status` is `queued`, `processing`, `completed`, `failed`, `partial`
(some URLs succeeded, some failed), or `canceled`. Set `output_mode` to
`zip` to bundle every artifact into a single ZIP, returned under the `zip`
field once the batch finishes.

### Durable and resumable

Batches are restart-safe. If the service restarts while a batch is
in flight, the batch **resumes automatically** and re-renders only the
URLs that had not finished, so already-completed URLs keep their artifacts.
You never need to resubmit a batch because of a restart.

### Cancelling a batch

`DELETE /v2/perceive/batch/{job_id}` cancels a running batch. The worker
stops between URLs, so URLs already rendered keep their results and the
rest are left unstarted. The call is idempotent, so cancelling a batch
that has already finished simply returns its current state, and the
batch's `status` becomes `canceled`.

```bash
curl -X DELETE https://api.enconvert.com/v2/perceive/batch/{job_id} \
  -H "X-API-Key: sk_your_private_key"
```

---

## Caching

`cache_mode` controls how perceive treats its 1-hour result cache, keyed
by your project, the URL, and the render-affecting request options.

| `cache_mode` | Behaviour |
|--------------|-----------|
| `enabled` (default) | Return a cached result when an identical request rendered within the last hour. `cache_hit` is `true`, `cost_cents` is `0`. |
| `bypass` | Skip the cache and render fresh. |
| `refresh` | Render fresh and replace the cached entry. |

Worth flagging: a cache hit still bills one op against your monthly
ops allowance. The allowance meters operations rather than browser
renders, so the cache saves you render time, not ops.

---

## Code examples

### curl: Markdown only

```bash
curl -X POST https://api.enconvert.com/v2/perceive \
  -H "X-API-Key: sk_your_private_key" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/blog/post",
    "outputs": ["markdown"]
  }'
```

### curl: Markdown plus structured data

```bash
curl -X POST https://api.enconvert.com/v2/perceive \
  -H "X-API-Key: sk_your_private_key" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/pricing",
    "outputs": ["markdown", "structured"],
    "extract": ["metadata", "structured_data", "tables"]
  }'
```

### curl: Full outputs plus PDF

```bash
curl -X POST https://api.enconvert.com/v2/perceive \
  -H "X-API-Key: sk_your_private_key" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/report",
    "outputs": ["markdown", "html_cleaned", "screenshot_full_page", "pdf"],
    "pdf_options": {"format": "A4", "print_background": true}
  }'
```

### Python

```python
import requests

response = requests.post(
    "https://api.enconvert.com/v2/perceive",
    headers={"X-API-Key": "sk_your_private_key"},
    json={
        "url": "https://example.com/pricing",
        "outputs": ["markdown", "structured"],
        "extract": ["metadata", "tables"],
    },
)
response.raise_for_status()
data = response.json()

# Download the Markdown artifact from its signed URL
markdown_url = data["outputs"]["markdown"]["url"]
markdown_text = requests.get(markdown_url).text

print(data["structured"])
print(markdown_text)
```

### Node.js

```javascript
const res = await fetch("https://api.enconvert.com/v2/perceive", {
    method: "POST",
    headers: {
        "Content-Type": "application/json",
        "X-API-Key": "sk_your_private_key"
    },
    body: JSON.stringify({
        url: "https://example.com/pricing",
        outputs: ["markdown", "structured"],
        extract: ["metadata", "tables"]
    })
});

const data = await res.json();

// Download the Markdown artifact from its signed URL
const markdownText = await fetch(data.outputs.markdown.url).then(r => r.text());

console.log(data.structured);
console.log(markdownText);
```

If you call EnConvert from Claude, Cursor, or another MCP client, the
same capability is exposed as the `perceive_url` tool. See [the MCP
server page](/mcp.md).

---

## Error responses

| Status | Condition |
|--------|-----------|
| `400 Bad Request` | URL is not `http(s)`, carries embedded credentials, or resolves to a private, loopback, or link-local address (SSRF protection). |
| `400 Bad Request` | Invalid `auth` (missing `username`/`password`), `cookies` (not an array, over 50 entries, missing fields), or `headers` (not an object, over 20 entries, blocked name). |
| `401 Unauthorized` | Missing or invalid API key / JWT token. |
| `402 Payment Required` | Perceive is not on your current plan, or your monthly ops allowance is exhausted. |
| `403 Forbidden` | `/v2/perceive` is not in the API key's allowed endpoints. |
| `403 Forbidden` | Batch not available on your plan, or batch size exceeds your plan's limit. |
| `403 Forbidden` | `respect_robots=true` and the site's `robots.txt` disallows the URL. |
| `404 Not Found` | Unknown `operation_id` or `job_id`, or one owned by another project. |
| `422 Unprocessable Entity` | Request validation failed (bad enum in `outputs`/`extract`, `wait_timeout_ms` out of range, viewport out of bounds, an unknown request key). |
| `422 Unprocessable Entity` | `proxy_url`, `geolocation`, or `action_chain` was sent. All three are reserved for a later release. |
| `500 Internal Server Error` | The render failed. The message includes the `operation_id` to quote to support. |
| `502 Bad Gateway` | Every engine was blocked and the origin served an anti-bot challenge with no page content behind it. Retry later, or send `allow_degraded: true` to receive the challenge page as-is. |

Unknown request keys are rejected with a `422` naming the field, on
`/v2/perceive`, `/v2/perceive/batch`, `/v2/discover`, and `/v2/lookup`
alike. They are never silently ignored. Every `422` body carries a
top-level `errors` array of human-readable messages alongside the raw
`detail` list.

The full status-code reference is in [the error-codes guide](/docs/reference/errors.md).

---

## Limits

| Limit | Value |
|-------|-------|
| URL length | 2,048 characters |
| `wait_timeout_ms` | 0–60,000 ms |
| `js_code` length | 20,000 characters |
| Viewport width | 320–3,840 px |
| Viewport height | 240–2,160 px |
| Cookies per request | 50 |
| Custom headers per request | 20 |
| `main_content` extract | 50,000 characters |
| Batch URLs per request | 1,000 (schema cap) |
| Inline batch threshold | 10 URLs (larger batches run asynchronously) |
| Result cache TTL | 1 hour |
| Signed URL expiry | 15 minutes |
| Monthly ops (shared across every endpoint) | 500 / 3,000 / 15,000 / 50,000 by tier; see [pricing](/pricing.md) |

---

## Frequently asked questions

### How do I convert a web page to Markdown with a REST API?

Send `POST /v2/perceive` with `{"url": "...", "outputs": ["markdown"]}`. The page is rendered in headless Chrome and the response carries a pre-signed download URL for the Markdown file. By default `only_main_content` strips site chrome so you get the article, not the navigation; set `"only_main_content": false` for the full page, or add `"direct_download": true` to receive the Markdown bytes directly in the response body.

### Can I get a screenshot and Markdown from the same render?

Yes. `outputs` accepts any combination, so `["markdown", "screenshot"]` (or `screenshot_full_page` for the entire scroll height) produces both from a single browser render. You never pay for the same page twice in one call.

### Does /v2/perceive render JavaScript pages?

Yes. Every request runs a real headless-Chrome render: cookie banners are dismissed, the page is scrolled to trigger lazy-loaded content, and you can control the page before capture with `wait_for` (a CSS selector or JS expression), `js_code`, and `block_resources`.

### Why did my signed download URL stop working?

Signed URLs expire after 15 minutes (`expires_in: 900`). Re-fetch the operation with `GET /v2/perceive/{operation_id}` to get freshly signed URLs. No re-render happens and no ops are consumed.

### Does a cached result still count against my quota?

Yes. A cache hit bills one op, because the monthly allowance meters operations rather than browser renders. Set `cache_mode` to `bypass` to skip the 1-hour cache, or `refresh` to render fresh and replace the cached entry.