---
title: "EnConvert Changelog"
description: "New features, improvements, and fixes shipped in EnConvert."
canonical: "https://www.enconvert.com/changelog"
locale: "en"
---

# EnConvert Changelog

New features, improvements, and fixes shipped in EnConvert, newest first. Mirrored as markdown for agents; the HTML page at https://www.enconvert.com/changelog is paginated and filterable.

## Cleaner ingest chunks

- Published: 2026-08-16
- Tags: improved, fixed
- Permalink: https://www.enconvert.com/changelog#cleaner-ingest-chunks

Ingest now builds its markdown with the same pipeline perceive uses and renders pages in a real browser, so JavaScript-rendered docs sites arrive complete and page titles, standfirsts and link-only table cells stop disappearing while cookie banners, "Was this page helpful?", icon-font glyphs and empty `##` headings stay out of the chunks.

- **Nothing fails quietly any more.** A `text/plain` or JSON URL such as `llms.txt` keeps its line structure instead of collapsing onto one line and chunking to nothing, a document made entirely of headings is chunked rather than shipped as an empty file, a page with no extractable text is reported as skipped instead of completing silently, and two URLs that redirect to the same page are ingested and billed once.
- **Uploaded files improved alongside.** PDFs no longer turn a rotated margin stamp into a heading, extract a figure as a table of reversed one-letter cells, or promote prose to headings in documents that mix type sizes, and landscape pages are extracted rather than dropped; Word heading styles no longer leak `**` into chunk metadata, embedded images no longer inline megabytes of base64, PowerPoint soft line breaks no longer leave control characters, and a saved page uploaded as HTML gets the same treatment as crawling it.
- **Two new options,** matching perceive's: `only_main_content` and `truncate_data_arrays`.

## Cleaner perceive markdown

- Published: 2026-08-13
- Tags: improved, fixed
- Permalink: https://www.enconvert.com/changelog#cleaner-perceive-markdown

Perceive's markdown now reflects what a reader sees rather than how the page was built, and it does so on every page instead of only the ones a particular extractor happened to win. Interface furniture is gone under only_main_content: buttons and tab strips, "Copy page" and "On this page" actions, keyboard shortcut hints, "Was this page helpful?" rating widgets, screen-reader-only labels such as the "Section titled ..." link many docs themes attach to every heading, skip links, breadcrumbs, and blocks a site marks data-nosnippet or data-pagefind-ignore. Structure holds up too: code fences keep their language whichever convention a site uses, so ```python arrives instead of a bare fence; a card link becomes a linked title followed by its description instead of one run-together [DatabaseXYZ provides...] link, with the destination URL preserved; headings stay on one line rather than emitting a bare ## with the text stranded below; and adjacent elements spaced by CSS no longer concatenate into YesNo or EvaluationDeploymentProduction. Zero-width spaces, icon-font glyphs and empty elements that rendered as stray __ are dropped, and duplicate blocks from responsive desktop/mobile twins are collapsed. Inactive tab panels are now kept, so a page's Python and JavaScript samples both reach the markdown instead of only whichever tab was selected at render time. Two new options: truncate_data_arrays collapses long numeric runs such as raw embedding vectors printed in notebook output cells to a leading sample plus a count, following only_main_content unless you set it explicitly, and allow_degraded controls whether an anti-bot challenge with no page content behind it is returned as-is or fails with 502 instead of passing the interstitial off as the page. 

Measured across fourteen live documentation and marketing pages, output shrank 67% overall, with one notebook page dropping 94% once its embedding vectors were truncated; the two pages that grew did so because content had previously been lost, regaining a dropped H1, twelve section headings and a table.

## Perceive returns main content by default

- Published: 2026-08-07
- Tags: new, improved, fixed
- Permalink: https://www.enconvert.com/changelog#perceive-returns-main-content-by-default

The markdown_fit output is gone. Perceive's markdown now strips navigation, headers, footers, sidebars and cookie banners on its own, controlled by only_main_content, which defaults to true; send false for the untouched page, and a strip that removes too much falls back to the full page with a warning. direct_download is new on single-URL perceive and returns the artifact bytes in the same response instead of a signed URL round trip, as long as exactly one artifact output was requested. Results now carry status_code, deductions and options_echo, so a 404, a soft 404 or a login wall shows up as the reason a render scored low, and those pages no longer reach tier 3 extraction. Unknown request keys return 422 naming the field instead of being quietly ignored. Shipped in CLI 1.1.0, MCP server 0.5.0, the n8n node 1.1.0 and nine SDKs.

## MCP server 0.3.1 ships a routing guide

- Published: 2026-08-03
- Tags: new, improved, fixed
- Permalink: https://www.enconvert.com/changelog#mcp-server-0-3-1-ships-a-routing-guide

@enconvert/mcp 0.3.1 includes SKILL.md, a routing table telling an agent which of the 24 tools to reach for. It exists because agents were calling convert_document on web pages and looping perceive_url one URL at a time instead of using perceive_batch. It also names the failures that are hard to diagnose from an error message alone: file paths must be absolute, a pk_ key authenticates but 403s on all 18 web tools, quota errors are not worth retrying, and watchers cannot run more often than every 60 minutes. The file ships inside the npm tarball, so it arrives with the install. Listing in the MCP Registry works again as well, after the server description was cut to the 100 character limit that had rejected 0.3.0 with a 422.

## EnConvert node for n8n

- Published: 2026-08-03
- Tags: new
- Permalink: https://www.enconvert.com/changelog#enconvert-node-for-n8n

@enconvert/n8n-nodes-enconvert 1.0.0 is on npm as an n8n community node. One node covers 16 operations across files, images, web pages, whole websites, search and jobs, with an EnConvert API credential whose Test button calls /v1/whoami. An unsupported conversion fails in the editor with a list of what that file can be converted to, instead of a server error. Scrape results are inlined, so a screenshot arrives as a real binary field and markdown as {{ $json.markdown }} with no second HTTP Request node, and anything too big for n8n Cloud's 16 MB item limit can come back as a link that expires after 15 minutes. PDF page options an office input cannot honor are dropped with a warning on the item rather than erroring. Crawl output can be emitted one chunk per item straight into a vector store. The node attaches to an AI Agent node as a tool, and it ships zero runtime dependencies, which is what n8n Cloud requires.

## Markdown conversions are smaller and faster

- Published: 2026-08-03
- Tags: improved, fixed
- Permalink: https://www.enconvert.com/changelog#markdown-conversions-are-smaller-and-faster

markdown-to-html no longer runs Pygments highlighting. The generated page never included a Pygments stylesheet, so all of those token spans rendered with no color at all while making the HTML about 3.6x larger and peak memory about 6x higher. A 763 KB file with 128 code blocks took 29 seconds and 1578 MB, and now takes 5 seconds and 252 MB. Fenced blocks come out as , which client side highlighters like Prism or highlight.js can pick up, and code heavy documents that used to die with a 504 convert. PDF to Markdown releases each page's cache as it goes and pulls table content out immediately instead of holding live page objects for the whole document, so peak memory tracks the largest page rather than the file. Spreadsheet and CSV to Markdown stop parsing at the 5000 row cap. One edge case changed there: blank rows are dropped after the cap, so a sheet with blank rows scattered through the first 5001 rows renders fewer body rows than it used to.

## A failed batch creation no longer strands queued pages

- Published: 2026-08-03
- Tags: fixed
- Permalink: https://www.enconvert.com/changelog#a-failed-batch-creation-no-longer-strands-queued-pages

The batch envelope is written before the per-URL rows now. With the old order, a failure between the two steps left operation rows that nothing could resume or sweep, because the resume path only walks envelopes. Those rows consumed the caller's quota and sat in queued forever.

## Batch credentials are no longer stored

- Published: 2026-08-03
- Tags: security, fixed
- Permalink: https://www.enconvert.com/changelog#batch-credentials-are-no-longer-stored

Basic auth, cookies and headers sent with a /v2/perceive batch were written into the job record as JSON and outlived the request, readable by anyone with database or backup access. They are stripped before the record is saved, and only the names of the dropped keys are kept. If the gateway restarts mid-batch and that batch used credentials, its unfinished URLs come back as failed with batch used credentials that are not stored; resubmit the batch instead of quietly rendering login walls for pages you paid to have rendered authenticated.

## Reusing a job_id returns 409 instead of clobbering a row

- Published: 2026-08-03
- Tags: fixed, security
- Permalink: https://www.enconvert.com/changelog#reusing-a-job-id-returns-409-instead-of-clobbering-a-row

job_id is client supplied, so re-sending your own after a timeout is normal, and it now resets the poll row idempotently instead of logging a duplicate key violation. A job_id that belongs to a different project returns 409 job_id already in use and leaves that project's row untouched. The success and failure writers are scoped by project as well, so a guessed ID can no longer flip someone else's job to failed with a caller supplied error message, or point it at a download URL signed for the wrong project.

## Runaway renders and post-processing have hard ceilings

- Published: 2026-08-03
- Tags: new, fixed
- Permalink: https://www.enconvert.com/changelog#runaway-renders-and-post-processing-have-hard-ceilings

Full page screenshots are capped at 25000 px tall and single page PDFs at 60000 px, tunable with SCREENSHOT_MAX_HEIGHT_PX and PDF_MAX_SINGLE_PAGE_HEIGHT_PX. Infinite scroll pages report heights of 50000 px and up, and a full page raster costs width times height times 4 bytes in one allocation, which took the whole worker down instead of returning anything. Ghostscript grayscale post-processing now times out after 120 seconds, and the child process is killed and reaped on timeout or cancellation, so a pathological PDF fails with an explicit error instead of pinning CPU and outliving the request that started it.

## Large files stream instead of being buffered

- Published: 2026-08-03
- Tags: improved, fixed
- Permalink: https://www.enconvert.com/changelog#large-files-stream-instead-of-being-buffered

The download proxy at /v1/convert/download streams from storage in 64 KB chunks instead of reading the whole object into memory first, so time to first byte drops on large files and several clients slowly pulling 150 MB files no longer risk an out of memory. Batch URL jobs, /v2/perceive zip bundles and /v2/ingest assembly write to a temp file as each result finishes and upload it as a stream, so at most one result is resident at a time. Multi-page PDF to JPEG archives are stored rather than deflated, which makes them marginally larger on the wire and quicker to produce, since JPEG does not deflate anyway. Image to PDF hands the image to the renderer as a file instead of a base64 data URI, which used to materialise about five copies of it. The S3 client is built once per process instead of once per call, which takes a TLS handshake off every upload, download, delete and health check.

## Upload size limits measure the real file

- Published: 2026-08-03
- Tags: improved, security, fixed
- Permalink: https://www.enconvert.com/changelog#upload-size-limits-measure-the-real-file

Fifty upload endpoints were reading the Content-Length header, which measures the whole multipart envelope, and a client that sends no Content-Length at all, which includes chunked and HTTP/2 uploads, was not size checked at all. They now measure the uploaded file's own bytes, so plan ceilings (5 MB Free, 15 MB Starter, 50 MB Pro, 150 MB Business) apply on every upload endpoint and the 413 boundary is exact. Admission control weighs bytes as well as request count: with more than 256 MB of conversions in flight the API answers 503 with Retry-After: 10 instead of running the box out of memory. That budget is tunable with MAX_PENDING_CONVERSION_BYTES.

## Images over 40 megapixels are rejected before decoding

- Published: 2026-08-03
- Tags: security, fixed
- Permalink: https://www.enconvert.com/changelog#images-over-40-megapixels-are-rejected-before-decoding

Fourteen image conversion entry points now check the declared canvas size right after reading the file header and return HTTP 400 before allocating a single pixel, with the message Image is too large to process: WxH (N pixels) exceeds the 40000000 pixel limit. A kilobyte upload can legally declare a canvas that decodes to several gigabytes, and Pillow's own bomb detector only fires around 358 MP. This is a real narrowing: a legitimate 50 MP camera photo that used to convert is now rejected. SVGs behave differently. An intrinsic canvas over 40 MP is scaled down with the aspect ratio preserved rather than refused, and an SVG whose intrinsic size cannot be read renders at 2048 px wide. The ceiling is tunable with IMAGE_MAX_PIXELS everywhere except compress-image, which keeps its own 40 MP limit and its own wording.

## Watchers stop reporting changes that never happened

- Published: 2026-08-03
- Tags: fixed, new
- Permalink: https://www.enconvert.com/changelog#watchers-stop-reporting-changes-that-never-happened

A watch check is a diff against a stored baseline, so it now always captures with Chromium, the same engine that captured that baseline. Before this a check could fall through to the no-browser fetch path, see raw un-hydrated HTML, and fire a change webhook and email for a page that had not changed. The trade-off is that every watch check consumes a Chromium slot. Render quality scoring also recognises an un-hydrated single page app shell: a framework mount node holding fewer than 20 words on a page under 500 visible words scores 0.30 instead of 1.00, which is below the quality floor, so /v2/perceive escalates to a real browser render and a bad watch check is recorded for audit without being diffed or notified on.

## The no-browser fetch path works again

- Published: 2026-08-03
- Tags: fixed, security
- Permalink: https://www.enconvert.com/changelog#the-no-browser-fetch-path-works-again

The TLS fetch engine passed a resolve= argument that curl_cffi does not accept, so every request on that path raised a TypeError and fell through to a full Chromium render. It had been failing 100% of the time for about two weeks, and the raw Python error text was being appended to the warnings array of /v2/perceive responses. Eligible HTML-only pages now skip the browser, which leaves the single Chromium slot for pages that actually need it. The DNS pin that stops a redirect from rebinding to an internal address between validation and connect also takes effect for the first time. It is reapplied on every hop, cleared when a hop does not resolve, skipped for bare IP hosts, and IPv6 addresses are bracketed so the entry parses.

## Pages with strict CSP render again

- Published: 2026-08-03
- Tags: fixed, improved, security
- Permalink: https://www.enconvert.com/changelog#pages-with-strict-csp-render-again

The renderer injects a small stylesheet to fix print colors and hide modals. That injection raced Chromium's CSP error stream and threw whenever any frame logged a violation, measured at 9 of 20 renders on a page whose own main frame carries no policy at all, and it failed the entire conversion over a cosmetic style. Injection now runs inside the page and a failure is logged and skipped. CSP is also disabled per page over the DevTools protocol, which covers policies delivered by  that no response rewrite can reach. The request interceptor no longer refetches and buffers every image, script and stylesheet just to strip a header that only matters on the main document. That removes 100 to 200 MB of copying per render on media heavy pages, and subresource requests now reach the SSRF re-validation check that never saw them before.

## Renders killed by a timeout or a disconnect get cleaned up

- Published: 2026-08-03
- Tags: fixed
- Permalink: https://www.enconvert.com/changelog#renders-killed-by-a-timeout-or-a-disconnect-get-cleaned-up

When a request hit the 300 second gateway timeout or the client went away, the page being rendered stayed alive and kept executing the site's JavaScript, so every later conversion competed with it. A cancelled render now queues cleanup that waits for the conversion slot to free: a healthy browser gets its leftover pages parked, a browser that fails its health probe gets relaunched. Browser shutdown is bounded at 10 seconds, so a wedged Chromium can no longer hang recovery or leave the browser manager half torn down and failing every request after it. The client still gets its 504 immediately.

## The EnConvert CLI: the whole API from your terminal

- Published: 2026-07-25
- Tags: new
- Permalink: https://www.enconvert.com/changelog#enconvert-cli

### Summary

EnConvert now has an official command-line interface. `enconvert` is a single, open-source (MIT) binary that covers the whole API from your terminal: file conversion across 40+ formats, URL and website rendering to PDF, screenshot or markdown, and the full v2 web-data surface (perceive, discover, lookup, distill, ingest). It installs natively on macOS, Linux and Windows, needs nothing but your existing secret API key, and sends no telemetry. Nothing in the API itself changes.

#### What changed

- **One command for every conversion.** `enconvert convert report.docx --to pdf` infers the endpoint from the input extension and target format across all 46 file-conversion routes (documents, spreadsheets, presentations, images, data formats, compression). Batch globs, `-O` output directories, `--skip-existing` and per-file progress are built in.
- **URL and website rendering.** `enconvert url pdf|screenshot|markdown <url>` exposes every render option (viewport, selectors, cookies, headers, basic auth, ad blocking, PDF geometry) 1:1 with the API, and `enconvert site pdf|screenshot` drives the async website crawls with `--wait` polling and ZIP downloads.
- **The v2 web-data verbs, first class.** `perceive` (including batches with 200/202 handling), `discover`, `lookup` (with enrichment and answer synthesis), `distill` (schema, prompt and CSS-schema modes) and the complete `ingest` family including file uploads and webhook-secret management.
- **Scripting-grade plumbing.** Stable documented exit codes, `--json` with the gateway's raw response, a bundled `--jq` filter (no jq install required), `--jsonl` streaming, paths-on-stdout output rules, `jobs wait <id>` for any job kind, and `enconvert api` — a gh-style passthrough that reaches every endpoint, including ones without a typed command.
- **Native installs on every platform.** Homebrew (`brew install enconvert/tap/enconvert`), Scoop, Winget, a checksum-verifying `curl -fsSL https://get.enconvert.com/install.sh | sh`, and `npm i -g @enconvert/cli`. Standalone binaries for macOS (Intel and Apple Silicon), Linux (glibc and musl, x64 and arm64) and Windows.
- **Profiles, config and safe credentials.** `~/.config/enconvert/config.toml` profiles, project-level `.enconvertrc.toml`, a `credential_helper` hook for 1Password/pass/Vault, 0600-permission key storage, and automatic migration of keys already saved by `npx @enconvert/mcp setup`.
- **No telemetry.** The CLI makes no requests other than the API calls you ask for and an optional once-daily version check that a single environment variable disables.

#### Scope

This is a new client only. No `/v1/*` or `/v2/*` endpoint behavior changed. The CLI covers every working v1 endpoint and every v2 endpoint except `/v2/watch` (watchers remain dashboard-managed for now; typed commands for them will follow). Source: https://github.com/enconvert/cli.

#### Docs

A new [CLI documentation page](/docs/cli.md) was added in all five languages (en, fr, de, es, it), the CLI joined the Integrations section of the homepage with a dedicated [/integrations/cli](/integrations/cli.md) page, and the SDKs section of the docs sidebar now lists the CLI alongside the Node.js SDK and MCP server.

## Image Compression and SVG Sizing

- Published: 2026-07-24
- Tags: new, improved
- Permalink: https://www.enconvert.com/changelog#image-compression-and-svg-sizing

### Summary

Two additions to the V1 conversion API, both synchronous `multipart/form-data` file conversions gated only by the global plan conversion limit. `POST /v1/convert/compress-image` compresses PNG, JPEG and WebP files without ever changing their format — lossless-first, with an optional `target_size_kb` budget met by aspect-ratio-locked downscaling. The three SVG rasterization endpoints (`svg-to-png`, `svg-to-jpeg`, `svg-to-webp`) gain optional `width` and `height` parameters that control the output dimensions in pixels; the playground exposes matching size inputs with an aspect-ratio lock derived from the uploaded SVG.

#### `POST /v1/convert/compress-image`

**Accepted extensions (4):** `.png` `.jpg` `.jpeg` `.webp` — magic-byte checked. The output keeps the input's extension and format; a file whose content does not match its extension is rejected with `400` rather than silently converted. Animated inputs (APNG, animated WebP) are rejected with `400` rather than silently flattened to their first frame.

**Request:** `file` (required). `target_size_kb` (optional; a zero/negative value is a `400` rejected before quota is burned, a non-numeric value is a `422` from request validation). `output_filename` (default: input basename; the input's extension is preserved). `job_id` (optional, enables status polling). `direct_download` (accepted for parity with the other file endpoints; responses currently always take the JSON path below, same as every V1 file conversion).

**Behavior — stage 1, lossless (always runs):** metadata (EXIF, XMP, PNG text chunks) is stripped; the ICC color profile and the EXIF orientation flag are preserved — orientation is re-emitted as a minimal single-tag EXIF block instead of being baked into pixels, which keeps the JPEG path free of extra quantization loss. PNG re-encodes at zlib level 9 + `optimize`, plus a palette candidate accepted only when the palette roundtrip is provably pixel-identical. JPEG re-encodes reusing the original quantization tables (`quality='keep'`) with optimized progressive Huffman coding. WebP re-encodes as true lossless VP8L at maximum effort. The smallest of the original bytes and all candidates wins, so **the output is never larger than the input**.

**Behavior — stage 2, dimension reduction (only when `target_size_kb` is set and stage 1 missed it):** LANCZOS downscale with the aspect ratio locked; the scale factor is binary-searched (up to 8 iterations, minimum scale 1%) for the largest dimensions that fit the budget. Downscaled JPEG and lossy-source WebP re-encode at quality 85; PNG and lossless-source WebP stay lossless at the reduced size (lossless vs lossy WebP sources are told apart by walking the RIFF chunk list for VP8L). **An unreachable target returns the smallest file achieved, not an error** — check `file_size` / `X-File-Size` to see what was reached.

**Response:** `200` with JSON — `presigned_url`, `object_key`, `filename` (timestamped, input extension preserved, e.g. `photo_20260717_101530123.png`), `file_size`, `conversion_time_seconds`, `job_id`. The same values are mirrored on the `X-Object-Key`, `X-File-Size`, `X-Conversion-Time`, `X-Filename` headers.

**Constraints:**

- The decoded canvas is capped at 40,000,000 pixels (e.g. 8000x5000), checked from the image header before any pixels are decoded, so a decompression bomb is rejected with `400` without allocating the full surface.
- Whole-request size is validated against the plan's max file size via `Content-Length` (`413`).
- The shared conversion concurrency gate applies: at capacity the endpoint returns `503` with `Retry-After: 10`.
- WebP lossless encoding uses maximum effort (`method=6`) up to 4 MP and drops to `method=4` above it, bounding CPU on large canvases; all requests are still bounded by the 300s gateway timeout (`504`, then poll with `job_id`).
- CMYK JPEGs are re-encoded in CMYK (no mode change); 16-bit PNGs skip the palette candidate and only get the plain lossless re-encode.

#### `width` / `height` on `svg-to-png`, `svg-to-jpeg`, `svg-to-webp`

**Request:** optional `width` and `height` integer form fields, `1`–`10000` each. One dimension alone scales the render proportionally — the other is derived from the SVG's own aspect ratio (CairoSVG native behavior). Both together set the exact canvas size, which may change the aspect ratio. Omitting both keeps the previous behavior (the SVG's `width`/`height`/`viewBox` attributes decide).

**Constraints:**

- Total output is capped at 25,000,000 pixels (`400`). For single-dimension requests the derived dimension is estimated server-side from the root `<svg>` `width`/`height` attributes (absolute units only) or `viewBox`, so an extreme-ratio SVG cannot request an unbounded render surface. If the ratio cannot be determined from those (e.g. the SVG sizes itself in `em`/`ex`/`%`, which CairoSVG can still resolve into a large canvas), a single-dimension request is rejected with `400` and asked to supply both `width` and `height` — the size must then be explicit and self-bounding.
- Validation runs before quota is burned or an activity row is logged; out-of-range dimensions are pure client errors.
- The converters' signatures stay backward-compatible — existing calls without `width`/`height` are byte-for-byte unaffected.

**Playground:** the three SVG conversions now show width/height inputs whose placeholders display the uploaded SVG's intrinsic size, plus a **Lock aspect ratio** toggle (on by default). The lock reads the SVG's `width`/`height` attributes client-side, falling back to the `viewBox`, and disables itself when neither yields a usable ratio. Blank fields mean "the SVG's own size". The controls are localized in all five languages.

#### Docs

Every affected page was updated in all five languages (en, fr, de, es, it): the three SVG endpoint pages now document `width`/`height` (parameter table, resolution notes, FAQs), a new `compress-image` endpoint page follows the standard skeleton, and the image-conversions category page, endpoints overview, parameters reference ("Image Options" section) and endpoint counts (48 → 49) were updated accordingly.

## Render Engine Ladder Added

- Published: 2026-07-24
- Tags: improved, new
- Permalink: https://www.enconvert.com/changelog#render-engine-ladder-added

### Summary

The V2 rendering endpoints — [perceive](/docs/v2-perceive.md), [distill](/docs/v2-distill.md), [ingest](/docs/v2-ingest.md), and [watch](/docs/v2-watch.md) — now render each URL through an automatic multi-engine fallback instead of a single headless-browser pass. A page is first fetched over a real-browser TLS fingerprint; if that path is blocked or the page needs JavaScript, the request escalates to a headless-Chrome render, and a page that still looks blocked by anti-bot protection escalates once more to a stealth-hardened render. The result: more real-world pages return usable content, and simple pages come back faster. Nothing in your request changes — this is automatic, and every existing call behaves the same or better.

#### What changed

- **TLS-first fast path.** Static and server-rendered pages are fetched over a real-browser TLS/HTTP-2 fingerprint with no headless browser involved. These pages return faster and free browser capacity for the pages that genuinely need it.
- **Automatic browser fallback.** If the fast path is blocked, hits a bot wall, or the page renders empty (a client-side JavaScript app), the request transparently escalates to the headless-Chrome render — the same capture pipeline (cookie-banner dismissal, lazy-load scrolling, sticky-header handling, image waiting) as before.
- **Stealth escalation for anti-bot pages.** A render that still looks blocked by anti-bot protection is retried once with additional browser-fingerprint hardening, recovering pages a plain render could not reach.
- **Same signals, same shape.** The `render_quality` score and the blocked-page warning still tell a real render from a challenge page, and no response field changes. When every engine is still blocked, the best attempt is returned and flagged — exactly as before.

#### Scope

This applies to the V2 rendering endpoints only: [perceive](/docs/v2-perceive.md), [distill](/docs/v2-distill.md), [ingest](/docs/v2-ingest.md), and [watch](/docs/v2-watch.md). The V1 [url-to-pdf](/docs/endpoints/web-pages/url-to-pdf.md), [url-to-screenshot](/docs/endpoints/web-pages/url-to-screenshot.md), and [url-to-markdown](/docs/endpoints/web-pages/url-to-markdown.md) endpoints are unchanged.

## V2 Competitive Hardening

- Published: 2026-07-24
- Tags: improved, security
- Permalink: https://www.enconvert.com/changelog#v2-competitive-hardening

### Summary

A round of V2 improvements closing the biggest gaps against dedicated web-data APIs: durable batch perception that survives restarts, high-value search enrichment (concurrent multi-format rendering + a synthesized cited answer + structured extraction across results), prompt-only structured extraction, more capable site discovery (gzipped sitemaps, WAF-resilient fetches, a far higher crawl ceiling), and a security-hardening pass (DNS-rebind protection on the render path, an opt-in local URL threat policy, and CSP respected by default).

---

#### `/v2/perceive/batch` — durable, resumable batches + cancel

Batches are now **restart-safe**. Previously the batch envelope (the shared render options and output mode) lived only in memory, so a server restart mid-batch failed the whole batch and you had to resubmit. The envelope is now persisted, so an interrupted batch **resumes automatically** and re-renders only the URLs that had not finished — already-completed URLs keep their artifacts.

- New: `DELETE /v2/perceive/batch/{job_id}` — cancel a running batch. The worker stops between URLs; already-completed URLs remain available. Idempotent; a terminal batch is unchanged. Project-scoped `404`.
- New batch status: `canceled` (joins `queued|processing|completed|partial|failed`).
- The batch status endpoint now reports the true `output_mode` and the `canceled` state from the durable batch record.

No request-shape change for submitting a batch — existing calls keep working.

---

#### `/v2/lookup` — high-value enrichment (`enrich`)

A new optional `enrich` object turns "search, then read the top results" from a slow, markdown-only, one-at-a-time pass into a fast, configurable one — and can synthesize a grounded answer across the results.

`enrich` fields:
- `outputs[]` — which perceive outputs to produce per enriched result (e.g. `markdown`, `html_cleaned`, `links`, `screenshot`, `structured`). Defaults to `["markdown"]`.
- `concurrency` (1–5, default 3) — how many result URLs to enrich in parallel. Markdown/HTML renders use the no-browser path and truly parallelize; screenshot/PDF renders serialize on the shared browser.
- `schema` (aliased) — run schema-driven structured extraction against each enriched result; the data appears under each result's `perceive.structured`.
- `synthesize_answer` (default `false`) — synthesize one cited, grounded answer to the query across the enriched results, returned on the response as `answer` (with `answer_sources`). Uses the perceived page content when available, otherwise result snippets.
- `answer_prompt` — an optional question to answer instead of the raw query.

New response fields: `answer`, `answer_sources[]`. When `enrich` is omitted, `perceive_top` keeps its previous behavior (markdown-only, sequential). Structured extraction and answer synthesis require an LLM-enabled plan; on plans without it they degrade to a warning.

---

#### `/v2/distill` — prompt-only extraction

You can now distill **without writing a schema**. Supply a natural-language `prompt` and the extraction schema is synthesized from it (single model), then the normal two-pass CSS→LLM engine runs.

- `schema` is now optional; provide either `schema` or `prompt` (schema wins if both are given).
- New response field `synthesized_schema` echoes the fields that were derived from the prompt.
- Prompt-only mode requires an LLM-enabled plan; without one it returns a clear warning.

---

#### `/v2/discover` — gzipped sitemaps, WAF resilience, higher ceiling

- **Gzipped sitemaps** (`sitemap.xml.gz` and any `Content-Type: application/gzip` sitemap) are now decompressed and parsed instead of silently failing.
- **WAF resilience**: sitemap and robots fetches now send realistic browser headers, so sites that 403/503 a bare client are far more likely to return their sitemap.
- **Higher crawl ceiling**: crawl-mode HTTP fetches are no longer capped at 50. The ceiling now equals your `max_urls` (up to 1000), so large sites are enumerated far more completely. Operators can lower it with `DISCOVER_CRAWL_MAX_PAGES` if needed. This also lifts the same ceiling for `/v2/ingest` crawl-mode jobs.

---

#### Security hardening

- **DNS-rebind / redirect protection on the render path**: every request the browser makes (the initial navigation, redirects, and subresources) is re-validated against the SSRF rules at request time — not only the seed URL before navigation — so a hostname that rebinds to a private address, or a redirect to an internal one, is blocked before the browser connects. The no-browser TLS engine additionally pins each connection to the validated IP.
- **Local URL threat policy** (opt-in, no external dependency): a configurable denylist of domains, TLDs, and host patterns enforced on every fetched URL, with an audit trail of blocks. Configure via `THREAT_BLOCKED_DOMAINS`, `THREAT_BLOCKED_TLDS`, or a `THREAT_POLICY_FILE`. Empty by default (blocks nothing).
- **Content-Security-Policy respected by default** for the raw browser-context path (previously bypassed). Callers that need CSP bypass for scripted DOM interaction opt in explicitly.

These changes require no API changes on your side; blocked URLs return `400`.

## Hardened URL rendering: SSRF screening, ad/media blocking, selector waits, and clearer errors

- Published: 2026-07-16
- Tags: new, security, improved
- Permalink: https://www.enconvert.com/changelog#hardened-url-rendering-ssrf-screening-ad-media-blocking-selector-waits-and-clearer-errors

### Summary

The URL rendering endpoints — [url-to-pdf](/docs/endpoints/web-pages/url-to-pdf.md), [url-to-screenshot](/docs/endpoints/web-pages/url-to-screenshot.md), [url-to-markdown](/docs/endpoints/web-pages/url-to-markdown.md), [website-to-pdf](/docs/endpoints/web-pages/website-to-pdf.md), and [website-to-screenshot](/docs/endpoints/web-pages/website-to-screenshot.md) — get new rendering controls, stronger URL-safety screening, and clearer error responses. Existing requests are unaffected: every new option defaults to its previous behavior.

#### New rendering options

- **`wait_for_selector`** (+ **`wait_for_selector_timeout`**, default `10000` ms, max `60000`): wait for a CSS selector before capture so single-page apps that hydrate after load are captured fully. Returns `422` if the element never appears.
- **`block_ads`**: abort requests to known ad and tracker domains so they never load, render, or slow the capture.
- **`block_media`**: abort image and audio/video requests entirely for a faster, lighter render (distinct from `load_media`, which only controls waiting).

#### Stronger URL safety (SSRF)

Every target URL is now screened before the browser fetches it. Requests to private, loopback, link-local, reserved, or cloud-metadata addresses — plus embedded-credential URLs, non-`http(s)` schemes, and non-standard IP notations — are rejected with `400`. This applies to single URLs, every URL in a batch, and pages discovered by the `website-to-*` crawl endpoints.

#### Credential scoping

An `Authorization` header (for example a Bearer token) passed via `headers`, or credentials from the `auth` object, are now sent **only to the target origin** — never to the third-party ad, analytics, or CDN subresources a page requests.

#### Clearer error responses

URL conversions now tell a target/input problem apart from an engine problem, using a structured `{error, code, detail}` body:

- `415 unsupported_content_type` — a non-HTML response such as JSON sent to `url-to-pdf`/`url-to-screenshot` (use `url-to-markdown` for JSON).
- `422 selector_not_found` — a `wait_for_selector` that never appeared.
- `502 upstream_unreachable` / `empty_render` — the target could not be reached, or rendered nothing.
- `504 upstream_timeout` — the target took too long.
- `503` with a `Retry-After` header when the render pool is momentarily at capacity.

A `500` now specifically means our engine faulted, rather than a catch-all.

#### Non-HTML URLs

`url-to-markdown` now returns `application/json` and `text/plain` bodies verbatim inside a fenced code block instead of running article extraction over the browser's viewer.

## LibreOffice conversion timeouts now return 504 instead of 500 or 400

- Published: 2026-07-15
- Tags: fixed, security
- Permalink: https://www.enconvert.com/changelog#libreoffice-conversion-timeouts-now-return-504-instead-of-500-or-400

A LibreOffice/unoserver conversion that exceeds its 120-second subprocess budget now returns `504`. The same timeout previously surfaced as `500` on the PDF path and `400` on the markdown path.

The 120-second budget itself is unchanged — only the status, the message, and the analytics fields changed. Nothing got faster and nothing new times out.

**Endpoints moving `500` → `504`:**

| Endpoint | Before | After |
|---|---|---|
| `POST /v1/convert/doc-to-pdf` | `500` | `504` |
| `POST /v1/convert/excel-to-pdf` | `500` | `504` |
| `POST /v1/convert/ppt-to-pdf` | `500` | `504` |
| `POST /v1/convert/odt-to-pdf` | `500` | `504` |
| `POST /v1/convert/ods-to-pdf` | `500` | `504` |
| `POST /v1/convert/odp-to-pdf` | `500` | `504` |
| `POST /v1/convert/ots-to-pdf` | `500` | `504` |
| `POST /v1/convert/pages-to-pdf` | `500` | `504` |
| `POST /v1/convert/numbers-to-pdf` | `500` | `504` |
| `POST /v1/convert/anything-to-pdf` — for `.doc .docx .xls .xlsx .ppt .pptx .odt .ods .odp .ots .pages .numbers .rtf .csv` input | `500` | `504` |

**Endpoint moving `400` → `504`:**

| Endpoint | Before | After |
|---|---|---|
| `POST /v1/convert/anything-to-markdown` — for `.doc .ppt .xls .odt .ods .odp .rtf` input only | `400` | `504` |

The two paths' format sets differ. `.docx`, `.pptx`, `.xlsx` and `.csv` go through LibreOffice on the PDF path and can time out at `504` there — on the markdown path they use pure-Python readers, never reach unoserver, and are unaffected, as are `.pdf` and `.epub`.

**Response:** `{"detail": "Document conversion timed out after 120 seconds."}` — `detail` is a plain string, identical text from both paths. Unlike the adjacent `500`, it carries no `Conversion failed: ` prefix.

**Breaking — retry semantics on `POST /v1/convert/anything-to-markdown`.** Moving `400` → `504` reclassifies these responses from client error to server error. A client that treats `4xx` as terminal and `5xx` as retryable will now retry a timed-out legacy-office markdown conversion instead of failing it outright. That is the intent — the document was valid and the request was not the caller's fault — but it is a change in your request volume against these endpoints and in what your own users see on failure. If you retry `5xx` with a backoff policy, check it before deploying against large `.doc`/`.xls`/`.odt` inputs.

**Breaking — message text.** `anything-to-markdown` previously returned `"Document conversion timed out."`. The text is now `"Document conversion timed out after 120 seconds."` A client string-matching the old message breaks.

**Fixed — the previous `500` leaked server internals.** On the PDF path the subprocess timeout propagated uncaught and its string form was interpolated into the response body, so callers received the full `unoconvert` argv and both server-side temporary file paths — e.g. `Conversion failed: Command '['unoconvert', '--convert-to', 'pdf', '/tmp/tmpXXXX.docx', '/tmp/tmpXXXX.pdf']' timed out after 120 seconds`. The `504` body carries the fixed message and nothing else.

**Analytics.** Failure events for these timeouts now carry `error_type="conversion_timeout"` and `error_code=504`. They previously reported `error_type="TimeoutExpired"` / `error_code=500` on the PDF path and `error_type="value_error"` / `error_code=400` on the markdown path. Dashboards or alerts filtering on those old values will stop seeing these events.

**Scope.** v1 conversion endpoints only. `POST /v2/ingest/files` reaches the same LibreOffice path for legacy-office input but does not use the v1 error mapping — a timeout there is still recorded as a failed page on the job, not returned as a `504`. Unsupported input formats continue to return `400`.

## Filename and title normalization in JSONL deliverable metadata

- Published: 2026-07-15
- Tags: security, fixed
- Permalink: https://www.enconvert.com/changelog#filename-and-title-normalization-in-jsonl-deliverable-metadata

Uploaded filenames flowed verbatim into the `metadata.source_url` and `metadata.title` fields of the JSONL deliverable produced by `POST /v2/ingest/files`. The same path also covered the URL side of ingest, where `metadata.title` is the `<title>` lifted from an arbitrary remote page.

**This was not a JSON injection.** JSON encoding already neutralizes quotes, newlines and C0 controls inside the file — the deliverable always parsed. The problems were what a downstream consumer does with the decoded value:

- **Line framing.** `U+2028` and `U+2029` survive non-ASCII JSON encoding and are emitted raw, and Python's `str.splitlines()` treats them as line breaks — a label containing one splits a record for any line-oriented reader.
- **Display spoofing.** Bidi override characters let `invoice<U+202E>fdp.exe` render as `invoice.pdf`.
- **Path prefixes.** `../../../../etc/passwd.csv` reached the label intact, carrying a traversal prefix into the deliverable.
- **Index bloat.** The label had no length cap and is duplicated into every chunk record of the page.

**Both labels are now normalized at emit time.** Stripped: bidi formatting controls (`U+200E`, `U+200F`, `U+202A`–`U+202E`, `U+2066`–`U+2069`), line/paragraph separators (`U+2028`, `U+2029`), and all Unicode category `Cc` — which covers C0 controls, `DEL`, C1 controls and `U+0085` — except `\t`. Whitespace runs collapse to a single space; the value is NFC-normalized first, so a decomposed name like `café.pdf` now emits as the composed `café.pdf`.

**Kept deliberately.** Spaces and punctuation survive — `Q3 Report (final) v2.pdf` is emitted unchanged. `U+200C` (ZWNJ) and `U+200D` (ZWJ) are **not** stripped: the filter targets `Cc` and the specific bidi/separator codepoints, not all of category `Cf`, because ZWNJ/ZWJ are required in legitimate Devanagari/Indic and emoji filenames. Clean URLs pass through untouched.

**Caps.** `metadata.title` is capped at 512 characters. `metadata.source_url` is capped at 2048 — generous enough that a real URL is never truncated, since truncating `source_url` destroys URL-page provenance. For file pages the label is built at the 512 cap, so a file page's `source_url` is effectively 512-capped.

**Path stripping is file-only.** An uploaded filename is reduced to its basename across both `/` and `\` before normalization, which removes traversal prefixes without losing legitimate information. URL-sourced `source_url` is **not** basenamed — a URL keeps its full path.

**Two other file-label changes.** A file page whose filename is empty — or which normalizes to empty, such as a bidi-only name or a trailing-separator path like `a/b/` — previously emitted `metadata.title: ""` and fell back to the internal storage object key for `metadata.source_url`, leaking an internal key into the deliverable. Both fields now emit `upload` in that case.

**New 400 conditions on `POST /v2/ingest/files`:**

- Filename longer than 255 characters → `400` `{"detail": "Filename exceeds the 255-character limit."}`. The filename is deliberately not echoed back.
- Filename containing a null byte → `400` `{"detail": "Filename contains an invalid null byte."}`. **Previously an opaque 500** — Postgres `TEXT` cannot hold `NUL` and the insert failed deep in the write path.

Both checks run before extension parsing. The pre-existing content-type and unsupported-extension 400s are unchanged and still echo the filename.

**Scope and limitations — read these:**

- **A file page's `source_url` is a display label, not an identifier.** It is not unique across files, and basenaming means two uploads sharing a basename — including two different paths — now emit an identical `source_url`. Do not key deduplication or provenance on it for file pages.
- **Normalization is emit-time only.** The uploaded filename is still stored raw in the ingest page record for audit. Any consumer of that value — admin UI, support tooling, log readers — is unprotected and must escape for itself.
- **Chunk `content` is not sanitized.** Only `metadata.source_url` and `metadata.title` pass through the normalizer. The document body is emitted as extracted.
- **Filename gates are `POST /v2/ingest/files` only.** `POST /v1/convert/anything-to-markdown` and the other v1 convert endpoints gain no length or null-byte gate.
- The storage object key is derived by a separate, more aggressive filename sanitizer and is unaffected by this change.

## Chunk ids for uploaded files on POST /v2/ingest/files no longer collide for same-named files

- Published: 2026-07-15
- Tags: fixed
- Permalink: https://www.enconvert.com/changelog#chunk-ids-for-uploaded-files-on-post-v2-ingest-files-no-longer-collide-for-same-named-files

Chunk ids in the assembled JSONL from `POST /v2/ingest/files` were derived from the uploaded filename. Two files sharing a filename in one job produced byte-identical ids across both pages' chunks — a downstream upsert keyed on `id` silently overwrote one document with the other, and no error was raised. Ids are now seeded from the page's storage object key, which is unique per file per job.

The id format is unchanged: 12 lowercase hex characters, a hyphen, a 4-digit zero-padded chunk index. Only the seed changed.

**Breaking: every file-page chunk id changes.** The seed moves from the filename (`report.pdf`) to the storage object key (`prod/files/42/v2-ingest-uploads/ing_abc_1_report.pdf`) — a different input to the same hash, so a completely different id. There is no overlap with the old ids and no migration path. Consequences:

- Re-ingesting a file corpus yields **new ids**. An upsert keyed on `id` will **insert duplicates rather than replace**. Delete the old vectors first, or re-index.
- The object key embeds the job id, so **the same file uploaded in a later job gets different ids**. File-page ids are stable within one job, not across jobs.
- A file-page id is **not reproducible from the deliverable**. The seed never appears in the JSONL (`metadata.source_url` carries the filename label), and the source object is deleted at assembly.
- A file-mode job that is **in flight across the deploy** assembles a mixed-id JSONL — pages completed before the deploy keep filename-seeded ids, pages completed after get key-seeded ids. Per-page JSONL is concatenated verbatim at assembly; nothing rewrites it. Drain in-flight file-mode jobs before deploying.

**URL-mode ids are byte-identical to before.** For URL pages the seed and the previous derivation are the same value, and the seed is deliberately hashed raw. A re-ingested URL corpus produces the same ids and existing vector-store rows upsert in place — no re-index. This covers ids only: the metadata-label sanitization shipping alongside it can still change `metadata.title` on URL pages, since that title is lifted from the remote page's HTML.

Determinism is preserved for both modes: the seed is a stored column, so a resumed or re-run page within a job re-derives the same id.

## POST /v2/ingest/files orphaned staged uploads on rejection and cancel

- Published: 2026-07-15
- Tags: fixed
- Permalink: https://www.enconvert.com/changelog#post-v2-ingest-files-orphaned-staged-uploads-on-rejection-and-cancel

`POST /v2/ingest/files` staged uploads one at a time, interleaved with per-file validation — file N was written to object storage before file N+1 was gated. A gate failure on any file after the first rejected the whole request, but the files already staged stayed in storage: the request never created a job row, so nothing referenced those objects and nothing removed them. Separately, a job canceled before assembly returned without deleting its staged sources — only the assembled-successfully path cleaned up.

**Staging is now two passes.** Every file in the batch is validated first; only then is any object written. A rejected batch stages nothing, so there is no orphan window to clean up. Peak memory is unchanged — one file's bytes at a time in both passes.

**Every staged object is registered for a timed cleanup sweep** at stage time, with a TTL of `24` hours. This is a backstop, not the prompt path: it covers a worker crash and any terminal path that never reaches the eager delete. Registration is idempotent and never fails a submit. If staging itself faults partway (a storage or database error), the partial batch is discarded on a best-effort basis immediately rather than waiting on the sweep; a discard failure is logged and never masks the underlying error.

**Source cleanup now runs on every terminal path** — assembled, all pages failed or skipped, all completed pages lost before assembly, canceled during the page loop, and canceled at assembly. Only the assembled path did this before. Each delete is now individually guarded: previously the cleanup loop was unguarded, so a storage error on page 3 of 10 propagated out and left pages 4–10 orphaned. A cleanup failure is logged and never sinks an otherwise-finished job. Cleanup is gated on source type — URL-mode pages are never touched.

**Object-key collision on path-like filenames.** The per-file `{job}_{index}_` uniqueness prefix was applied before filename sanitization, and sanitization takes the basename — so for a filename containing a path separator (`../../etc/passwd.csv`) the prefix was stripped off again and two such uploads in one job collided onto a single key, one silently overwriting the other. The filename is sanitized before the prefix is applied. For filenames without a separator the resulting key is byte-identical to before.

**Two new filename gates, both 400.** A filename longer than `255` characters is now rejected with `Filename exceeds the 255-character limit.` — the filename is deliberately not echoed back. A filename containing a null byte is rejected with `Filename contains an invalid null byte.`; this previously surfaced as an opaque 500, so it is a **500 → 400 status change on an existing condition**. Both are checked before the extension and content-type gates, which are unchanged and still echo the filename.

**Residual — a job canceled in the queued window relies on the backstop.** Eager cleanup runs where cancellation is observed: inside the page loop and at assembly. A job canceled before the worker begins that loop, or one whose job row is already gone when cleanup would run, has no eager path — its staged sources are removed by the 24-hour sweep, not immediately.

**Constraints:** scoped to `POST /v2/ingest/files`. No request or response fields changed and no schema change — the backstop writes rows through the existing retention path. Status codes are unchanged for every existing condition except the null-byte filename noted above.

## Upload size limit on anything-to-pdf enforced against received bytes, not Content-Length

- Published: 2026-07-15
- Tags: security, fixed
- Permalink: https://www.enconvert.com/changelog#upload-size-limit-on-anything-to-pdf-enforced-against-received-bytes-not-content-length

`POST /v1/convert/anything-to-pdf` enforced the upload size ceiling by reading the `content-length` request header. When the header was absent — a chunked or HTTP/2 client — the check was skipped in its entirety, not merely relaxed: an over-ceiling file was accepted and converted. The ceiling is now measured against the bytes actually received.

**Scope:** this changed `POST /v1/convert/anything-to-pdf` and nothing else. Every other upload route that calls the shared size check still measures `content-length` and still performs no size check at all when the header is absent. The defect class is fixed at one endpoint, not across the API.

**What the 413 does and does not do:** the multipart body is fully parsed — and spooled, to disk above 1 MB — before the handler runs, so the size check happens after receipt. It prevents the oversized file being loaded and converted; it does not abort the upload. An oversized body still crosses the wire and still occupies the spool. Do not treat the 413 as a bandwidth or ingress control.

**Behavior changes on `anything-to-pdf`:**

- Chunked/HTTP2 request with no `content-length`, file over the ceiling — was `200`, now `413`.
- Request with `content-length` where the file is under the ceiling but the multipart envelope is over it — was `413`, now `200`. The old check counted boundaries and other form fields against the file's budget; the new one measures the file. This is a relaxation, and near-limit uploads that previously failed will now go through.
- `detail.file_size` now reports the file's byte count rather than the envelope's.

The `413` body shape, the default ceiling applied when the subscription carries none, and the rejection analytics event are unchanged.

Separately, a non-numeric `content-length` on the fallback path previously surfaced as a `500`; it is now ignored.

## pdf_options no longer silently discarded on anything-to-pdf and the office PDF endpoints

- Published: 2026-07-15
- Tags: fixed
- Permalink: https://www.enconvert.com/changelog#pdf-options-no-longer-silently-discarded-on-anything-to-pdf-and-the-office-pdf-endpoints

`POST /v1/convert/anything-to-pdf` and the nine dedicated office conversion endpoints accepted and validated a `pdf_options` payload, then discarded everything except `grayscale` — page geometry never reached the converter. Geometry is now honoured wherever the engine can apply it, and rejected with a 400 wherever it cannot. No endpoint validates an option it then throws away.

**Geometry now applied on `anything-to-pdf`.** The geometry fields — `page_size`, `page_width`, `page_height`, `orientation`, `margins`, `scale`, `header`, `footer` — are honoured for HTML (`.html`, `.htm`, `.xhtml`), Markdown (`.md`, `.markdown`, `.mdown`, `.mkd`), plain text (`.txt`, `.text`), `.epub`, raster images (`.png`, `.jpg`, `.jpeg`, `.gif`, `.bmp`, `.tiff`, `.tif`, `.webp`, `.heic`, `.heif`), and `.svg`.

**Geometry now rejected with 400 on `anything-to-pdf`.** For office/ODF/iWork/RTF/CSV input (`.doc`, `.docx`, `.xls`, `.xlsx`, `.ppt`, `.pptx`, `.odt`, `.ods`, `.odp`, `.ots`, `.pages`, `.numbers`, `.rtf`, `.csv`) and `.pdf` passthrough, page layout comes from the source document. Previously these options were silently dropped and the request returned 200 with the source document's layout. An explicitly-set geometry option now returns 400 before any conversion runs:

```
pdf_options ['page_size', 'scale'] cannot be applied to '.docx' input: page geometry for this format is determined by the source document (LibreOffice), not by the converter. Remove these options (only 'grayscale' is supported for '.docx'), or set the page layout in the source file before uploading.
```

`detail` is a plain string. The engine clause reads `LibreOffice` for office input and `PDF passthrough` for `.pdf`.

**Constraints:** the trigger is explicitness, not value. A field counts as set if it appears in the request body at all — sending `{"page_size": "A4"}` (the default value) on a `.docx` returns 400, while `{"grayscale": true}` returns 200. Unsupported extensions still fail as unsupported formats, unchanged.

**`grayscale` is unaffected on every input family**, including office and `.pdf`. It was never broken and is applied as a post-process regardless of input format.

---

### Behaviour change: geometry options rejected on the dedicated office endpoints

The same rejection now applies to `POST /v1/convert/doc-to-pdf`, `excel-to-pdf`, `ppt-to-pdf`, `odt-to-pdf`, `ods-to-pdf`, `odp-to-pdf`, `ots-to-pdf`, `pages-to-pdf` and `numbers-to-pdf`. All nine render through LibreOffice, which takes no page-geometry arguments — page size, orientation and margins are properties of the source document's page style, applied at layout time before PDF export. These endpoints previously accepted and validated the geometry fields, then dropped them and returned 200 with the source document's own layout.

**This is breaking for callers who send geometry to these endpoints.** Such a request returns 400 before conversion instead of 200, with the same message shape shown above. The options never had any effect, so the PDF a caller receives is unchanged for every request that does not set geometry — but a caller that has been sending `page_size` and ignoring the fact that it did nothing must now remove it.

`grayscale` is unchanged on all nine and still applies. Omitting `pdf_options` entirely is unchanged. A wrong-extension upload (for example `.html` sent to `doc-to-pdf`) still fails with the unsupported-format error first — the format check takes precedence over the options check.

---

### Behaviour change: `scale` now takes effect on `html-to-pdf` and `markdown-to-pdf`

`POST /v1/convert/html-to-pdf` and `POST /v1/convert/markdown-to-pdf` already honoured `page_size`, `orientation`, `margins`, `header` and `footer`. `scale` was parsed and range-checked, then dropped — nothing consumed it. It is now applied.

**This changes the output of existing requests.** A caller that has been sending `scale` on either endpoint has been receiving unscaled PDFs and will now receive scaled ones. Callers relying on the old no-op must remove `scale` from the request. The accepted range is unchanged: `0.1` to `2.0` inclusive, default `1.0`; `scale: 1.0` remains a no-op.

**Semantics:** page geometry is fixed, content scales. An A4 request at `scale: 2.0` returns an A4 page with content rendered at 2×, not an A2 page. This matches the `scale` semantics already in effect on the `url-to-pdf` path. The same semantics apply to `scale` on `anything-to-pdf` for every geometry-capable input family listed above.

---

### Other observable effects

- **Requesting geometry on an image or SVG changes the rendering engine.** Raster input moves from the direct image path to an HTML layout pass, with a PNG re-encode of the raster; `.svg` moves from CairoSVG to the same layout pass (still vector). Output is not byte- or pixel-comparable to a no-options run on the same file. A request with no geometry options is byte-identical to before on both paths.
- **SVG with geometry: external resources are hard-blocked.** Neither path fetches external resources, but on the geometry path an `.svg` that references one now fails with a 400 — `SVG to PDF conversion failed: External resources are not fetched: <url>`.
- **`.txt` default margin.** Plain text with no geometry option (including a `grayscale`-only request) keeps its hardcoded 2cm margin. Sending any geometry option drops that default so the supplied `margins` win.
- **`header` / `footer` content containing a double quote or backslash now renders literally** on `anything-to-pdf`, `html-to-pdf` and `markdown-to-pdf`. On `html-to-pdf` and `markdown-to-pdf` — the two endpoints that honoured `header`/`footer` before this change — such content terminated the generated CSS string early, breaking the render or injecting caller-supplied rules into the document. That is fixed. On `anything-to-pdf` the fields were never applied before, so the escaping is in place from the first request that reaches it.

No new request or response fields. `pdf_options` is unchanged as a schema.

## New endpoint: POST /v1/convert/anything-to-markdown

- Published: 2026-07-15
- Tags: new
- Permalink: https://www.enconvert.com/changelog#new-endpoint-post-v1-convert-anything-to-markdown

### Summary

`POST /v1/convert/anything-to-markdown` accepts an uploaded document and returns a single UTF-8 Markdown file, routing to a per-format extractor by the uploaded filename's extension. 22 extensions are accepted. Conversion is synchronous and in-process — no job queue, no polling requirement. This is a v1 conversion route and is not part of the `/v2` surface; it authenticates via the existing `X-API-Key` / `Authorization: Bearer` mechanism used by the other `/v1/convert/*` routes.

#### `POST /v1/convert/anything-to-markdown`

Accepted extensions, by dispatch group:

- **Native extractors** — `.pdf`, `.docx`, `.pptx`, `.xlsx`, `.csv`, `.epub`
- **Markup** — `.html`, `.htm`, `.xhtml`
- **Legacy / ODF office** — `.doc`, `.ppt`, `.xls`, `.odt`, `.ods`, `.odp`, `.rtf` — converted through headless LibreOffice to HTML first, then to Markdown. Fidelity for this group is LibreOffice's HTML export, not a native reader.
- **Plain text / Markdown** — `.txt`, `.text`, `.md`, `.markdown`, `.mdown`, `.mkd` — encoding-normalised passthrough (tries `utf-8-sig`, `utf-8`, `cp1252`, `latin-1`, then lossy `utf-8`).

**Known catalog gap — client-side filtering.** The public endpoint catalog that drives the dashboard playground's file picker currently declares only 17 of these 22. `.xhtml`, `.text`, `.markdown`, `.mdown` and `.mkd` are accepted by the API but may be filtered out by the picker's `accept=` list until the catalog is refreshed. Direct API calls with those extensions succeed today. The catalog is a strict subset of what the code accepts, so no advertised extension is rejected — the failure mode is only that five working extensions are hidden.

**Request:** `multipart/form-data`. `file` (required). `output_filename` (default `null` — falls back to the input basename). `job_id` (optional, client-supplied; echoed back in the response and readable afterwards via `GET /v1/convert/status/{job_id}` if the connection drops mid-request). `direct_download` (accepted, default `true`, but **inert on this route** — the response is always the JSON body below; the parameter exists for request-shape parity with the `*-to-pdf` routes). Unlike `html-to-pdf` / `markdown-to-pdf` / `anything-to-pdf`, this route takes no `pdf_options`.

**Response:** `200` with JSON containing a pre-signed URL — never raw bytes, never a server-minted job handle.

```json
{
  "presigned_url": "…",
  "object_key": "…",
  "filename": "report_20260715_142233123.md",
  "file_size": 18432,
  "conversion_time_seconds": 1.84,
  "job_id": null
}
```

The same values are mirrored on `X-Object-Key`, `X-File-Size`, `X-Conversion-Time` and `X-Filename` headers. The `.md` is written to object storage before the response returns; output name is `{base}_{YYYYmmdd_HHMMSSfff}.md`. `job_id` echoes the caller's value (`null` if none was sent) — it does not indicate async processing.

**Constraints:** request timeout `300` s → `504 {"error": "Request timeout"}`. The legacy/ODF/RTF conversion subprocess is capped at `120` s, and both its failure and its timeout surface as `400`, not `504`. Upload size is checked from `Content-Length` before the body is read → `413`. PDFs are capped at `2000` pages (`400` above that). ZIP-backed formats (`.docx`, `.pptx`, `.xlsx`, `.epub`) reject archives declaring more than `400` MB uncompressed or more than `10000` entries → `400`. Concurrency is bounded in-process; a full pending queue returns `503` "Server is at capacity. Please retry shortly." with `Retry-After: 10`.

**Silent truncation points** — these do not error, they quietly shorten output: PDF words beyond `20000` per page are dropped; PDF pages carrying more than `4000` vector edges skip table detection entirely, so tables on those pages are simply not extracted; table cells are truncated at `500` chars (`…` appended); table body rows are capped at `5000` (`_(table truncated to 5000 rows)_` appended); EPUB chapter text is bounded at `100` MB total, and exceeding it stops the chapter loop.

#### Fidelity

**Tables — preserved** as GFM pipe tables across PDF, `.xlsx`, `.csv`, `.pptx` and `.docx`. Cells are flattened to a single line (`\r`/`\n` → space) and `|` is escaped to `\|`. Spreadsheet and CSV values are read as strings, so leading zeros (`007`), long integers (no scientific notation) and trailing-zero decimals (`1.50`) survive verbatim.

**Headings — preserved** as real ATX, with a per-format mechanism:

- `.docx` — Word Heading 1–6 styles map to `#`–`######`.
- **PDF — inferred from font size**, not from structure. The most common rounded size is treated as body text; larger sizes are ranked and mapped largest → `#`, and the result is **capped at `###`**. A line is only promoted if it is ≤14 words, ≥3 characters and contains a letter. Prose lines beginning with `#` are escaped to `\#` so they are not misread as headings.
- `.pptx` — synthetic `## Slide N: Title` per slide, plus `### Notes` for speaker notes.
- `.xlsx` — synthetic `## SheetName` per sheet; an empty sheet emits `_(empty sheet)_`.

**Images — effectively dropped. Do not rely on them.** There is no asset extraction and nothing is uploaded. An `<img>` becomes a Markdown image *link* to its original `src`, which for file input is relative and unresolvable by the caller; an `<img>` with no `src` is removed. PDF extraction is word- and table-only, so images never appear at all. `.pptx` picture shapes are skipped. **One exception:** `.docx` embedded images arrive as base64 `data:` URIs and pass straight through as `![](data:image/png;base64,…)`. Nothing strips them — expect substantially inflated output for image-heavy Word documents.

**OCR — not performed.** Scanned or image-only PDFs raise rather than returning an empty file: `400` "No extractable text found in the PDF. Scanned or image-only PDFs require OCR, which this endpoint does not perform." Image uploads (`.png`, `.jpg`, `.jpeg`, `.gif`, `.bmp`, `.webp`, `.tiff`, `.tif`) are not in the allowlist and are rejected with `400` by the extension gate, carrying the standard invalid-format message below.

**PDF layout —** multi-column PDFs read column-interleaved; this is a known limitation of the v1 extractor. Running headers/footers repeating on ≥60% of pages (minimum 3 pages) are dropped. Words inside a detected table region are removed from the prose stream so content is not duplicated. Consecutive prose lines merge into a paragraph, breaking on a heading, a table, or a vertical gap greater than 1.7× the previous font size.

**HTML, EPUB, DOCX and legacy office — whole-document and faithful.** Only `<script>`, `<style>`, `<noscript>`, `<iframe>`, `<svg>`, `<canvas>` and `<template>` are stripped — `<nav>`, `<footer>`, `<aside>`, `<form>` and `<button>` are **kept**. `style`/`class`/`id` and all `on*` handlers are removed; XML processing instructions are removed rather than leaking as literal text.

**EPUB —** chapters in spine (reading) order. The EPUB3 nav document is skipped so the table of contents does not pollute the output. Only `.xhtml`/`.html`/`.htm` spine items are read, and XML declaring a DTD or entity is rejected outright.

**Links and code —** anchor-only (`#…`), `javascript:` and empty-`href` links are unwrapped to plain text; bare URLs become autolinks (`<url>`); relative links stay relative. Fenced code blocks carry a language hint when one can be sniffed from `class="language-*"` or `data-lang`.

**Output —** always UTF-8. Runs of three or more blank lines collapse to one, CRLF/CR normalise to LF, and the file ends with exactly one newline. No YAML frontmatter.

#### Errors

`400` — extension outside the allowlist, checked before any extractor runs: "Invalid file format '{ext}' for anything-to-markdown. Allowed: {list}". This is also what image uploads receive. Also returned for a filename with no extension; for corrupt, empty or text-free documents ("The document contains no extractable text.", "Invalid EPUB: the file is not a valid ZIP archive.", "The CSV file contains no rows.", "Archive rejected: decompressed content is too large.", "Invalid EPUB: XML declaring a DTD/entity is not allowed."); for a scanned PDF; and for legacy-office conversion failure or its 120 s timeout ("Document conversion timed out.").

`400` — content/extension mismatch. Magic bytes are checked for `.pdf`, `.docx`, `.xlsx`, `.pptx`, `.odt`, `.ods`, `.odp`, `.epub`, `.doc`, `.xls` and `.ppt`; a high-confidence mismatch returns "File content does not match the 'anything-to-markdown' input type." `.rtf`, `.csv`, `.html`, `.txt` and `.md` are not sniffed at all.

`413` — `Content-Length` above the permitted request size. `503` — capacity gate, with `Retry-After: 10`. `504` — the 300 s request timeout. `500` — anything else, as `"Conversion failed: {message}"`; extractor-library exceptions surface here rather than as a typed error.

#### Choosing between `anything-to-markdown` and `url-to-markdown`

Separate routes, separate implementations, different inputs. `POST /v1/convert/url-to-markdown` is unchanged by this release.

| | `POST /v1/convert/anything-to-markdown` | `POST /v1/convert/url-to-markdown` |
|---|---|---|
| Input | uploaded file (`multipart/form-data`) | JSON body with URL(s) — no file input |
| Rendering | no browser; per-format extractor | headless browser render, sees JS-injected content |
| Modes | synchronous only | sync, async and batch (ZIP) |
| Extra params | none | viewport, scroll, cookies, auth, headers |
| HTML handling | whole document, faithful | main-article extraction — nav/footer/aside dropped |
| Output | body Markdown, **no frontmatter** | Markdown **with YAML frontmatter** (`title`, `description`, `url`, `links`, `images`) |

The two share only their HTML-to-Markdown conversion. Uploaded files sent to `/v2/ingest` run through the same converter as this route; `/v1/convert/anything-to-markdown` is the standalone one-file-in, one-file-out form of it.

## New endpoint: POST /v1/convert/anything-to-pdf

- Published: 2026-07-15
- Tags: new
- Permalink: https://www.enconvert.com/changelog#new-endpoint-post-v1-convert-anything-to-pdf

### Summary

`POST /v1/convert/anything-to-pdf` accepts 36 input file extensions and returns a PDF. The engine is selected from the file extension — callers do not specify a source format. Synchronous `multipart/form-data`; file upload only, no URL input.

#### `POST /v1/convert/anything-to-pdf`

**Accepted extensions (36):**

| Family | Extensions | Engine |
|---|---|---|
| Office | `.doc` `.docx` `.xls` `.xlsx` `.ppt` `.pptx` | LibreOffice headless |
| OpenDocument | `.odt` `.ods` `.odp` `.ots` | LibreOffice headless |
| Apple iWork | `.pages` `.numbers` | LibreOffice headless |
| Other document | `.rtf` `.csv` | LibreOffice headless |
| Markup | `.html` `.htm` `.xhtml` | WeasyPrint |
| Markdown | `.md` `.markdown` `.mdown` `.mkd` | Markdown → HTML → WeasyPrint |
| Plain text | `.txt` `.text` | WeasyPrint — HTML-escaped, wrapped in a monospace `<pre>`, 2cm page margins |
| Ebook | `.epub` | text extraction → Markdown → WeasyPrint |
| Raster image | `.png` `.jpg` `.jpeg` `.gif` `.bmp` `.tiff` `.tif` `.webp` `.heic` `.heif` | Pillow, embedded at 100 DPI |
| Vector image | `.svg` | CairoSVG — true vector output, not rasterized |
| PDF | `.pdf` | validated passthrough |

For the LibreOffice families, conversion fidelity is bounded by LibreOffice's — relevant in particular for iWork and legacy binary Office formats.

**Request:** `file` (required). `output_filename` (default: input basename). `job_id` (default `null`, used only for timeout-recovery polling). `pdf_options` (JSON string). `direct_download` (default `true`).

`pdf_options` is parsed and validated, but **only `grayscale` has any effect on this endpoint** — it is applied to the finished PDF. `page_size`, `orientation`, `margins`, `scale`, `header`, and `footer` are accepted and validated (an invalid `page_size` returns `400`), but are never passed to the conversion engine and have no effect on the output. Do not rely on them here. `direct_download` is likewise accepted for request-shape parity with other convert routes and has no effect — the response is always JSON.

**Response:** `200` with JSON — `presigned_url`, `object_key`, `filename` (timestamped, e.g. `report_20260714_101530123.pdf`), `file_size`, `conversion_time_seconds`, `job_id`. The same values are mirrored on the `X-Object-Key`, `X-File-Size`, `X-Conversion-Time`, and `X-Filename` headers. This is a completed conversion, not a queued job.

**Constraints:**

- Routing is by extension alone. A file with no extension returns `400`. Magic bytes never select an engine — content checking is a veto that runs before dispatch and rejects only a high-confidence mismatch between a declared binary type and recognizably different bytes (`400`, `"File content does not match the 'anything-to-pdf' input type."`). It fails open: unrecognized signatures pass.
- 21 of the 36 extensions are byte-checked (`.doc` `.docx` `.epub` `.gif` `.heic` `.heif` `.jpeg` `.jpg` `.numbers` `.odp` `.ods` `.odt` `.ots` `.pages` `.pdf` `.png` `.ppt` `.pptx` `.webp` `.xls` `.xlsx`). The other 15 are not, and `.bmp`, `.tif`, and `.tiff` are binary formats that are currently **not** sniffed despite having well-known signatures. Practical consequence: a `.docx` that is really a PNG is caught by the veto (`400` content mismatch), but a `.rtf` that is really a PNG is not — it passes to the LibreOffice engine and surfaces as a `400` conversion error instead.
- Animated GIF and multi-page TIFF convert **first frame only** — a multi-frame raster always yields a one-page PDF. Transparency (RGBA/LA/palette-alpha) is flattened onto white.
- No page-count limit. Images beyond Pillow's decompression-bomb threshold return `400` (`"Image is too large to process safely."`).
- Max upload size is read from the caller's subscription and enforced against the `content-length` header, not against bytes read. A request that omits `content-length` is **not size-checked**. Oversized returns `413`.
- Concurrent conversions are capped; requests beyond the pending queue depth return `503` with `Retry-After: 10`.
- Global request timeout is 300s → `504` `{"error": "Request timeout"}`. The LibreOffice subprocess has its own 120s cap, which surfaces as **`500`, not `504`** — a slow Office/iWork/RTF/CSV conversion fails with `Conversion failed: ...` before the global timeout is reached. The WeasyPrint, Pillow, and CairoSVG paths have no per-conversion timeout and are bounded only by the 300s global.
- Corrupt or malformed input (broken image, invalid SVG, malformed EPUB, a `.pdf` that isn't a PDF, LibreOffice failure) returns `400` with the underlying reason. Unsupported extension returns `400` with the full allowed list in `detail`. Malformed `pdf_options` JSON returns `400`.

## File uploads: POST /v2/ingest/files

- Published: 2026-07-15
- Tags: new
- Permalink: https://www.enconvert.com/changelog#file-uploads-post-v2-ingest-files

`/v2/ingest` previously accepted only URLs — an explicit list, a declared sitemap, or a crawl. It now also accepts uploaded documents, via a new multipart route. File jobs reuse the URL path's job model, chunker, assembled JSONL deliverable, and signed webhook; the `mode` field is widened to `"urls" | "sitemap" | "crawl" | "files"`, and every existing `/v2/ingest` route operates on file jobs unchanged.

### `POST /v2/ingest/files`

Takes N uploaded documents, extracts text from each, chunks each with the same semantic chunker applied to crawled pages, and assembles one JSONL containing every chunk from every upload. Returns `202` with the same `ing_`-prefixed `job_id` and the same response body as the JSON path. No browser process is spun up — file jobs are CPU-bound extraction, not renders, and are not subject to browser serialization.

**Request:** `multipart/form-data` only — no JSON, no base64. Documents go in `files` (repeated part, named `files`, not `files[]`). Remaining fields are form fields, not a body model: `max_words` (default `512`), `sentence_overlap` (default `1`), `webhook_url` (optional).

Accepted extensions (allowlist on extension — the declared part `Content-Type` is not consulted): `.txt` `.text` `.md` `.markdown` `.mdown` `.mkd` `.html` `.htm` `.xhtml` `.pdf` `.docx` `.pptx` `.xlsx` `.csv` `.epub` `.doc` `.ppt` `.xls` `.odt` `.ods` `.odp` `.rtf`. The last seven are not handled by a native extractor — they are converted through an unoserver/LibreOffice subprocess first. Images are rejected at the extension gate with `400`; there is no OCR path.

No `mode` field — the route is hard-wired to `mode="files"`. None of the URL-path fields exist here (`max_pages`, `max_depth`, `same_domain_only`, `include_patterns`, `exclude_patterns`, `respect_robots`, `wait_for`, `wait_timeout_ms`). Because the route takes discrete form fields rather than a strict body model, unknown fields are silently ignored instead of returning `422` — a misspelled `max_words` falls back to the default with no error.

**Response** (`IngestJobResponse`, unchanged): `job_id`, `status`, `mode`, `pages_discovered`, `pages_processed`, `pages_failed`, `total_chunks`, `output_url` (signed URL to the assembled JSONL, present once completed), `error_message`, `webhook_url`, `webhook_delivered`, `created_at`, `completed_at`, `warnings`. For file jobs `pages_discovered` is the uploaded file count, and the job moves `queued → processing` — it never enters `discovering`, since pages exist at submit. Existing routes apply as-is: `GET /v2/ingest/{job_id}`, `DELETE /v2/ingest/{job_id}`, `POST /v2/ingest/{job_id}/retry-webhook`, `GET /v2/ingest` (summaries render `mode: "files"`), `GET /v2/ingest/webhook-secret`, `POST /v2/ingest/webhook-secret/rotate`.

**Output:** chunk records are the same shape as URL jobs — `id`, `content`, `metadata{source_url, title, headings_path, section, word_count, chunk_index}`. For file chunks, `metadata.source_url` and `metadata.title` carry the original filename rather than a URL: same key, different semantics per mode. There is no per-chunk marker distinguishing a file chunk from a URL chunk — job `mode` is the only discriminator. `id` derives from a hash of `metadata.source_url`, so two uploads sharing a filename in one request produce colliding chunk ids in the assembled JSONL; both files still process. Filenames are emitted as supplied — unsanitized in `metadata`; sanitization applies to storage keys only.

**Constraints:** `200` files per request maximum (`400` above it) — the JSON path's 1000-page ceiling does not apply here. Empty file → `400`. Per-file byte ceiling is read from the caller's account configuration; over it → `413`. No aggregate request-size cap exists — 200 files each at the per-file ceiling is accepted. `max_words` clamps to `32`–`4000` and `sentence_overlap` to `0`–`10`: out-of-range values are silently clamped, not rejected with `422` as on the JSON path (`max_words=999999` becomes `4000`). `max_words` remains a soft cap — code blocks and tables stay atomic and may exceed it.

Submit is not instant. Each file is read, magic-byte checked, and staged to storage one at a time inside the request before the `202` returns, so submit wall-clock scales with file count × size and the 300 s request window applies to the upload. Content sniffing runs after the extension gate and rejects only when the bytes resolve to a recognizably different known binary type — indeterminate bytes pass. A single unsupported, empty, oversized, or mismatched file rejects the whole request: no job is created and no file in the batch is processed, but files already staged before the failing one are left orphaned in storage — no job row exists to clean them up. Retrying a rejected batch re-uploads everything.

No conversion timeout except 120 s on the unoserver-routed formats — `.pdf`, `.docx`, `.pptx`, `.xlsx`, `.csv`, `.epub`, text, and HTML extraction is uncapped, and there is no job-level deadline. Uploaded source files are deleted once the JSONL is assembled; that cleanup runs at the tail of assembly only, so a job canceled before it reaches assembly leaves its uploads in storage. `webhook_url` is scheme-checked at submit (`http://` or `https://`, else `400`) and SSRF-screened at delivery; a file job fetches no other URL, so nothing else is screened.

Activity rows for file jobs record zero input bytes and log the same `/v2/ingest` endpoint as the JSON path, so the two are indistinguishable in activity history.

## New v2 API surface: perceive, discover, lookup, distill, ingest, and watch endpoints

- Published: 2026-07-15
- Tags: new, improved
- Permalink: https://www.enconvert.com/changelog#new-v2-api-surface-perceive-discover-lookup-distill-ingest-and-watch-endpoints

### Summary

Six new endpoint groups (20 routes total) ship under the `/v2` prefix: `perceive`, `discover`, `lookup`, `distill`, `ingest`, and `watch`. These cover headless-browser page capture, site URL enumeration, live web search, schema-driven structured extraction, async URL-to-JSONL ingestion, and scheduled page-change monitoring. All routes authenticate via the existing `X-API-Key` / `Authorization: Bearer` mechanism used elsewhere in the API. Rate limiting applies to billable `POST` routes; `GET` status-polling routes are exempt. There is no shared response envelope — each endpoint defines its own response model; check `status`/`error` fields per-endpoint rather than assuming a common shape.

---

#### `/v2/perceive`

- `POST /v2/perceive` — single-URL headless-browser render producing any combination of requested outputs in one page load: `markdown`, `markdown_fit`, `html_cleaned`, `html_raw`, `screenshot`, `screenshot_full_page`, `pdf`, `links`, `images`, `structured` (via `outputs[]`).
- `GET /v2/perceive/{operation_id}` — poll/fetch a single operation.
- `POST /v2/perceive/batch` — same flow fanned out over `urls[]` (max 1000, de-duplicated) under one shared `options` block.
- `GET /v2/perceive/batch/{job_id}` — poll aggregate batch status and per-URL results.

Request fields: `url` (max 2048 chars, `http(s)://` only), `outputs[]` (default `["markdown","structured"]`), `extract[]` (`tables`, `metadata`, `main_content`, `headings`, `structured_data` implemented; `prices`, `contacts`, `technologies`, `all` are accepted by the schema but degrade to a warning — not implemented), `extraction_schema` (aliased `schema`), `wait_for` (`css:`/`js:` expression), `wait_timeout_ms` (0–60000, default 30000), `js_code` (max 20000 chars), `viewport`, `headers`/`cookies`/`auth`, `cache_mode` (`enabled`|`bypass`|`refresh`, default `enabled`), `pdf_options`, `block_resources[]`, `respect_robots` (default `false`), `mobile` (default `false`). `proxy_url`, `geolocation`, `action_chain` are accepted by the schema but return `422` — not implemented.

Response (`PerceiveResponse`): `operation_id`, `status` (`queued|processing|completed|failed`), `url`/`url_final`, `content_hash`, `render_quality` (0.0–1.0), `cache_hit`, `outputs{}` (each a pre-signed URL artifact, default 900s expiry), `structured`, `extraction_tier` (`heuristic|css|llm`), `tokens{input,output}`, `duration_ms`, `error`, `warnings[]`. Structured extraction always runs a heuristic pass (metadata/JSON-LD/headings/tables/main content); the LLM tier fires only when a schema is supplied, the render isn't flagged bot-blocked, and heuristic fields are unfilled.

Batch specifics: `output_mode` (`manifest`|`zip`, default `manifest`) — `zip` bundles all successful artifacts into one archive, exposed via a `zip` field on `PerceiveBatchResponse`. Batches process **strictly sequentially** through a single in-process worker — no concurrency. Batches of ≤10 URLs attempt an inline response, waiting up to 240s before degrading to `202` + `job_id`; batches of >10 URLs always return `202` immediately. `PerceiveBatchResponse` fields: `job_id`, `status` (`queued|processing|completed|failed|partial`), `output_mode`, `total`/`completed`/`failed`/`pending`, `zip`, `items[]` (`PerceiveResponse[]`), `warnings[]`. The batch queue is in-memory only — a server restart mid-batch marks remaining rows failed ("interrupted by server restart"); batches must be resubmitted.

Constraints: `main_content` extraction truncated to 50,000 chars; every URL is SSRF-screened before any row is created; `respect_robots=true` adds a robots.txt check (`403` if disallowed); `pdf` output always wins hook-chain selection over screenshot when both are requested; cache keyed by a fingerprint of render-affecting fields (URL, outputs, extract list, schema, pdf_options, viewport, mobile, js_code, wait config, block_resources, headers, sorted cookies, auth), 1-hour window.

---

#### `/v2/discover`

- `POST /v2/discover` — enumerates a site's URLs without rendering (no browser process spun up). Three modes: `sitemap` (robots.txt-declared sitemaps, probed `sitemap.xml`/`<sitemapindex>` recursion, RSS/Atom feeds), `crawl` (HTTP-only breadth-first crawl harvesting `<a href>` links from raw HTML — cannot see JS-injected routes on client-rendered SPAs, by design), `hybrid` (default — union of both, deduplicated).

Request: `url` (max 2048), `mode` (default `hybrid`), `max_urls` (1–1000, default 100), `max_depth` (1–5, default 2, crawl mode only), `include_patterns[]`/`exclude_patterns[]` (regex, max 50 entries each, compiled at request-validation time — malformed pattern returns `422`), `same_domain_only` (default `true`), `respect_robots` (default `false`; filters the output list post-hoc rather than gating fetches).

Response (`DiscoverResponse`): `url`, `mode`, `total`, `urls[]`, `pages_crawled`, `truncated`, `robots_respected`, `sources{}` (raw per-source counts before dedup/filter), `warnings[]`.

Constraints: crawl mode hard-caps actual HTTP fetches at `min(max_urls, 50)` regardless of `max_urls`; seed URL and every BFS-followed link are SSRF-screened independently; synchronous request/response — no operation row, no job queue, no persisted artifact. Handler catches all internal errors and returns a generic `500` to avoid leaking library/path details.

---

#### `/v2/lookup`

- `POST /v2/lookup` — live web search proxy (Serper/Google SERP) across six categories. No caching and no retrieval against previously ingested/watched content — each call is a fresh outbound query. Optionally auto-renders the top-N result URLs through the `/v2/perceive` flow (markdown-only) in the same round trip via `perceive_top`.

Request: `query` (1–512 chars), `category` (`web|news|images|scholar|patents|maps`, default `web`), `country`/`locale`/`time_filter`/`location`, `num_results` (1–100, default 10), `page` (1–10, default 1), `autocorrect` (default `true`), `perceive_top` (0–10, default 0).

Response (`LookupResponse`): `lookup_id`, echoed `query`/`category`/filters, `total`, `results[]` (`title`, `url`, `snippet`, `position`, `source`, `date`, `image_url`, `thumbnail_url`, `extra{}`, optional `perceive`), `perceive_top`, `perceive_operation_ids[]`, `answer_box`, `knowledge_graph`, `warnings[]`.

Constraints: 15s timeout per attempt, up to 3 attempts with 0.5s/1.0s backoff on `429/500/502/503/504`; `401/403` are not retried; a shared circuit breaker gates the provider — an open breaker returns `503` immediately without attempting the call; auto-perceive runs sequentially, not in parallel, and degrades to a per-result warning rather than failing the whole search on a single render failure; result order is passed through from the provider unmodified (no re-ranking); raw provider error bodies are never surfaced to the client (mapped to generic `502`/`503`).

---

#### `/v2/distill`

- `POST /v2/distill` — schema-driven structured extraction from one or more URLs via a two-pass engine. Pass 1: a free, no-LLM CSS selector extraction against a caller-supplied `css_schema`. Pass 2: escalates only fields left empty by pass 1 to an LLM extractor (`claude-haiku-4-5`) with a reduced schema covering just those fields. Results merge and normalize to exactly the caller's `schema` (missing fields become `null`/`[]`).

Request: `urls[]` (max 50 entries, mutually exclusive with `discover_from`) or `discover_from` (`{url, mode, max_pages: 1–50}`), `schema` (aliased `extraction_schema`, required, max 200 top-level properties, JSON-Schema or flat `{field: description}` form), `css_schema` (`baseSelector`, `fields[]` — `name`/`type`(`text|attribute|html|regex|nested|list|nested_list`)/`selector`/`attribute`/`pattern`/`transform`, nested recursion depth max 5), `wait_for` (max 1024 chars), `wait_timeout_ms` (0–60000, default 30000), `headers`/`cookies`, `respect_robots` (default `false`). Regex patterns in `css_schema` are compiled at request time and rejected on a nested-quantifier ReDoS heuristic.

Response (`DistillResponse`): `operation_id`, `total`/`completed`/`failed`, `results[]` (`url`/`url_final`, `status` (`completed|failed`), `data`, `extraction_tier` (`css|llm|mixed|none`), `fields_from_css`, `fields_from_llm`, `render_quality`, `tokens{input,output}`, `error`, `warnings[]`), `warnings[]`.

Constraints: 50-URL hard ceiling per request; each URL renders sequentially through a shared browser instance (not concurrent); CSS pass is wall-clock bounded at 10s and falls through to the LLM pass with a warning on timeout; LLM pass is skipped (CSS-only fallback, no error) when the render is flagged bot-blocked or a per-request escalation limit is reached (at most one LLM call per URL); LLM request timeout is 60s; SSRF/robots protection is inherited per-URL from the underlying render step — a rejected render fails only that URL (`status: "failed"`) without aborting the rest of the batch.

---

#### `/v2/ingest`

- `POST /v2/ingest` — creates an ingestion job.
- `GET /v2/ingest` — paginated job list.
- `GET /v2/ingest/{job_id}` — job status/progress.
- `DELETE /v2/ingest/{job_id}` — cancel (idempotent).
- `POST /v2/ingest/{job_id}/retry-webhook` — manual webhook redelivery.
- `GET /v2/ingest/webhook-secret` / `POST /v2/ingest/webhook-secret/rotate` — signing-secret management.

Discovers and/or renders a URL or set of URLs (`mode`: `urls` explicit list, or `sitemap`/`crawl` — seed URL expanded via the same crawler used by `/v2/discover`, up to `max_pages`), renders each page (credential-free — no auth/cookies/custom headers accepted on any URL-mode request), converts HTML to Markdown, and splits it with a heading-aware chunker. On completion, all pages' chunks are concatenated into a single JSONL deliverable (LangChain `JSONLoader` / LlamaIndex `SimpleDirectoryReader` / vector-DB bulk-import compatible), returned as a signed download URL. Always asynchronous — `POST /v2/ingest` returns `202`.

Request (`IngestRequest`, `extra="forbid"` — unknown fields reject the request): `mode` (default `urls`), `url` (seed, required for `sitemap`/`crawl`, max 2048 chars), `urls[]` (required for `urls` mode, max 1000 entries, each max 2048 chars), `max_pages` (1–1000, default 50), `max_depth` (1–5, default 2), `same_domain_only` (default `true`), `include_patterns[]`/`exclude_patterns[]` (max 50 each, validated at request time), `respect_robots` (default `false`), `wait_for` (max 1024 chars), `wait_timeout_ms` (0–60000, default 30000), `chunk: {max_words, sentence_overlap}` (both bounded), `webhook_url` (max 2048 chars, `http(s)://` only at submit).

Response (`IngestJobResponse`): `job_id`, `status` (`queued|discovering|processing|completed|failed|canceled`), `mode`, `pages_discovered`/`pages_processed`/`pages_failed`/`total_chunks`, `output_url` (present only once `status == "completed"`), `error_message`, `webhook_url`, `webhook_delivered`, `created_at`/`completed_at`, `warnings[]`. `GET /v2/ingest` paginates via `skip`/`limit` (default 20, max 100) + `has_more`. `DELETE` cancellation is observed between page renders — the worker stops without assembling output; terminal jobs are unchanged by a repeat call.

Webhooks: fired automatically on completion if `webhook_url` is set; best-effort — delivery failure never fails the job. SSRF-screened at delivery time (not at submit time). Signed with a per-project HMAC-SHA256 secret over `"<unix_ts>.<raw body>"`, sent as `X-Enconvert-Signature: sha256=<hex hmac>` and `X-Enconvert-Timestamp: <unix seconds>` (timestamp bound into the MAC for replay protection; default consumer-side freshness tolerance 300s). Payload: `{job_id, status, output_url, pages_processed, total_chunks}`. Retries at `1.0s, 4.0s, 16.0s` backoff (4 attempts worst case), each retry re-signed with a fresh timestamp; any `2xx` counts as success. `POST /v2/ingest/{job_id}/retry-webhook` returns `400` if no webhook is configured or the target now resolves to a private address, `409` if the job isn't `completed`.

---

#### `/v2/watch`

- `POST /v2/watch` (`201`) — create a watcher.
- `GET /v2/watch` — list watchers.
- `GET /v2/watch/{watcher_id}` — get one.
- `GET /v2/watch/{watcher_id}/snapshots` — capture history.
- `PATCH /v2/watch/{watcher_id}` — update.
- `DELETE /v2/watch/{watcher_id}` — soft delete (idempotent).

Schedules recurring headless-browser captures of a URL and diffs each capture against the prior one, persisting change records and optionally delivering a signed webhook and/or email when a change is detected.

Request (`POST`, `extra="forbid"`): `url` (max 2048, `http(s)://` only), `frequency_minutes` (60–43,200, default 60), `diff_mode` (`auto|text|structured|tables|metadata`, default `auto`), `track_fields` (optional field/selector subset), `webhook_url` (max 2048, `http(s)://` only), `notify_email` (default `true`). `PATCH` rejects an all-`null` body with `422` (no silent no-op).

Response (`WatcherResponse`): `watcher_id` (`wat_<uuid4hex>`), `url`, `status` (`active|paused|deleted`), `frequency_minutes`, `diff_mode`, `track_fields`, `webhook_url`, `notify_email`, `consecutive_errors`, `checks_count`, `last_check_at`, `next_check_at`, `last_change_at`, `created_at`/`updated_at`. `GET /v2/watch` list returns a leaner `WatcherSummary` (same minus `diff_mode`/`track_fields`/`webhook_url`/`notify_email`/`updated_at`), `limit` clamped to 1–100.

Diffing: four strategies run under `diff_mode=auto` — text diffing (similarity-ratio comparison on main-content text, flagged changed below 0.98 similarity, one capped unified diff), structured-list diffing (key-matched add/remove/modify for links by `href`, JSON-LD `structured_data`, `prices`, `contacts`), table diffing (matched by heading/caption/header signature/position), and metadata diffing (key-by-key). Each `Change` carries `section`, `kind` (`added|removed|modified`), `key`, `field`, `before`/`after` — string values >2000 chars truncated; `before`/`after` are documented as untrusted page content requiring HTML-escaping by any consumer. Capped at 500 changes per diff (overflow appends a `summary` record with the true count). A render flagged blocked or scoring below the quality floor is persisted as an audit-only snapshot — no diff, no baseline eligibility, no notification. `GET /v2/watch/{watcher_id}/snapshots` returns newest-first, `limit` clamped 1–100 at the handler (200 hard cap in the store).

Webhooks: same signing scheme as `/v2/ingest` (`X-Enconvert-Signature`/`X-Enconvert-Timestamp`, HMAC-SHA256 over `"<ts>.<body>"`) — the signing secret is shared per-project across `/v2/ingest` and `/v2/watch` deliveries and is re-screened for SSRF immediately before each send (independent of the scheme-only check performed at create/PATCH time). Retries at `1.0s/4.0s/16.0s`, each re-signed with a fresh timestamp; any `2xx` is success; a dead endpoint is logged as a non-delivery, never raised. Payload: `{event: "change_detected", watcher_id, url, checked_at, similarity, change_count, changes}`.

Constraints: `frequency_minutes` floor of 60 is enforced redundantly at schema validation, flow computation, and poller-claim time; ceiling 43,200 minutes (30 days); 3 consecutive check failures auto-pauses a watcher (`status="paused"`, `next_check_at` cleared, optional owner email); the poller claims due watchers in batches of ≤50; prior-snapshot reads are scoped to a project-namespaced storage path as a cross-project isolation guard independent of write-time checks.
