Ingest, the RAG ingestion API
Crawl a site or upload files, get heading-aware chunks ready to embed.
Free while in beta. No card. 500 ops a month.
POST /v2/ingestcurl -X POST https://api.enconvert.com/v2/ingest \
-H "X-API-Key: sk_your_private_key" \
-H "Content-Type: application/json" \
-d '{
"mode": "urls",
"urls": [
"https://www.enconvert.com/blog/url-to-markdown-api",
"https://www.enconvert.com/blog/rag-web-ingestion-api"
]
}'{
"job_id": "ing_d3d7d4feedc045e38ac34adfad3e4610",
"status": "completed",
"pages_processed": 2,
"total_chunks": 16
}{"id": "15be36a18e37-0001", "content": "The first thing I tested was turning the content filter up. […]", "metadata": {"headings_path": ["Turning the filter up made it worse"], […]}}A job in, one JSONL out
Each line is a chunk with its heading path and its page’s score.
{
"job_id": "ing_d3d7d4feedc045e38ac34adfad3e4610",
"status": "queued",
"mode": "urls",
"pages_discovered": 0,
"pages_processed": 0,
"pages_failed": 0,
"total_chunks": 0,
"webhook_delivered": false,
"created_at": "2026-11-11T14:35:56.501203Z"
}{
"id": "15be36a18e37-0001",
"content": "The first thing I tested was turning the content filter up. I ran it on seven of the captured pages […]",
"metadata": {
"source_url": "https://www.enconvert.com/blog/url-to-markdown-api",
"title": "URL to Markdown API: 52.6% of our markdown lines […]",
"headings_path": ["Turning the filter up made it worse"],
"section": "Turning the filter up made it worse",
"word_count": 338,
"chunk_index": 1,
"render_quality": 1.0,
"deductions": {}
}
}Real output, captured 2026-11-11. […] marks a cut.
What you send and get
| Field | What it does |
|---|---|
mode | An explicit list of up to 1,000 URLs, or sitemap or crawl to discover the site first. |
chunk.max_words | A soft cap per chunk, 512 by default. Chunks split on headings, and code blocks and tables are never split. |
webhook_url | One HMAC-signed callback when the job finishes, with up to three retries. |
output_url | A signed link to the single JSONL file, present once the status is completed. |
render_quality | A page that scores under 0.40 or comes back blocked is skipped: not chunked and not billed. |
What one job replaces
A crawler
Sitemap parsing and link crawling find the pages, up to the cap you set.
A chunker
Splits follow the headings, and every chunk keeps its full heading path.
A junk filter
Challenge pages and login walls never land in your vector store.
From a site to your vector store
Get an API key
X-API-Key: sk_your_private_keyStart a job
POST /v2/ingestLoad the JSONL
JSONLoader(content_key="content", json_lines=True)
More of the same API
- URL to Markdown APIAny URL as clean Markdown, with a score.
Perceive POST /v2/perceive - Web Scraping APIOne render, every output, each read scored.
Perceive POST /v2/perceive - Structured Data Extraction APIYour schema in, exactly that JSON out.
Distill POST /v2/distill - Web Search APIA query in, ranked results and their pages out.
Lookup POST /v2/lookup - Website Change Monitoring APIWatch a URL and get a webhook when it changes.
Watch POST /v2/watch
Questions about RAG ingestion
How do I crawl a website for RAG with an API?
Send POST /v2/ingest with mode "crawl" and a seed url. The call answers 202 with a job_id. The worker discovers pages, renders each in headless Chrome, chunks the Markdown by heading and assembles one JSONL file you download from the signed output_url.
Can I ingest files such as PDFs and Word documents?
Yes. Send POST /v2/ingest/files as multipart/form-data with up to 200 files. Each document is converted to Markdown, chunked by heading and assembled into the same single JSONL as a crawl job. PDF, DOCX, PPTX, XLSX, CSV, HTML, EPUB, OpenDocument and plain-text files are supported.
Does the JSONL load into LangChain and LlamaIndex directly?
Yes. Each line carries a top-level content string with a sibling metadata object, so the file loads through LangChain JSONLoader(content_key="content", json_lines=True), LlamaIndex SimpleDirectoryReader and any line-oriented vector database import without reshaping.
How does the chunker split a page?
It splits on #, ## and ### headings with a soft max_words cap (512 by default, from 32 to 4,000) and an optional sentence overlap. Fenced code blocks and Markdown tables are never split, and every chunk carries its full headings_path.
How do I know when a job has finished?
Poll GET /v2/ingest/{job_id}, which consumes no ops, or set webhook_url on the POST and EnConvert sends one HMAC-signed callback, with up to three retries after the first attempt.
What does it cost?
One op per page that completes. A skipped page is not billed. The free Founding plan includes 500 ops a month with no credit card, and paid plans start at $29 a month.