--- title: "RAG Ingestion API (Ingest): Sites and Files to JSONL" description: "POST /v2/ingest crawls a site or takes your files, renders each page in headless Chrome, chunks the Markdown by heading and returns one JSONL for LangChain, LlamaIndex or a vector database." canonical: "https://www.enconvert.com/de/rag-ingestion-api" locale: "de" --- # Ingest, the RAG ingestion API Crawl a site or upload files, get heading-aware chunks ready to embed. [Get a free API key](/de/auth?mode=signup)[Read the docs](/docs/endpoints/ingest.md) Free while in beta. No card. 500 ops a month. `POST /v2/ingest` Copy ``` curl -X POST https://api.enconvert.com/v2/ingest \ -H "X-API-Key: sk_your_private_key" \ -H "Content-Type: application/json" \ -d '{ "mode": "urls", "urls": [ "https://www.enconvert.com/blog/url-to-markdown-api", "https://www.enconvert.com/blog/rag-web-ingestion-api" ] }' ``` `GET /v2/ingest/{job_id}` Trimmed ``` { "job_id": "ing_d3d7d4feedc045e38ac34adfad3e4610", "status": "completed", "pages_processed": 2, "total_chunks": 16 } ``` `output_url` ``` {"id": "15be36a18e37-0001", "content": "The first thing I tested was turning the content filter up. […]", "metadata": {"headings_path": ["Turning the filter up made it worse"], […]}} ``` ## A job in, one JSONL out Each line is a chunk with its heading path and its page’s score. The job record the POST answers with ``` { "job_id": "ing_d3d7d4feedc045e38ac34adfad3e4610", "status": "queued", "mode": "urls", "pages_discovered": 0, "pages_processed": 0, "pages_failed": 0, "total_chunks": 0, "webhook_delivered": false, "created_at": "2026-11-11T14:35:56.501203Z" } ``` One line of the finished JSONL ``` { "id": "15be36a18e37-0001", "content": "The first thing I tested was turning the content filter up. I ran it on seven of the captured pages […]", "metadata": { "source_url": "https://www.enconvert.com/blog/url-to-markdown-api", "title": "URL to Markdown API: 52.6% of our markdown lines […]", "headings_path": ["Turning the filter up made it worse"], "section": "Turning the filter up made it worse", "word_count": 338, "chunk_index": 1, "render_quality": 1.0, "deductions": {} } } ``` Real output, captured 2026-11-11. \[…\] marks a cut. ## What you send and get | Field | What it does | | --- | --- | | `mode` | An explicit list of up to 1,000 URLs, or sitemap or crawl to discover the site first. | | `chunk.max_words` | A soft cap per chunk, 512 by default. Chunks split on headings, and code blocks and tables are never split. | | `webhook_url` | One HMAC-signed callback when the job finishes, with up to three retries. | | `output_url` | A signed link to the single JSONL file, present once the status is completed. | | `render_quality` | A page that scores under 0.40 or comes back blocked is skipped: not chunked and not billed. | ## What one job replaces - ### A crawler Sitemap parsing and link crawling find the pages, up to the cap you set. - ### A chunker Splits follow the headings, and every chunk keeps its full heading path. - ### A junk filter Challenge pages and login walls never land in your vector store. ![Left: an untidy heap of loose papers, folders and books. Right: a card-catalog drawer of even index cards, one with a green tab.](/landing/ingest-720.webp) ## From a site to your vector store 1. ### Get an API key [`X-API-Key: sk_your_private_key`](/de/auth?mode=signup) 2. ### Start a job [`POST /v2/ingest`](/docs/endpoints/ingest.md) 3. ### Load the JSONL [`JSONLoader(content_key="content", json_lines=True)`](/docs/endpoints/ingest.md#the-jsonl-record-shape) ## More of the same API - [URL to Markdown API](/de/url-to-markdown-api.md) Any URL as clean Markdown, with a score. `Perceive POST /v2/perceive` - [Web Scraping API](/de/web-scraping-api.md) One render, every output, each read scored. `Perceive POST /v2/perceive` - [Structured Data Extraction API](/de/structured-data-extraction-api.md) Your schema in, exactly that JSON out. `Distill POST /v2/distill` - [Web Search API](/de/web-search-api.md) A query in, ranked results and their pages out. `Lookup POST /v2/lookup` - [Website Change Monitoring API](/de/website-change-monitoring-api.md) Watch a URL and get a webhook when it changes. `Watch POST /v2/watch` ## Questions about RAG ingestion ### How do I crawl a website for RAG with an API? Send POST /v2/ingest with mode "crawl" and a seed url. The call answers 202 with a job\_id. The worker discovers pages, renders each in headless Chrome, chunks the Markdown by heading and assembles one JSONL file you download from the signed output\_url. ### Can I ingest files such as PDFs and Word documents? Yes. Send POST /v2/ingest/files as multipart/form-data with up to 200 files. Each document is converted to Markdown, chunked by heading and assembled into the same single JSONL as a crawl job. PDF, DOCX, PPTX, XLSX, CSV, HTML, EPUB, OpenDocument and plain-text files are supported. ### Does the JSONL load into LangChain and LlamaIndex directly? Yes. Each line carries a top-level content string with a sibling metadata object, so the file loads through LangChain JSONLoader(content\_key="content", json\_lines=True), LlamaIndex SimpleDirectoryReader and any line-oriented vector database import without reshaping. ### How does the chunker split a page? It splits on #, ## and ### headings with a soft max\_words cap (512 by default, from 32 to 4,000) and an optional sentence overlap. Fenced code blocks and Markdown tables are never split, and every chunk carries its full headings\_path. ### How do I know when a job has finished? Poll GET /v2/ingest/{job\_id}, which consumes no ops, or set webhook\_url on the POST and EnConvert sends one HMAC-signed callback, with up to three retries after the first attempt. ### What does it cost? One op per page that completes. A skipped page is not billed. The free Founding plan includes 500 ops a month with no credit card, and paid plans start at $29 a month. ## Turn a site into chunks [Get a free API key](/de/auth?mode=signup)[Read the docs](/docs/endpoints/ingest.md)