Ingesting a PDF into LlamaIndex with a standard extraction library works until the documents get complex. Multi-column layouts produce interleaved text that reads across columns simultaneously rather than down each one. Tables collapse into flat sequences of cell values with no row or column context. Heading hierarchies are stripped out entirely, so SentenceSplitter cuts at token boundaries and severs sections mid-thought. The result is a node store where retrieval quality is capped by ingestion quality, and no amount of reranking or query rewriting fixes chunks that were already malformed before they were embedded.
How to Ingest a PDF into LlamaIndex with EnConvert
Ingest PDFs into LlamaIndex as clean, heading-aware chunks with metadata preserved. No custom parsing logic. Free tier, 500 ops per month.
Get API keyExample
python
import os
import json
import time
import requests
from llama_index.core import VectorStoreIndex
from llama_index.core.schema import TextNode
ENCONVERT_API_KEY = os.environ["ENCONVERT_API_KEY"]
# LlamaIndex embeds and answers with OpenAI by default, so OPENAI_API_KEY must be set too.
PDF_PATH = "/path/to/your/document.pdf"
HEADERS = {"X-API-Key": ENCONVERT_API_KEY}
# Step 1: Upload the PDF (multipart form data, file field named "files")
with open(PDF_PATH, "rb") as pdf_file:
submit = requests.post(
"https://api.enconvert.com/v2/ingest/files",
headers=HEADERS,
files={"files": pdf_file},
timeout=60,
)
submit.raise_for_status()
job_id = submit.json()["job_id"]
# Step 2: Poll until the job is completed (queued -> processing -> completed)
deadline = time.monotonic() + 300 # give up after 5 minutes
while True:
time.sleep(2)
poll = requests.get(
f"https://api.enconvert.com/v2/ingest/{job_id}",
headers=HEADERS,
timeout=30,
)
poll.raise_for_status()
job = poll.json()
if job["status"] == "completed":
break
if job["status"] in ("failed", "canceled"):
raise RuntimeError(f"Ingest {job['status']}: {job.get('error_message')}")
if time.monotonic() > deadline:
raise TimeoutError(f"Ingest job {job_id} is still {job['status']} after 5 minutes")
# Step 3: Download the JSONL, one chunk per line. The output_url is a signed URL
# (valid 15 minutes). Plain GET, no API key.
jsonl_response = requests.get(job["output_url"], timeout=60)
jsonl_response.raise_for_status()
# Step 4: Turn each chunk into a LlamaIndex node, keeping its metadata
nodes = []
for line in jsonl_response.iter_lines():
if not line:
continue
chunk = json.loads(line)
metadata = chunk["metadata"]
# headings_path is a list of headings; join it into one string
metadata["headings_path"] = " > ".join(metadata["headings_path"])
nodes.append(TextNode(id_=chunk["id"], text=chunk["content"], metadata=metadata))
# Step 5: Index the nodes directly and query them. VectorStoreIndex.from_documents()
# would re-split these chunks with SentenceSplitter and undo the heading-aware boundaries.
index = VectorStoreIndex(nodes)
query_engine = index.as_query_engine()
response = query_engine.query("What are the key findings?")
print(response)
# Each retrieved node still carries its heading path from the PDF
for source in response.source_nodes:
print(source.node.metadata["headings_path"], source.score)
What you get back
Naive PDF extraction gives LlamaIndex one undifferentiated text blob per document — tables flattened into word salad, headings lost, no metadata to anchor retrieval — versus EnConvert returning heading-aware chunks that already carry their full breadcrumb path, section name, and render quality score so every node arrives with the structural context retrieval actually needs.