---
title: "Ingest Markdown Files into LlamaIndex with EnConvert"
description: "Use EnConvert to ingest Markdown files into LlamaIndex as heading-aware chunks with a 512 word soft cap and section metadata, ready to load as filterable nodes"
canonical: "https://www.enconvert.com/solutions/ingest-markdown-files-into-llamaindex-with-enconvert"
locale: "en"
---

# How to Ingest Markdown Files into LlamaIndex with EnConvert

A developer building a LlamaIndex agent or RAG pipeline that reads Markdown files needs heading-aware chunks where each section is a discrete retrieval unit, not arbitrary splits that break across heading boundaries. LlamaIndex’s default path loads each file with SimpleDirectoryReader and splits it by size with the sentence splitter, which cuts across headings and attaches no section context. MarkdownNodeParser solves part of that by splitting at headers and recording a header_path, but it applies no size limit to long sections and covers Markdown only, so a knowledge base that also holds PDFs, Word files, or web pages needs a different path for each format. EnConvert’s Ingest endpoint takes the uploaded Markdown file and returns JSONL chunks split at H1 to H3 heading boundaries, each carrying its headings_path breadcrumb and section name in metadata. The same schema applies to PDFs, DOCX, and HTML, so a knowledge base with mixed file types produces one consistent chunk format.

## Example

```python
import json
import os
import time

import requests
from llama_index.core import VectorStoreIndex
from llama_index.core.schema import TextNode
from llama_index.core.vector_stores import (
    FilterOperator,
    MetadataFilter,
    MetadataFilters,
)
from llama_index.embeddings.openai import OpenAIEmbedding

ENCONVERT_API_KEY = os.environ["ENCONVERT_API_KEY"]
OPENAI_API_KEY = os.environ["OPENAI_API_KEY"]
MARKDOWN_PATH = "/path/to/your/engineering-runbook.md"
TARGET_SECTION = "Deployment Checklist"  # replace with a heading from your own file

# Step 1: Upload the Markdown file to EnConvert Ingest (field name is "files")
with open(MARKDOWN_PATH, "rb") as f:
    upload_response = requests.post(
        "https://api.enconvert.com/v2/ingest/files",
        headers={"X-API-Key": ENCONVERT_API_KEY},
        files={"files": (os.path.basename(MARKDOWN_PATH), f, "text/markdown")},
        data={"max_words": 512},
        timeout=120,
    )
upload_response.raise_for_status()
job_id = upload_response.json()["job_id"]

# Step 2: Poll until the job completes
deadline = time.monotonic() + 900
while True:
    status_response = requests.get(
        f"https://api.enconvert.com/v2/ingest/{job_id}",
        headers={"X-API-Key": ENCONVERT_API_KEY},
        timeout=30,
    )
    status_response.raise_for_status()
    job = status_response.json()
    if job["status"] == "completed":
        break
    if job["status"] in ("failed", "canceled"):
        raise RuntimeError(
            f"Ingest job {job_id} {job['status']}: {job.get('error_message')}"
        )
    if time.monotonic() >= deadline:
        raise TimeoutError(f"Ingest job {job_id} is still running; poll again later")
    time.sleep(5)

# Step 3: Download the JSONL output. The signed URL is valid for 15 minutes
# and is fetched with a plain GET, no API key.
jsonl_response = requests.get(job["output_url"], timeout=120)
jsonl_response.raise_for_status()

# Step 4: Turn each JSONL chunk into a TextNode. Nodes go straight into the index,
# so LlamaIndex does not re-split the heading-aware chunks.
nodes = []
for line in jsonl_response.content.decode("utf-8").splitlines():
    if not line.strip():
        continue
    chunk = json.loads(line)
    metadata = chunk["metadata"]
    # headings_path is a list; join it into a breadcrumb string so every
    # metadata value is a scalar
    metadata["headings_path"] = " > ".join(metadata["headings_path"])
    nodes.append(
        TextNode(
            id_=chunk["id"],
            text=chunk["content"],
            metadata=metadata,
            excluded_embed_metadata_keys=[
                "chunk_index",
                "word_count",
                "source_url",
                "headings_path",
            ],
        )
    )

if not nodes:
    raise RuntimeError("The Markdown file produced no chunks")

# Fail early if the section to filter on does not exist in the returned chunks
sections = sorted(
    {node.metadata["section"] for node in nodes if node.metadata["section"]}
)
if TARGET_SECTION not in sections:
    raise ValueError(
        f"Section {TARGET_SECTION!r} not found. Sections in this file: {sections}"
    )

# Step 5: Build the vector index
index = VectorStoreIndex(
    nodes=nodes,
    embed_model=OpenAIEmbedding(
        model="text-embedding-3-small", api_key=OPENAI_API_KEY
    ),
)

# Step 6: Retrieve restricted to one section with a metadata filter
filters = MetadataFilters(
    filters=[
        MetadataFilter(key="section", value=TARGET_SECTION, operator=FilterOperator.EQ)
    ]
)
retriever = index.as_retriever(similarity_top_k=4, filters=filters)

results = retriever.retrieve("What must be completed before a production deployment?")
for result in results:
    print(result.node.metadata["section"])
    print(result.node.metadata["headings_path"])
    print(result.node.get_content()[:200])
```

## Output

Ingest returns the Markdown file as JSONL chunks split at H1 to H3 heading boundaries, with tables and code blocks kept whole and a 512 word soft cap on prose sections, each carrying its headings_path breadcrumb, section name, and chunk index, so LlamaIndex retrieval can filter by section. LlamaIndex’s default path of SimpleDirectoryReader plus the sentence splitter cuts by size across headings, while MarkdownNodeParser splits at headers but applies no size limit and handles Markdown only.

## Get started

[Get free API key — 500 ops per month, no card required.](https://www.enconvert.com/auth?mode=signup)

## Related solutions

- [How to Ingest Web Pages into Vertex AI with EnConvert](https://www.enconvert.com/solutions/ingest-web-pages-into-vertex-ai-with-enconvert.md)
- [How to Ingest Markdown into OpenAI File Search with EnConvert](https://www.enconvert.com/solutions/ingest-markdown-into-openai-file-search-enconvert.md)
- [How to Ingest HTML Files into LlamaIndex with EnConvert](https://www.enconvert.com/solutions/ingest-html-files-into-llamaindex-with-enconvert.md)
- [Best Browserless Alternative for Document Parsing](https://www.enconvert.com/solutions/browserless-alternative-for-document-parsing-enconvert.md)
- [How to Ingest DOCX into CrewAI with EnConvert](https://www.enconvert.com/solutions/ingest-docx-into-crewai-with-enconvert.md)
- [How to Ingest a Research Paper into LlamaIndex with EnConvert](https://www.enconvert.com/solutions/ingest-a-research-paper-into-llamaindex-enconvert.md)