A developer building a DSPy program that reads Confluence documentation needs clean structured content from each page before the program can reason over it. The Confluence REST API returns page body content in Confluence Storage Format, an XML-based representation containing macro definitions and layout containers that must be stripped before the text is usable as a DSPy input. EnConvert Distill accepts the public URL for each Confluence page and a schema the developer defines, and returns structured fields in one synchronous call. The DSPy tool function returns schema-defined page content with no Confluence API token and no storage format parsing.
How to Summarise Confluence with DSPy
Summarise Confluence pages in a DSPy program using EnConvert Distill, which returns schema-defined fields with no API token or storage format parsing
Get API keyExample
python
import os
import dspy
import requests
# dspy.LM reads OPENAI_API_KEY from the environment
CONFLUENCE_PAGE_URL = os.environ["CONFLUENCE_PAGE_URL"] # a public Confluence page that opens without signing in
BASE_URL = "https://api.enconvert.com/v2"
headers = {
"X-API-Key": os.environ["ENCONVERT_API_KEY"],
"Content-Type": "application/json",
}
# Configure the language model the program uses
dspy.configure(lm=dspy.LM("openai/gpt-4.1"))
# The fields the program receives. The response keys match these names exactly.
# Each description tells the extractor to leave a field empty rather than guess.
CONFLUENCE_SCHEMA = {
"page_title": "The page title",
"summary": (
"A three sentence summary of the page, using only text that is visible on the page. "
"Return an empty string if the page has no text content."
),
"section_headings": (
"The headings of the main sections of the page, in order, as a list. "
"Return an empty list if the page has none."
),
"last_updated": (
"The date the page was last updated, only if the page shows one. "
"Return an empty string otherwise."
),
}
def as_text(value) -> str:
if isinstance(value, list):
return "; ".join(str(item) for item in value) or "not stated"
return str(value) if value else "not stated"
def summarise_confluence_pages(urls: list[str]) -> str:
"""Reads publicly accessible Confluence pages and returns structured fields for each:
title, summary, section headings, and last updated date. Accepts up to 50 URLs.
Pages that need a Confluence login cannot be read."""
# Distill accepts at most 50 URLs per call
if len(urls) > 50:
return "Too many URLs. Distill accepts at most 50 per call."
# One synchronous call: no job_id, no polling, no download step
response = requests.post(
f"{BASE_URL}/distill",
headers=headers,
json={"urls": urls, "schema": CONFLUENCE_SCHEMA},
timeout=300,
)
# Distill runs on paid plans only, and the free tier returns 402
if response.status_code == 402:
return "Distill requires a paid plan."
response.raise_for_status()
pages = []
for result in response.json()["results"]:
if result["status"] != "completed":
pages.append(f"Source: {result['url']}\nCould not be read ({result['status']}).")
continue
# A low render_quality can mean a Confluence login wall or an inaccessible page
render_quality = result.get("render_quality", 1.0)
if render_quality < 0.5:
pages.append(
f"Source: {result['url']}\nLow render quality ({render_quality}), "
"likely a login wall. Skipped."
)
continue
data = result["data"] # keys match CONFLUENCE_SCHEMA exactly
if not data.get("page_title") and not data.get("summary"):
pages.append(f"Source: {result['url']}\nNo page content was found.")
continue
pages.append(
f"Source: {result['url']}\n"
f"Title: {as_text(data['page_title'])}\n"
f"Last updated: {as_text(data['last_updated'])}\n"
f"Sections: {as_text(data['section_headings'])}\n"
f"Summary: {as_text(data['summary'])}"
)
return "\n\n---\n\n".join(pages)
class AnswerFromConfluence(dspy.Signature):
"""Answer a question about Confluence documentation using only what the tool returns.
Cite the page URL for every statement. If a field is not stated, say so and do not guess."""
question: str = dspy.InputField()
answer: str = dspy.OutputField(desc="The answer, with the page URL cited for each statement")
agent = dspy.ReAct(AnswerFromConfluence, tools=[summarise_confluence_pages], max_iters=5)
result = agent(
question=(
f"Read {CONFLUENCE_PAGE_URL} and list its section headings "
"and last updated date."
)
)
print(result.answer)
What you get back
Distill returns schema-defined fields inline as structured JSON in the data object, with no polling and no storage format parsing. The render_quality score lets the program detect an inaccessible or login-gated page before reasoning over incomplete content. Reading Confluence pages through the REST API requires authenticating with an API token and parsing the storage format body to extract text from macro tags and layout containers, while Distill returns schema-defined fields from the page URL in one synchronous call.