A developer building a Pydantic AI agent that reads GitHub pages needs clean, structured content before reasoning over repositories, READMEs, or release notes. GitHub repo pages mix the content developers care about with contributor sidebars, file trees, star counts, and fork metadata, and scraping them directly requires HTML selectors that break when GitHub updates its layout. EnConvert Distill accepts the GitHub URL, applies a schema the agent defines, and returns structured fields in one synchronous call. The agent gets structured content ready for downstream reasoning with no HTML parsing.
How to Summarise GitHub with Pydantic AI
GitHub repo summarisation in Pydantic AI: use EnConvert Distill to return schema-defined fields from up to 50 URLs in one synchronous call, no HTML parsing
Get API keyExample
python
import os
import requests
from pydantic_ai import Agent, RunContext
# Pydantic AI's OpenAI provider reads OPENAI_API_KEY from the environment
BASE_URL = "https://api.enconvert.com/v2"
headers = {
"X-API-Key": os.environ["ENCONVERT_API_KEY"],
"Content-Type": "application/json",
}
# The fields the agent receives. The response keys match these names exactly.
REPO_SCHEMA = {
"description": "The one line repository description shown under the repository name",
"readme_summary": "A three sentence summary of what the README says the project does",
"latest_release": "The name and date of the latest release, if the page shows one",
"primary_language": "The main programming language of the repository",
"license": "The licence named on the page, if any",
}
agent = Agent(
"openai:gpt-4.1",
system_prompt=(
"You summarise GitHub repositories. Use summarise_github_pages to read the pages you "
"need, answer only from what it returns, and cite the page URL for every statement."
),
)
@agent.tool
def summarise_github_pages(ctx: RunContext[None], urls: list[str]) -> str:
"""Reads GitHub repository pages and returns structured fields for each page:
description, README summary, latest release, primary language, and licence."""
if len(urls) > 50:
return "Too many URLs. Distill accepts at most 50 per call."
# One synchronous call: no job_id, no polling, no download step
response = requests.post(
f"{BASE_URL}/distill",
headers=headers,
json={"urls": urls, "schema": REPO_SCHEMA},
timeout=300,
)
# Distill runs on paid plans only, and the free tier returns 402
if response.status_code == 402:
raise RuntimeError("Distill requires a paid plan. Free tier returns 402.")
response.raise_for_status()
pages = []
for result in response.json()["results"]:
# Failed pages come back as rows with a failed status, not as errors
if result["status"] != "completed":
pages.append(f"Source: {result['url']}\nCould not be read ({result['status']}).")
continue
# A render_quality below 0.5 may mean a login wall or challenge page
render_quality = result.get("render_quality", 1.0)
if render_quality < 0.5:
pages.append(
f"Source: {result['url']}\nLow render quality ({render_quality}), skipped."
)
continue
data = result["data"] # keys match REPO_SCHEMA exactly
pages.append(
f"Source: {result['url']}\n"
f"Description: {data['description']}\n"
f"Primary language: {data['primary_language']}\n"
f"Licence: {data['license']}\n"
f"Latest release: {data['latest_release']}\n"
f"README summary: {data['readme_summary']}"
)
return "\n\n---\n\n".join(pages)
result = agent.run_sync(
"Read https://github.com/pydantic/pydantic-ai and tell me what the project does, "
"which language it is written in, and what its latest release is."
)
print(result.output)
What you get back
Distill returns the schema-defined fields inline as structured JSON in the data object, with no polling, no job_id, and no signed URL to fetch. The render_quality score lets the agent detect a login wall or challenge page before reasoning over incomplete content. Scraping GitHub repo pages directly returns HTML containing contributor lists, file tree entries, and star and fork counts embedded throughout, and a dedicated parser needs CSS selectors that break when GitHub changes its layout.