A developer building a CrewAI agent that processes documents stored in AWS S3 needs clean structured content from each document before the agent can reason over it. Downloading the files and running a separate parser for each document type adds infrastructure that the agent itself should not manage, and maintaining separate libraries for PDF, DOCX, and other formats increases dependency overhead. EnConvert Distill accepts the S3 URL for each document, public or pre-signed, and a schema the agent defines, and returns structured fields in one synchronous call. The @tool function returns schema-defined document content ready for the agent to reason over with no download or parsing step.
How to Summarise AWS S3 Documents with CrewAI
AWS S3 document summarisation in CrewAI: EnConvert Distill returns schema-defined fields from up to 50 document URLs in one synchronous call, no file downloads
Get API keyExample
python
import os
import requests
from crewai import Agent, Crew, Process, Task
from crewai.tools import tool
# CrewAI reads OPENAI_API_KEY from the environment for the gpt-4.1 model
S3_DOCUMENT_URLS = [url.strip() for url in os.environ["S3_DOCUMENT_URLS"].split(",")] # public or pre-signed S3 URLs
BASE_URL = "https://api.enconvert.com/v2"
headers = {
"X-API-Key": os.environ["ENCONVERT_API_KEY"],
"Content-Type": "application/json",
}
DOC_SCHEMA = {
"title": "The document title",
"executive_summary": "A three sentence summary of what the document covers and concludes",
"key_findings": "The main findings or conclusions in the document, as a list",
"document_type": "The kind of document, such as report, contract, memo, or policy",
"date_published": "The publication or issue date, if the document states one",
}
@tool("summarise_s3_documents")
def summarise_s3_documents(urls: list[str]) -> str:
"""Reads documents stored in AWS S3 from public or pre-signed URLs and returns
structured fields for each: title, executive summary, key findings, type, and date."""
if len(urls) > 50:
return "Too many URLs. Distill accepts at most 50 per call."
# One synchronous call: no job_id, no polling, no download step
response = requests.post(
f"{BASE_URL}/distill",
headers=headers,
json={"urls": urls, "schema": DOC_SCHEMA},
timeout=300,
)
# Distill runs on paid plans only, and the free tier returns 402
if response.status_code == 402:
return "Distill requires a paid plan."
response.raise_for_status()
documents = []
for result in response.json()["results"]:
if result["status"] != "completed":
documents.append(f"Source: {result['url']}\nCould not be read ({result['status']}).")
continue
# A low render_quality can mean an expired link or an error page, not the document
render_quality = result.get("render_quality", 1.0)
if render_quality < 0.5:
documents.append(
f"Source: {result['url']}\nLow render quality ({render_quality}), skipped."
)
continue
data = result["data"] # keys match DOC_SCHEMA exactly
findings = data["key_findings"]
documents.append(
f"Source: {result['url']}\n"
f"Title: {data['title']}\n"
f"Type: {data['document_type']}\n"
f"Published: {data['date_published']}\n"
f"Key findings: {'; '.join(findings) if isinstance(findings, list) else findings}\n"
f"Executive summary: {data['executive_summary']}"
)
return "\n\n---\n\n".join(documents)
analyst = Agent(
role="Document Analyst",
goal="Summarise the documents you are given accurately, and cite the document URL behind every statement",
backstory="You read business documents closely and never state a finding you cannot trace to a source.",
tools=[summarise_s3_documents],
llm="gpt-4.1",
verbose=True,
)
task = Task(
description=(
"Read these documents: {document_urls}. Summarise each one, list its key findings, "
"and say which document is the most recent. Cite the URL for every statement."
),
expected_output="A short summary of each document with its key findings, and the most recent one named.",
agent=analyst,
)
crew = Crew(agents=[analyst], tasks=[task], process=Process.sequential, verbose=True)
# Pre-signed URLs expire, so generate them just before the run
result = crew.kickoff(inputs={"document_urls": ", ".join(S3_DOCUMENT_URLS)})
print(result)
What you get back
Distill returns schema-defined fields inline as structured JSON in the data object, with no polling, no job_id, and no signed URL to fetch. The render_quality score lets the agent detect an inaccessible or expired URL before reasoning over an empty result. Processing S3 documents without Distill requires downloading each file, selecting a parser by file extension, and running it, with a separate dependency for each format; Distill returns structured fields from the URL in one call regardless of document type.