If you are building an OpenAI Agents SDK agent that needs to read Stripe documentation, you need clean structured content from pages that were built for browsers, not for language models. Stripe’s documentation is JavaScript-rendered, which means a standard requests call returns the page shell with placeholder elements where the API reference content should be, and a generic scraper returns raw HTML cluttered with navigation menus, sidebar elements, and cookie banners that inflate token usage and obscure the content your agent needs. Maintaining custom scraping and cleanup logic adds work whenever Stripe updates its page structure. EnConvert’s Perceive endpoint runs a full browser pass, strips the surrounding noise, and returns the Stripe documentation content as clean Markdown ready to pass into an OpenAI Agents SDK agent as context or store in a retrieval layer.
How to Scrape Stripe Docs with OpenAI Agents SDK
Use EnConvert’s Perceive endpoint to scrape Stripe docs into clean Markdown for your OpenAI Agents SDK pipeline without custom parsers or preprocessing
Get API keyExample
import os
import requests
from agents import Agent, Runner
ENCONVERT_API_KEY = os.environ["ENCONVERT_API_KEY"]
# Step 1: Scrape the Stripe docs page with Enconvert Perceive
perceive_response = requests.post(
"https://api.enconvert.com/v2/perceive",
headers={
headers={
"X-API-Key": ENCONVERT_API_KEY,
"Content-Type": "application/json",
},
json={
"url": "https://docs.stripe.com/api/charges",
"outputs": ["markdown"],
},
timeout=120,
)
perceive_response.raise_for_status()
perceived = perceive_response.json()
if perceived.get("error") or "markdown" not in perceived.get("outputs", {}):
raise RuntimeError(f"Perceive failed: {perceived.get('error')}")
# The markdown is a signed artifact URL (valid 15 minutes). Plain GET, no API key.
markdown_response = requests.get(perceived["outputs"]["markdown"]["url"], timeout=60)
markdown_response.raise_for_status()
markdown = markdown_response.text
# Step 2: Pass the Markdown into an OpenAI Agents SDK agent as context
agent = Agent(
name="Stripe Docs Assistant",
instructions=f"""You answer questions about the Stripe API.
Stripe documentation:
{markdown}
Use this context to answer questions about endpoints, parameters, and error codes.""",
)
result = Runner.run_sync(agent, "What parameters are required to create a charge?")
print(result.final_output)What you get back
Perceive returns the Stripe documentation page as clean Markdown: the API reference or guide content with headings, parameters, code examples, and tables preserved, and navigation menus, cookie banners, sidebar elements, and JavaScript-rendered noise stripped before the response is returned. A standard requests call to the same URL returns the pre-render HTML shell with empty content placeholders where the documentation should be, because Stripe’s docs require JavaScript execution to populate the page body. A generic scraper that runs JavaScript still returns raw HTML with surrounding markup that requires significant preprocessing before any of it is usable as agent context.