If you are building a LangChain pipeline that reads Substack newsletters, you need the newsletter content, not the page infrastructure Substack wraps around it. Substack pages include subscriber counts, author bios, comment sections, recommended newsletter carousels, paywall prompts, and social sharing elements that inflate token usage and dilute the signal your chain needs. Writing your own extraction and summarisation logic works until Substack updates its page structure, at which point your pipeline returns degraded output with no error to surface the failure. EnConvert’s Distill endpoint takes a Substack URL and returns a clean, concise summary of the newsletter content, ready to pass into a LangChain chain as context or store in a memory layer.
How to Summarise Substack Newsletters with LangChain
Use EnConvert Distill to summarise Substack newsletters in one API call and pass clean, LLM-ready output directly into a LangChain pipeline or memory store.
Get API keyExample
import os
import requests
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
ENCONVERT_API_KEY = os.environ["ENCONVERT_API_KEY"]
SUBSTACK_URL = os.environ["SUBSTACK_URL"] # a public Substack post URL
# Step 1: Summarise the Substack newsletter with Enconvert Distill
response = requests.post(
"https://api.enconvert.com/v2/distill",
headers={
"X-API-Key": ENCONVERT_API_KEY,
"Content-Type": "application/json",
},
json={
"urls": [SUBSTACK_URL],
"schema": {
"summary": (
"a thorough summary of the post, including its main argument "
"and any practical recommendations it makes"
),
},
},
timeout=60,
)
response.raise_for_status()
item = response.json()["results"][0]
if item.get("error") or not item.get("data"):
raise RuntimeError(f"Distill failed for {SUBSTACK_URL}: {item.get('error')}")
summary = item["data"]["summary"]
# Step 2: Pass the summary into a LangChain chain as context
prompt = ChatPromptTemplate.from_messages([
(
"system",
"Answer using the supplied newsletter summary. "
"If the summary does not contain the answer, say so.",
),
(
"human",
"Newsletter summary:\n{summary}\n\nQuestion: {question}",
),
])
llm = ChatOpenAI(
model="gpt-4.1",
api_key=os.environ["OPENAI_API_KEY"],
)
chain = prompt | llm
answer = chain.invoke({
"summary": summary,
"question": (
"What is the newsletter's main argument, "
"and what practical recommendations does it make?"
),
})
print(answer.content)What you get back
Distill returns the Substack newsletter as a clean summary: the core content condensed into coherent paragraphs or structured points, with subscriber counts, author bios, comment sections, recommended newsletter carousels, paywall prompts, and social sharing elements removed before the response is returned. Passing raw Substack page content into a LangChain chain instead produces bloated token usage from surrounding noise, inconsistent output structure across posts from different authors who use different page layouts, and a higher probability of the chain reasoning over promotional or navigational content rather than the newsletter itself. The problem compounds when running chains across multiple newsletters where token budgets and context consistency both affect output quality.