🦜🔗 LangChain Connector
LangChain Web Scraping & Ingestion
Load live web pages and entire sitemaps directly into LangChain `Document` objects. Eliminate 90% of token noise before chunking and vector storage.
Python LangChain Snippet
from langchain_core.documents import Document
from flycrawl import FlyCrawl
client = FlyCrawl(api_key="YOUR_API_KEY")
res = client.scrape_url("https://example.com")
doc = Document(
page_content=res["markdown"],
metadata={"source": res["url"]}
)
# Ready for LangChain text splitters & vector stores