💎 1-CALL WEB-TO-VECTOR INGESTION

تبدیل مستقیم صفحات وب به وکتور امبدینگ (Vector Embeddings)

دیگر نیازی به نوشتن پایپ‌لاین‌های پیچیده ۵ مرحله‌ای (اسکرپ، حذف کدهای اضافه، بخش‌بندی متنی، فراخوانی API پرهزینه OpenAI و ایندکس در دیتابیس) ندارید. FlyCrawl هر آدرس اینترنتی را گرفته، تمیز می‌کند و بردارهای ۱۵۳۶ و ۳۰۷۲ بعدی استاندارد تحویل می‌دهد.

شروع تولید امبدینگ رایگان اتصال به پایگاه‌های برداری
✓ Compatible with OpenAI text-embedding-3
✓ Zero External Embedding Keys Required
✓ Pinecone, Qdrant, Chroma, Milvus Ready
ARCHITECTURE COMPARISON

روش سنتی در برابر خط لوله یکپارچه FlyCrawl

❌ Fragile & Expensive

Traditional RAG Ingestion

  • 1. Scraper tool: Launch Playwright/Puppeteer (crashes under heavy RAM usage).
  • 2. Regex/HTML cleanup: Spend days writing custom parsers to remove cookie banners and navbars.
  • 3. Text splitters: Blindly slice characters resulting in broken sentences and ruined table context.
  • 4. Embedding API ($$): Pay high OpenAI token costs because HTML noise was included in the prompt.
  • 5. Vector DB Upsert: Write custom boilerplate to format payloads.
✔ FlyCrawl Unified

FlyCrawl 1-Call Vector API

  • Single API Endpoint: Send target URL, receive pre-calculated vector embeddings.
  • 90% Less HTML Noise: Fit-Markdown isolates the core article and preserves Markdown tables.
  • Semantic Chunks: Natural chunk boundaries by headings and semantic paragraphs.
  • Pre-Calculated Vectors: 1536-dim or 3072-dim floats ready for immediate DB insertion.
  • Structured YAML Metadata: Document URL, title, word count, and timestamp embedded automatically.
1-CALL API INTEGRATION

دریافت وکتور امبدینگ فقط با یک درخواست REST

Request "formats": ["markdown", "chunks", "embeddings"] to retrieve clean markdown, segmented chunks, and high-density vector embeddings in sub-second latency.

Pinecone Upsert: index.upsert(vectors=result["pinecone_records"])
Qdrant Points: client.upsert(collection_name="web", points=result["qdrant_points"])
Read Vector API Reference →
curl_vector_extraction.sh cURL / JSON-RPC 
curl -X POST https://api.flycrawl.net/api/v1/scrape \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://docs.pinecone.io/guides/get-started",
    "formats": ["markdown", "chunks", "embeddings"],
    "embeddings": {
      "dimensions": 3072,
      "model": "native-large"
    }
  }'

Direct Compatibility with Modern Vector Databases & Frameworks

🌲 Pinecone ⚡ Qdrant 🟡 ChromaDB 🐘 pgvector 🌐 Weaviate 🚀 Milvus 🦜 LangChain 🦙 LlamaIndex