pathway.com favicon

pathway.com
Powering your RAG and ETL at scale with Live Data

What is pathway.com?

Pathway is a robust and high-performance data processing framework designed to build and manage AI/ML applications utilizing live data and real-time pipelines. It facilitates seamless data ingestion from over 300 sources, complete with automatic synchronization.

Pathway supports a range of functionalities, including real-time feature generation, live vector search, and anomaly alerts. The framework allows building applications to derive accurate AI insights from terabytes of connected documents and data tables.

Features

  • High-performance input/output connectors: Kafka, S3, cloud file systems, databases
  • Predefined API connectors: 300+ data sources
  • REST API endpoint: Serving query/answer and realtime features with sub-millisecond latency
  • Python programming API: Table API
  • SQL programming API
  • Incremental stream and table operations: join, filter, group-by, reduce
  • Advanced join types: temporal joins, windows, and ranges
  • User Defined Functions: Call external libraries
  • Async data processing: Call APIs, libraries, LLM services
  • Data schema support: Python/mypy compatible typing
  • LLM extension pack: Unstructured data parsing toolkit (data source sync, parsing, extraction, indexing)
  • Advanced data indexing: Vector with HNSW, BM24, hybrid
  • Horizontal Scalability: Enterprise Feature

Use Cases

  • Real-time GPS data analytics
  • Logistics and automotive IoT analytics
  • Fraud detection
  • RAG for sales and marketing
  • Search in slide decks
  • Log monitoring
  • Social media sentiment analysis
  • High-accuracy RAG
  • Sharepoint AI search
  • Delta lake ETL
  • Monitored instances

FAQs

  • What is the contract duration for the Department Pilot plan?
    The contract duration for the Department Pilot plan can be 30, 60, or 90 days.
  • Which cloud providers are supported by Pathway RAG pipelines?
    Pathway RAG pipelines support AWS, Azure, Google Cloud, and on-premises deployments.
  • What data sources can be used in department pilot plan?
    Supported data sources include Google Drive, Microsoft SharePoint, and the Local File System.

Related Queries

Helpful for people in the following professions

Related Tools:

Blogs:

Didn't find tool you were looking for?

Be as detailed as possible for better results