Skip to content

Latest commit

 

History

History
271 lines (223 loc) · 18.4 KB

README.md

File metadata and controls

271 lines (223 loc) · 18.4 KB
Raw

Apache OpenDAL™: One Layer, All Storage.

OpenDAL is an Open Data Access Layer that enables seamless interaction with diverse storage services.

OpenDAL's development is guided by its vision of One Layer, All Storage and its core principles: Open Community, Solid Foundation, Fast Access, Object Storage First, and Extensible Architecture. Read the explained vision at OpenDAL Vision.

OpenDAL Architectural

For ANY languages

Name Release Docs
Rust Core Rust Core Image Docs Release Docs Dev
C Binding - Docs Dev
Cpp Binding - Docs Dev
D Binding - -
Dotnet Binding - -
Go Binding Go Binding Image Docs Release
Haskell Binding - -
Java Binding Java Binding Image Docs Release Docs Dev
Lua Binding - -
Node.js Binding Node.js Binding Image Docs Dev
Ocaml Binding - -
PHP Binding - -
Python Binding Python Binding Image Docs Dev
Ruby Binding - -
Swift Binding - -
Zig Binding - -

For ANY methods

Name Description Release
oay Access data via API Gateway oay image
oli Access data via Command Line (alternative to s3cmd, s3cli, azcopy) oli image
ofs Access data via POSIX file system API (alternative to s3fs) ofs image

For ANY integrations

Name Description Release Docs
dav-server-opendalfs a dav-server-rs implementation using opendal. dav-server image Docs Release Docs Dev
object_store_opendal an object_store implementation using opendal. object_store image Docs Release Docs Dev
fuse3_opendal Access data via integrations to fuse3 fuse3 image Docs Release Docs Dev
virtiofs_opendal Access data via integrations to vhost-user-backend virtiofs image Docs Release Docs Dev
unftp-sbe-opendal an unftp storage backend implementation using opendal. unftp-sbe image Docs Release Docs Dev
parquet_opendal Provides parquet efficient IO utilities parquet image Docs Release Docs Dev

For ANY services

Type Services
Standard Storage Protocols ftp http sftp webdav
Object Storage Services azblob cos gcs obs oss s3
b2 openstack_swift upyun vercel_blob
File Storage Services fs alluxio azdls azfile chainsafe compfs
dbfs gridfs hdfs hdfs_native ipfs webhdfs
Consumer Cloud Storage Service aliyun_drive gdrive onedrive dropbox icloud koofr
pcloud seafile yandex_disk
Key-Value Storage Services cacache cloudflare_kv dashmap memory etcd
foundationdb persy redis rocksdb sled
redb tikv atomicserver
Database Storage Services d1 mongodb mysql postgresql sqlite surrealdb
Cache Storage Services ghac memcached mini_moka moka vercel_artifacts
Git Based Storage Services huggingface

Examples

The examples are available at here.

Documentation

The documentation is available at https://opendal.apache.org.

Contribute

OpenDAL is an active open-source project. We are always open to people who want to use it or contribute to it. Here are some ways to go.

Who is using OpenDAL?

Rust Core

  • apache/iceberg-rust: Native Rust implementation of Apache Iceberg, the open table format for analytic datasets.
  • CrateDB: An open-source, distributed SQL database for Real-Time Analytics and Hybrid Search
  • Databend: A modern Elasticity and Performance cloud data warehouse.
  • deepeth/mars: The powerful analysis platform to explore and visualize data from blockchain.
  • GreptimeDB: An open-source, cloud-native, distributed time-series database.
  • mozilla/sccache: sccache is ccache with cloud storage
  • OctoBase: the open-source database behind AFFiNE, local-first, yet collaborative.
  • ParadeDB: Postgres for Search and Analytics - fast full-text search and analytics in Postgres and over cloud storage. Built as an extension.
  • Pants: A fast, scalable, user-friendly build system for codebases of all sizes.
  • QuestDB: An open-source time-series database for high throughput ingestion and fast SQL queries with operational simplicity.
  • RisingWave: A Distributed SQL Database for Stream Processing
  • Vector: A high-performance observability data pipeline.

C Binding

  • Milvus: A cloud-native vector database, storage for next generation AI applications

Java Binding

  • QuestDB: An open-source time-series database for high throughput ingestion and fast SQL queries with operational simplicity.

Branding

The first and most prominent mentions must use the full form: Apache OpenDAL™ of the name for any individual usage (webpage, handout, slides, etc.) Depending on the context and writing style, you should use the full form of the name sufficiently often to ensure that readers clearly understand the association of both the OpenDAL project and the OpenDAL software product to the ASF as the parent organization.

For more details, see the Apache Product Name Usage Guide.

License and Trademarks

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0

Apache OpenDAL, OpenDAL, and Apache are either registered trademarks or trademarks of the Apache Software Foundation.