v0.33.1

NOTE: This release is not yet an official ASF release. We are still in the process of learning how to create a formal one.

v0.33.1 - 2023-04-25

Added

• feat: Add behavior test for read_with_if_match & stat_with_if_match (#2088)
• feat(tests): Add fuzz test for writer without content length (#2100)
• feat: add if_none_match support for obs (#2103)
• feat(services/oss): Add server side encryption support for oss (#2092)
• feat(core): update errorKind PreconditionFailed to ConditionNotMatch (#2104)
• feat(services/s3): Add start-after support for list (#2096)
• feat: gcs support cache control (#2116)

Fixed

• fix(services/gcs): set content length=0 for gcs initiate_resumable_upload (#2110)
• fix(bindings/nodejs): Fix index.d.ts not updated (#2117)

Docs

• chore: improve LoggingLayer docs and pub use log::Level (#2089)
• docs(refactor): Add more detailed description of operator, accessor, and builder (#2094)

CI

• chore(bindings/nodejs): update package.json repository info (#2078)
• ci: Bring hdfs test back (#2114)

New Contributors

• @junaire made their first contribution in #2103

Full Changelog: v0.33.0...v0.33.1

v0.33.0

NOTE: This release is not yet an official ASF release. We are still in the process of learning how to create a formal one.

Upgrade to v0.33

Public API

OpenDAL 0.33 has redesigned the Writer API, replacing all instances of writer.append() with writer.write(). For more information, please refer to Writer.

Raw API

In addition to the redesign of the Writer API, we have removed append from oio::Write. Therefore, users who implement services and layers should also remove it.

After v0.33 landing, services should handle OpWrite::content_length correctly by following these guidelines:

• If the writer does not support uploading unsized data, return a response of NotSupported if content length is None.
• Otherwise, continue writing data until either close or abort has been called.

Furthermore, OpenDAL 0.33 introduces a new concept called Capability which replaces AccessorCapability. Services must adapt to this change.

---

v0.33.0 - 2023-04-23

Added

• feat: Add OpenTelemetry Trace Layer (#2001)
• feat: add if_none_match support for azblob (#2035)
• feat: add if_none_match/if_match for gcs (#2039)
• feat: Add size check for sized writer (#2038)
• feat(services/azblob): Add if-match support (#2037)
• feat(core): add copy&rename to error_context layer (#2040)
• feat: add if-match support for OSS (#2034)
• feat: Bootstrap new (old) project oay (#2041)
• feat(services/OSS): Add override_content_disposition support (#2043)
• feat: add IF_MATCH for http (#2044)
• feat: add IF_MATCH for http HEAD request (#2047)
• feat: add cache control header for azblob and obs (#2049)
• feat: Add capability for operation's variant and args (#2057)
• feat(azblob): Add override_content_disposition support (#2065)
• feat(core): test for read_with_override_content_composition (#2067)
• feat(core): Add start-after support for list (#2071)

Changed

• refactor: Polish Writer API by merging append and write together (#2036)
• refactor(raw/http_util): Add url in error context (#2066)
• refactor: Allow reusing the same operator to speed up tests (#2068)

Fixed

• fix(bindings/ruby): use rb_sys_env to help find ruby for building (#2051)
• fix: MadsimLayer should be able to built without cfg (#2059)
• fix(services/s3): Ignore prefix if it's empty (#2064)

Docs

• docs(bindings/python): ipynb examples for users (#2061)

CI

• ci(bindings/nodejs): publish support --provenance (#2046)
• ci: upgrade typos to 1.14.8 (#2055)
• chore(bindings/C): ignore the formatting of auto-generated opendal.h (#2056)

New Contributors

• @leenstx made their first contribution in #2033
• @Essoz made their first contribution in #2035
• @imp2002 made their first contribution in #2037
• @Renkai made their first contribution in #2034
• @Weijun-H made their first contribution in #2044
• @everpcpc made their first contribution in #2051
• @Justin-Xiang made their first contribution in #2061
• @lexcao made their first contribution in #2071

Full Changelog: v0.32.0...v0.33.0

v0.32.0

NOTE: This release is not yet an official ASF release. We are still in the process of learning how to create a formal one.

Upgrade to v0.32

OpenDAL 0.32 doesn't have much breaking changes.

We changed Accessor::create into Accessor::create_dir. Only users who implement Layer need to change.

v0.32.0 - 2023-04-18

Added

• feat: Add wasabi service implementation (#2004)
• feat: improve the readability of oli command line error output (#2016)
• feat: add If-Match Support for OpRead, OpWrite, OpStat (#2017)
• feat: add behavioral test for Write::abort (#2018)
• feat: add if-match support for obs (#2023)
• feat: Add missing functions for trace layers (#2025)
• feat(layer): add madsim layer (#2006)
• feat(gcs): add support for gcs append (#1801)

Changed

• refactor: Rename Create to CreateDir for its behavior changed (#2019)

Fixed

• fix: Cargo lock not updated (#2027)
• fix(services/s3): Ignore empty query to make it more compatible (#2028)
• fix(services/oss): Fix env not loaded for oss signer (#2029)

Docs

• docs: fix some typos (#2022)
• docs: add dev dependency section (#2021)

New Contributors

• @czybjtu made their first contribution in #2016
• @Yansongsongsong made their first contribution in #2022
• @haohuaijin made their first contribution in #2023
• @SkyFan2002 made their first contribution in #2006
• @cuichenli made their first contribution in #1801

Full Changelog: v0.31.1...v0.32.0

v0.31.1

NOTE: This release is not yet an official ASF release. We are still in the process of learning how to create a formal one.

v0.31.1 - 2023-04-17

Added

• feat(services/azdfs): support rename (#1929)
• test: Increate copy/move nested path test depth (#1932)
• feat(layers): add a basic minitrace layer (#1931)
• feat: add Writer::abort method (#1937)
• feat(services/gcs): Allow setting PredefinedAcl (#1989)
• feat(services/oss): add oss cache-control header support (#1986)
• feat: Add PreconditionFailed Error so users can handle them (#1993)
• feat: add http if_none_match support (#1995)
• feat: add oss if-none-match support (#1997)
• feat(services/gcs): Allow setting default storage_class (#1996)
• feat(binding/C): add clang-format for c binding (#2003)

Changed

• refactor: Polish the behavior of scan (#1926)
• refactor: Polish the implementation of webhdfs (#1935)

Fixed

• fix: sled should not be enabled by default (#1923)
• fix: kv adapter's writer implementation fixed to honour empty writes (#1934)
• fix(services/azblob): fix copy missing content-length (#2000)

Docs

• docs: Adding docs link to python binding (#1921)
• docs(bindings/python): fix wrong doc link (#1928)
• docs: Add contributing for OpenDAL (#1984)
• docs: Add explanation in contributing (#1985)
• docs: Feel relax in community and don't hurry (#1987)
• docs: update contributing (#1998)
• docs(services/memory): Fix wrong explanation (#2002)
• docs: Add OpenDAL VISION (#2007)
• docs: update VISION and behavior tests doc (#2009)

CI

• ci(bindings/nodejs): Access should be set to public before publish (#1919)
• ci: Re-enable webhdfs test (#1925)
• chore: add .editorconfig (#1988)
• ci: Fix format after adding editorconfig (#1990)

New Contributors

• @sa- made their first contribution in #1921
• @ahonn made their first contribution in #1988
• @silver-ymz made their first contribution in #1989
• @Nickqiaoo made their first contribution in #1995
• @thorseraq made their first contribution in #1997

Full Changelog: v0.31.0...v0.31.1

v0.31.0

NOTE: This release is not yet an official ASF release. We are still in the process of learning how to create a formal one.

Upgrade to v0.31

In version v0.31 of OpenDAL, we made some internal refactoring to improve its compatibility with the ecosystem.

MSRV Bump

We increased the MSRV to 1.64 from v0.31 onwards. Although it is still possible to build OpenDAL under older rustc versions, we cannot guarantee that any issues related to them will be fixed.

Accept std::time::Duration instead

Previously, OpenDAL accepted time::Duration as input for presign_xxx. However, since v0.31, we have changed this to accept std::time::Duration so that users do not need to depend on time. Internally, we migrated from time to chrono for better integration with other parts of the ecosystem.

disable_ec2_metadata for services s3

We have added a new configuration option called disable_ec2_metadata for the S3 service in response to a mistake where it was mixed up with another option called disable_config_load. Users who want to disable loading credentials from EC2 metadata should set this option instead.

Services Feature Flag

Starting from v0.31, all services in OpenDAL are split into different feature flags. To enable only S3 support, use the following TOML configuration:

opendal = {
    version = "0.31",
    default-features = false,
    features = ["services-s3"]
}

---

v0.31.0 - 2023-04-12

Added

• feat(bindings/java): add cucumber test (#1809)
• feat(bindings/java): setup Java CI (#1823)
• feat: add if_none_match support (#1832)
• feat: Retry when some of batch operations are failing (#1840)
• feat: add writer support for aliyun oss (#1842)
• feat(core): Add Copy Support (#1841)
• feat(bindings/c): fix c bindings makefile (#1849)
• feat(core): add behavior tests for copy & blocking_copy (#1852)
• feat(s3): allow users to specify storage_class (#1854)
• feat(s3): Support copy (#1856)
• Add check for s3 bucket name (#1857)
• feat(core): Support rename (#1862)
• feat(bindings/nodejs): add copy and rename (#1866)
• feat(azblob): Support copy (#1868)
• feat(gcs): copy support for GCS (#1869)
• feat(bindings/c): framework of add basic io and init logics (#1861)
• feat(webdav): support copy (#1870)
• feat(services/oss): Add Copy Support (#1874)
• feat(services/obs): Add Copy Support (#1876)
• feat(services/webdav): Support Rename (#1878)
• binding/c: parse opendal to use typed BlockingOperator (#1881)
• binding/c: clean up comments and type assertion for BlockingOperator (#1883)
• binding(python): Support python binding benchmark for opendal (#1882)
• feat(bindings/c): add support for free heap-allocated operator (#1890)
• feat(binding/c): add is_exist to operator (#1892)
• feat(bindings/java): add Stat support (#1894)
• feat(services/gcs): Add customed token loader support (#1908)
• feat(services/oss): remove unused builder prop allow_anonymous (#1913)
• feat: Add feature flag for all services (#1915)

Changed

• refactor(core): Simplify the usage of BatchOperation and BatchResults (#1843)
• refactor: Use reqwest blocking client instead of ureq (#1853)
• refactor: Bump MSRV to 1.64 (#1864)
• refactor: Remove not used blocking http client (#1895)
• refactor: Change presign to async for future refactor (#1900)
• refactor(services/gcs): Migrate to async reqsign (#1906)
• refactor(services/azdfs): Migrate to async reqsign (#1903)
• refactor(services/azblob): Adopt new reqsign (#1902)
• refactor(services/s3): Migrate to async reqsign (#1909)
• refactor(services/oss): Migrate to async reqsign (#1911)
• refactor: Use chrono instead of time to work well with ecosystem (#1912)
• refactor(service/obs): Migrate obs to async reqsign (#1914)

Fixed

• fix: podling website check (#1838)
• fix(website): copyright update (#1839)
• fix(core): Add checks before doing copy (#1845)
• fix(core): S3 Copy should set SSE headers (#1860)
• fix: Fix presign related unit tests (#1910)

Docs

• docs(bindings/nodejs): fix build failed (#1819)
• docs: fix several typos in the documentation (#1846)
• doc(bindings/nodejs): update presign example in doc (#1901)

CI

• ci: Fix build for nodejs binding on macos (#1813)
• binding/c: build: add phony to makefile, and some improve (#1850)
• ci: upgrade hawkeye action (#1834)

Chore

• chore(bindings/nodejs): add deno benchmark (#1814)
• chore: Add CODEOWNERS (#1817)
• chore(deps): bump opentelemetry-jaeger from 0.16.0 to 0.18.0 (#1829)
• chore(deps): bump opentelemetry from 0.17.0 to 0.19.0 (#1830)
• chore(deps): bump tokio from 1.26.0 to 1.27.0 (#1828)
• chore(deps): bump napi-derive from 2.12.1 to 2.12.2 (#1827)
• chore(deps): bump async-trait from 0.1.67 to 0.1.68 (#1826)
• chore: Cleanup code for oss writer (#1847)
• chore: Make clippy happy (#1865)
• binding(python): Format python code in binding (#1885)

New Contributors

• @gitccl made their first contribution in #1832
• @v0y4g3r made their first contribution in #1842
• @Ji-Xinyou made their first contribution in #1846
• @darknight made their first contribution in #1857
• @Zheaoli made their first contribution in #1882

Full Changelog: v0.30.5...v0.31.0

v0.30.5

NOTE: This release is not yet an official ASF release. We are still in the process of learning how to create a formal one.

v0.30.5 - 2023-03-31

Added

• feat(oli): implement oli rm (#1774)
• feat(bindings/nodejs): Support presign (#1772)
• feat(oli): implement oli stat (#1778)
• feat(bindings/object_store): Add support for list and list_with_delimiter (#1784)
• feat(oli): implement oli cp -r (#1787)
• feat(bindings/nodejs): Make PresignedRequest serializable (#1797)
• feat(binding/c): add build.rs and cbindgen as dep to gen header (#1793)
• feat(bindings/nodejs): Add more APIs and examples (#1799)
• feat: reader_with and writer_with (#1803)
• feat: add override_cache_control (#1804)
• feat: add cache_control to OpWrite (#1806)

Changed

• refactor(oli): switch to Operator::scan and Operator::remove_all (#1779)
• refactor(bindings/nodejs): Polish benchmark to make it more readable (#1810)

Fixed

• fix(oli): set the root of fs service to '/' (#1773)
• fix: align WebDAV stat with RFC specification (#1783)
• fix(bindings/nodejs): fix read benchmark (#1805)

CI

• ci: Split clippy and docs check (#1785)
• ci(bindings/nodejs): Support aarch64-apple-darwin (#1780)
• ci(bindings/nodejs): publish with LICENSE & NOTICE (#1790)
• ci(services/redis): Add dragonfly test (#1808)

Chore

• chore(bindings/python): update maturin to 0.14.16 (#1777)
• chore(bin/oli): Set oli version from package version (#1786)
• chore(oli): set cli version in a central place (#1789)
• chore: don't pin time version (#1788)
• chore(bindings/nodejs): init benchmark (#1798)
• chore(bindings/nodejs): Fix generated headers (#1802)

New Contributors

• @bxb100 made their first contribution in #1783
• @nasen23 made their first contribution in #1784
• @promer94 made their first contribution in #1798

Full Changelog: v0.30.4...v0.30.5

v0.30.4

NOTE: This release is not yet an official ASF release. We are still in the process of learning how to create a formal one.

v0.30.4 - 2023-03-26

Added

• feat(oli): add config file to oli (#1706)
• feat: make oli support more services (#1717)
• feat(bindings/ruby): Setup the integrate with magnus (#1712)
• feat(bindings/ruby): setup cucumber tests (#1725)
• feat(bindings/python): convert to mixed Python/Rust project layout (#1729)
• RFC-1735: Operation Extension (#1735)
• feat(oli): load config from both env and file (#1737)
• feat(bindings/ruby): support read and write (#1734)
• feat(bindings/ruby): support stat, and pass all blocking bdd test (#1743)
• feat(bindings/ruby): add namespace (#1745)
• feat: Add override_content_disposition for OpRead (#1742)
• feat(bindings/java): add java binding (#1736)
• feat(oli): implement oli ls (#1755)
• feat(oli): implement oli cat (#1759)

Fixed

• fix(bindings/nodejs): Publish sub-package name (#1704)

Docs

• docs: Update comparison vs object_store (#1698)
• docs(bindings/python): add pdoc to docs env (#1718)
• docs: List working on bindings in README (#1753)

CI

• ci: Fix workflow not triggered when itself changed (#1716)
• ci: Remove ROCKSDB_LIB_DIR after we didn't install librocksdb (#1719)
• ci: Fix nodejs built failed for "Unexpected token o in JSON at position 0" (#1722)
• ci: Split cache into more parts (#1730)
• ci: add a basic ci for ruby (#1744)
• ci: Remove target from cache (#1764)

Chore

• chore: Fix CHANGELOG not found (#1694)
• chore: Remove publish=false of oli (#1697)
• chore: Fix a few typos in code comment (#1701)
• chore(bindins/nodejs): Update README (#1709)
• chore: rename binaries to bin (#1714)
• chore: bump rocksdb to resolve dependency conflicts with magnus (#1713)
• chore(bindings/nodejs): Remove outdated napi patches (#1727)
• chore: Add CITATION file for OpenDAL (#1746)
• chore: improve NotADirectory error message with ending slash (#1756)
• chore(bindings/python): update pyo3 to 0.18.2 (#1758)

New Contributors

• @viirya made their first contribution in #1701
• @knight42 made their first contribution in #1706
• @kidylee made their first contribution in #1736

Full Changelog: v0.30.3...v0.30.4

v0.30.3

NOTE: This release is not yet an official ASF release. We are still in the process of learning how to create a formal one.

v0.30.3 - 2023-03-20

Added

• feat: Infer storage name based on endpoint (#1551)
• feat(bindings/python): implement async file-like reader API (#1570)
• feat: website init (#1580)
• feat(bindings/python): implement list and scan for AsyncOperator (#1586)
• feat: Implement logging/metrics/tracing support for Writer/BlockingWriter (#1588)
• feat(bindings/python): expose layers to Python (#1591)
• feat(bindings/c): Setup the integrate with cbindgen (#1603)
• feat(bindings/nodejs): Auto-generate docs (#1625)
• feat: add max_batch_operations for AccessorInfo (#1615)
• feat(azblob): Add support for batch operations (#1610)
• services/redis: Implement Write::append with native support (#1651)
• feat(tests): Introducing BDD tests for all bindings (#1654)
• feat(bindings/nodejs): Migrate to BDD test (#1661)
• feat(bindings/nodejs): Add generated index.d.ts (#1664)
• feat(bindings/python): add auto-generated api docs (#1613)
• feat(bindings/python): add __repr__ to Operator and AsyncOperator (#1683)

Changed

• *: Change all files licenses to ASF (#1592)
• refactor(bindings/python): only enable pyo3/entension-module feature when building with maturin (#1680)

Fixed

• fix(bindings/python): Fix the metadata for Python binding (#1568)
• fix: Operator::remove_all behaviour on non-existing object fixed (#1587)
• fix: reset Reader::SeekState when poll completed (#1609)
• fix: Bucket config related error is misleadling (#1684)
• fix(services/s3): UploadId should be percent encoded (#1690)

CI

• ci: Fix typo in workflows (#1582)
• ci: Don't check dep updates so frequently (#1599)
• ci: Setup asf config (#1622)
• ci: Use gh-pages instead (#1623)
• ci: Update Github homepage (#1627)
• ci: Update description for OpenDAL (#1628)
• ci: Send notifications to commits@o.a.o (#1629)
• ci: set main branch to be protected (#1631)
• ci: Add release scripts for OpenDAL (#1637)
• ci: Add check scripts (#1638)
• ci: Remove rust-cache to allow we can test rust code now (#1643)
• ci: Enable license check back (#1663)
• ci(bindings/nodejs): Enable formatter (#1665)
• ci: Bring our actions back (#1668)
• ci: Use korandoru/hawkeye@v1.5.4 instead (#1672)
• ci: Fix license header check and doc check (#1674)
• infra: Add odev to simplify contributor's setup (#1687)

Docs

• docs: Migrate links to o.a.o (#1630)
• docs: update the old address and the LICENSE size. (#1633)
• doc: update doc-link (#1642)
• docs(blog): Way to Go: OpenDAL successfully entered Apache Incubator (#1652)
• docs: Reorganize README of core and whole project (#1676)
• doc: Update README.md for quickstart (#1650)
• doc: uncomment the use expr for operator example (#1685)

Website

• website: Let's deploy our new website (#1581)
• website: Fix CNAME not set (#1590)
• website: Fix website publish (#1626)
• website: Add GitHub entry (#1636)
• website: move some content of footer to navbar. (#1660)

Chore

• chore(bindings/nodejs): fix missing files to publish (#1569)
• chore(deps): bump lazy-regex from 2.4.1 to 2.5.0 (#1573)
• chore(deps): bump tokio from 1.25.0 to 1.26.0 (#1577)
• chore(deps): bump hyper from 0.14.24 to 0.14.25 (#1575)
• chore(deps): bump serde from 1.0.152 to 1.0.155 (#1576)
• chore(deps): bump peaceiris/actions-gh-pages from 3.9.0 to 3.9.2 (#1593)
• chore(deps): bump async-trait from 0.1.64 to 0.1.66 (#1594)
• chore(deps): bump serde_json from 1.0.93 to 1.0.94 (#1596)
• chore(deps): bump paste from 1.0.11 to 1.0.12 (#1598)
• chore(deps): bump napi from 2.11.2 to 2.11.3 (#1595)
• chore(deps): bump serde from 1.0.155 to 1.0.156 (#1600)
• chore: fix the remaining license (#1605)
• chore: add a basic workflow for c bindings (#1608)
• chore: manage deps with maturin (#1611)
• chore: Rename files to yaml (#1624)
• chore: remove PULL_REQUEST_TEMPLATE (#1634)
• chore: add NOTICE and DISCLAIMER (#1635)
• chore(operator): apply max_batch_limit for async operator (#1641)
• chore: replace datafuselabs/opendal with apache/incubator-opendal (#1647)
• chore: make check.sh be executable and update gitignore (#1648)
• chore(automation): fix release.sh packaging sha512sum (#1649)
• chore: Update metadata (#1666)
• chore(website): Remove authors.yml (#1669)
• chore: Move opendal related staffs to core (#1673)
• chore: Remove not needed ignore from licenserc (#1677)
• chore: Ignore generated docs from git (#1686)

New Contributors

• @xinlifoobar made their first contribution in #1551
• @flisky made their first contribution in #1609
• @tao12345666333 made their first contribution in #1631
• @tzssangglass made their first contribution in #1633
• @ZHOUSH41 made their first contribution in #1684
• @xunfeng1980 made their first contribution in #1650
• @zwpaper made their first contribution in #1685

Full Changelog: v0.30.2...v0.30.3

v0.30.2

What's Changed

• ci(bindings/nodejs): Fix nodejs package can't uploaded by @Xuanwo in #1564
• Bump to version 0.30.2 by @Xuanwo in #1565

Full Changelog: v0.30.1...v0.30.2

v0.30.1

What's Changed

• chore(bindings/nodejs): update license in package.json by @suyanhanx in #1556
• ci: Fix python & nodejs not released correctly by @Xuanwo in #1559
• docs: Fix Operator::create() has been removed by @Xuanwo in #1560
• Bump to version 0.30.1 by @Xuanwo in #1561

Full Changelog: v0.30.0...v0.30.1