Releases: streamthoughts/kafka-connect-file-pulse
v2.2.0
Features
5bc2024 feat(plugin): allow excluding from processing files based on maximum age in ms (#161)
a3ef5db feat(filesystems): improve XMLFileInputIterator to allow excluding empty element (#159)
b00ef42 feat(scripts): add arg to specify number of connect workers
ef8fbc3 feat(plugin): add support for task file listing delegation
137c1a6 feat(filesystems): enchance XMLFileInputIterator with new config props
Improvements & Bugfixes
a693b52 fix(plugin): fix FilePulseSourceConnector should raise an error when FileSystemMonitorThread crash
c6f1f13 fix(api): fix empty document removing in XMLFileInputIterator
c0cc249 fix(api): improve support for XML by adding capabilities to merge schemas (#160)
Docs
0c92901 docs(site): fix page date and css
7000b84 docs(site): add release note for 2.1.0
Sub-Tasks
4f0add2 project(issue): add github stale bot config
34ab52a fix(scripts): update docker-compose for debug
a2fb4a5 sub-task(plugin): add new interface TaskPartitioner
bbeebda subtask(plugin): add new interface FileURIProvider
e2825c0 build(deps): bump commons-compress from 1.20 to 1.21
Docker Images
*Released on Docker hub
$ docker pull streamthoughts/kafka-connect-file-pulse:2.2.0v2.1.0
Improvements & Bugfixes
f1c071e refactor(plugin): change default value of offset.attributes.string to uri (#154)
d8aac2b refactor(plugin): enhance error handling when file do not exist anymore
8a17051 fix(plugin): fix ClassCastException when offset.attributes.string=inode (#153)
defee21 fix(plugin): remove duplicate log when closing file iterator
7897bcb fix(plugin): improve DefaultFileSystemMonitor to avoid scheduling files that may be cleanup by remaining tasks (#152)
28456db fix(plugin): fix task must close resources on error while starting
ddf8b86 fix(api): fix DeadLock on KafkaStateBackingStore
370a1d6 fix(filesystems): fix FileSystemMonitorThread should not fail if file metadata cannot be retrieved (#150)
e9462c8 fix(plugin): fix NPE using version 2.0 with KafkaFileObjectStateBackingStore (#149)
Docs
1b72a96 docs(site): fix syntax for 'exists' ScEL Built-in Function (#148)
7da7e41 docs(site): fix documentation error for StateBackingStore (#147)
Sub-Tasks
ec70611 build(deps): bump commons-io from 2.5 to 2.7
aa62022 fix(script): fix docker-compose for debugging
2723516 refactor(build): add mvn profiles for different storages
c1da2f8 refactor(build): improve makefile and add utility script for debugging
54ba1e5 build(maven): add meta info
Docker Images
Released on Docker hub
$ docker pull streamthoughts/kafka-connect-file-pulse:2.1.0v2.0.0
Features
13eed7b feat(plugin): add support for auto-creating the internal topic used by ConnectFilePulse (#139)
5c88877 feat(plugin): add InMemoryStateBackingStore for tracking status of file objects (#138)
52adca9 feat(filesystems): add support for Google Cloud Storage (#121)
7b49b81 feat(plugin): add new property max.scheduled.files (#122) (#123)
390ad82 feat(filesystems): add support for AWS S3 (#111)
92e3341 feat(filesystems): add support for Azure Blob Storage (#112)
685618a refactor(filters): migrate GrokFilter to use classes from grok-transformer (#118)
Improvements & Bugfixes
6ef5162 fix(api): fix decimal numbers not being correctly parsed (#142)
6c779af refactor(filesystems): make cleanup policy storage aware
d13c236 fix(filesystems): make compression codec more robust to encoding
7d2ddac docs(site): fix DateFilter formats config
e222414 fix(api): change digest value to string
SubTasks
1658d35 refaractor(api/filesystems): move FileInputIterator implementation to commons-fs
06385b3 refactor(filesystems): add module filepulse-commons-fs
57da04c subtask(all): refactor FilePulse API to support remote storages (#100)
ee4acad add github workflow
a3eb908 build(all): update to java 11
98eb51f build(mvn): add mavenwrapper
Docker Image
Released on Docker Hub
$ docker pull streamthoughts/kafka-connect-file-pulse:2.0.0v2.0.0-RC2
v2.0.0-RC1
Features
52adca9 feat(filesystems): add support for Google Cloud Storage (#121)
7b49b81 feat(plugin): add new property max.scheduled.files (#122) (#123)
390ad82 feat(filesystems): add support for AWS S3 (#111)
92e3341 feat(filesystems): add support for Azure Blob Storage (#112)
685618a refactor(filters): migrate GrokFilter to use classes from grok-transformer (#118)
Improvements & Bugfixes
d13c236 fix(filesystems): make compression codec more robust to encoding
7d2ddac docs(site): fix DateFilter formats config
e222414 fix(api): change digest value to string
SubTasks
1658d35 refaractor(api/filesystems): move FileInputIterator implementation to commons-fs
06385b3 refactor(filesystems): add module filepulse-commons-fs
57da04c subtask(all): refactor FilePulse API to support remote storages (#100)
ee4acad add github workflow
a3eb908 build(all): update to java 11
98eb51f build(mvn): add mavenwrapper
Docker Image
Released on Docker Hub
$ docker pull streamthoughts/kafka-connect-file-pulse:2.0.0-RC1v1.6.3
v1.6.2
v1.6.1
Improvements and Bug fixes
747c546 Fix python3 iteration (#89)
2f627fb ci(ci): bump confluentinc docker image versions (#89)
d325212 fix(build): cleanup unused import
42cf6d7 fix(api): fix regression with ConnectSchemaMapper (#95)
Docker Image
Released on Docker Hub
$ docker pull streamthoughts/kafka-connect-file-pulse:1.6.1
v1.6.0
Features
473cdda feat(plugin): add new config allow.tasks.reconfiguration.after.timeout.ms
e7878d8 feat(filters): Add split function to SCEL (#88)
Improvements and Bug fixes
87b9ff1 build(pom/docker): update Dockerfile base image and bump kafka version
668ce19 fix(filters): fix DelimitedRowFilter must convert fields based on schema (#87)
810ee27 fix - continue reading post JSON null value element observed
d0c0e97 fix(expression): should support empty string literal (#76)
Docker Image
Released on Docker Hub
$ docker pull streamthoughts/kafka-connect-file-pulse:1.6.0
v1.5.4
Improvements and Bug fixes
199db65 fix(plugins): fix XMLFileInputReader on the root document when force.array.on.fields is used (#75)
1c75a3e feat(expression): add new SCEL functions
3809af0 feat(plugin): update XMLFileInputReader to support path for forcing array
Docker Image
Released on Docker Hub
$ docker pull streamthoughts/kafka-connect-file-pulse:1.5.4