Skip to content

Commit

Permalink
dev-revise-http-connector (#645)
Browse files Browse the repository at this point in the history
* refactor jsonl connector
* add performance improvements for uvicorn config
* make timeout parameter configurable
* update changelog
* find efficient kafka producer settings
* add throtteling queue
* exclude setuptools from build
  • Loading branch information
ekneg54 authored Aug 20, 2024
1 parent e2caf31 commit 1b7cb39
Show file tree
Hide file tree
Showing 15 changed files with 2,159 additions and 44 deletions.
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,23 @@
## next release
### Breaking
### Features
### Improvements
### Bugfix

## 13.1.0
### Features

* `pre_detector` now normalizes timestamps with configurable parameters timestamp_field, source_format, source_timezone and target_timezone
* `pre_detector` now writes tags in failure cases
* `ProcessingWarnings` now can write `tags` to the event
* add `timeout` parameter to logprep http generator to set the timeout in seconds for requests
* add primitive rate limiting to `http_input` connector

### Improvements

* switch to `uvloop` as default loop for the used threaded http uvicorn server
* switch to `httptools` as default http implementation for the used threaded http uvicorn server

### Bugfix

* remove redundant chart features for mounting secrets
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ ARG no_proxy

ADD . /logprep
WORKDIR /logprep
RUN python -m pip install --upgrade pip wheel setuptools>=72.2.0
RUN python -m venv /opt/venv
# Make sure we use the virtualenv:
ENV PATH="/opt/venv/bin:$PATH"
RUN python -m pip install --upgrade pip wheel setuptools

RUN if [ "$LOGPREP_VERSION" = "dev" ]; then pip install .;\
elif [ "$LOGPREP_VERSION" = "latest" ]; then pip install git+https://github.com/fkie-cad/Logprep.git@latest; \
Expand Down
5 changes: 2 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,9 +232,8 @@ Details about the rule language and how to write rules for the processors can be

## Getting Started

For installation instructions see: https://logprep.readthedocs.io/en/latest/getting_started.html#installation
For execution instructions see: https://logprep.readthedocs.io/en/latest/getting_started.html#run-logprep

For installation instructions see: https://logprep.readthedocs.io/en/latest/installation.html
For execution instructions see: https://logprep.readthedocs.io/en/latest/user_manual/execution.html

### Reload the Configuration

Expand Down
Loading

0 comments on commit 1b7cb39

Please sign in to comment.