Skip to content

Commit

Permalink
Merge commit '926546b153a701dfbfc71eb8157f9a41320444f8'
Browse files Browse the repository at this point in the history
* commit '926546b153a701dfbfc71eb8157f9a41320444f8': (486 commits)
  chore: bump version to 0.14.1 (langgenius#11784)
  feat:add hunyuan model(hunyuan-role, hunyuan-large, hunyuan-large-rol… (langgenius#11766)
  chore(opendal_storage): remove unused comment (langgenius#11783)
  feat: Disable the "Forgot your password?" button when the mail server setup is incomplete (langgenius#11653)
  chore(.env.example): add comments for opendal (langgenius#11778)
  Lindorm vdb bug-fix (langgenius#11790)
  fix: imperfect service-api introduction text (langgenius#11782)
  feat: add openai o1 & update pricing and max_token of other models (langgenius#11780)
  fix: file upload auth (langgenius#11774)
  feat: add parameters for JinaReaderTool (langgenius#11613)
  feat: full support for opendal and sync configurations between .env and docker-compose (langgenius#11754)
  feat(app_factory): speed up api startup (langgenius#11762)
  fix: Prevent redirection to /overview when accessing /workflow. (langgenius#11733)
  (doc) fix: update cURL examples to include Authorization header (langgenius#11750)
  Fix explore app icon (langgenius#11742)
  chore: improve gemini models (langgenius#11745)
  feat: use Gemini response metadata for token counting (langgenius#11743)
  chore: update comments in docker env file (langgenius#11705)
  feat(ark): support doubao vision series models (langgenius#11740)
  chore: the consistency of MultiModalPromptMessageContent (langgenius#11721)
  ...

# Conflicts:
#	api/configs/app_config.py
#	api/core/helper/code_executor/code_executor.py
#	web/yarn.lock
  • Loading branch information
Scorpion1221 committed Dec 19, 2024
2 parents 4ac2e00 + 926546b commit 7e8234c
Show file tree
Hide file tree
Showing 1,427 changed files with 49,580 additions and 10,460 deletions.
4 changes: 2 additions & 2 deletions .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
FROM mcr.microsoft.com/devcontainers/python:3.10
FROM mcr.microsoft.com/devcontainers/python:3.12

# [Optional] Uncomment this section to install additional OS packages.
# RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
# && apt-get -y install --no-install-recommends <your-package-list-here>
# && apt-get -y install --no-install-recommends <your-package-list-here>
2 changes: 1 addition & 1 deletion .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/anaconda
{
"name": "Python 3.10",
"name": "Python 3.12",
"build": {
"context": "..",
"dockerfile": "Dockerfile"
Expand Down
3 changes: 2 additions & 1 deletion .devcontainer/post_create_command.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,6 @@ echo 'alias start-api="cd /workspaces/dify/api && poetry run python -m flask run
echo 'alias start-worker="cd /workspaces/dify/api && poetry run python -m celery -A app.celery worker -P gevent -c 1 --loglevel INFO -Q dataset,generation,mail,ops_trace,app_deletion"' >> ~/.bashrc
echo 'alias start-web="cd /workspaces/dify/web && npm run dev"' >> ~/.bashrc
echo 'alias start-containers="cd /workspaces/dify/docker && docker-compose -f docker-compose.middleware.yaml -p dify up -d"' >> ~/.bashrc
echo 'alias stop-containers="cd /workspaces/dify/docker && docker-compose -f docker-compose.middleware.yaml -p dify down"' >> ~/.bashrc

source /home/vscode/.bashrc
source /home/vscode/.bashrc
36 changes: 36 additions & 0 deletions .github/actions/setup-poetry/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: Setup Poetry and Python

inputs:
python-version:
description: Python version to use and the Poetry installed with
required: true
default: '3.11'
poetry-version:
description: Poetry version to set up
required: true
default: '1.8.4'
poetry-lockfile:
description: Path to the Poetry lockfile to restore cache from
required: true
default: ''

runs:
using: composite
steps:
- name: Set up Python ${{ inputs.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}
cache: pip

- name: Install Poetry
shell: bash
run: pip install poetry==${{ inputs.poetry-version }}

- name: Restore Poetry cache
if: ${{ inputs.poetry-lockfile != '' }}
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}
cache: poetry
cache-dependency-path: ${{ inputs.poetry-lockfile }}
41 changes: 16 additions & 25 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -1,34 +1,25 @@
# Checklist:
# Summary

> [!IMPORTANT]
> Please review the checklist below before submitting your pull request.
- [ ] Please open an issue before creating a PR or link to an existing issue
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I ran `dev/reformat`(backend) and `cd web && npx lint-staged`(frontend) to appease the lint gods

# Description

Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. Close issue syntax: `Fixes #<issue number>`, see [documentation](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword) for more details.
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes
> [!Tip]
> Close issue syntax: `Fixes #<issue number>` or `Resolves #<issue number>`, see [documentation](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword) for more details.
## Type of Change

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] This change requires a documentation update, included: [Dify Document](https://github.com/langgenius/dify-docs)
- [ ] Improvement, including but not limited to code refactoring, performance optimization, and UI/UX improvement
- [ ] Dependency upgrade

# Testing Instructions
# Screenshots

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
| Before | After |
|--------|-------|
| ... | ... |

- [ ] Test A
- [ ] Test B
# Checklist

> [!IMPORTANT]
> Please review the checklist below before submitting your pull request.
- [ ] This change requires a documentation update, included: [Dify Document](https://github.com/langgenius/dify-docs)
- [x] I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
- [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
- [x] I've updated the documentation accordingly.
- [x] I ran `dev/reformat`(backend) and `cd web && npx lint-staged`(frontend) to appease the lint gods

34 changes: 5 additions & 29 deletions .github/workflows/api-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ on:
paths:
- api/**
- docker/**
- .github/workflows/api-tests.yml

concurrency:
group: api-tests-${{ github.head_ref || github.run_id }}
Expand All @@ -19,24 +20,18 @@ jobs:
strategy:
matrix:
python-version:
- "3.10"
- "3.11"
- "3.12"

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
- name: Setup Poetry and Python ${{ matrix.python-version }}
uses: ./.github/actions/setup-poetry
with:
python-version: ${{ matrix.python-version }}
cache-dependency-path: |
api/pyproject.toml
api/poetry.lock
- name: Install Poetry
uses: abatilo/actions-poetry@v3
poetry-lockfile: api/poetry.lock

- name: Check Poetry lockfile
run: |
Expand Down Expand Up @@ -67,7 +62,7 @@ jobs:
run: sh .github/workflows/expose_service_ports.sh

- name: Set up Sandbox
uses: hoverkraft-tech/compose-action@v2.0.0
uses: hoverkraft-tech/compose-action@v2.0.2
with:
compose-file: |
docker/docker-compose.middleware.yaml
Expand All @@ -77,22 +72,3 @@ jobs:
- name: Run Workflow
run: poetry run -C api bash dev/pytest/pytest_workflow.sh

- name: Set up Vector Stores (Weaviate, Qdrant, PGVector, Milvus, PgVecto-RS, Chroma, MyScale, ElasticSearch, Couchbase)
uses: hoverkraft-tech/compose-action@v2.0.0
with:
compose-file: |
docker/docker-compose.yaml
services: |
weaviate
qdrant
couchbase-server
etcd
minio
milvus-standalone
pgvecto-rs
pgvector
chroma
elasticsearch
- name: Test Vector Stores
run: poetry run -C api bash dev/pytest/pytest_vdb.sh
21 changes: 7 additions & 14 deletions .github/workflows/db-migration-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ on:
- main
paths:
- api/migrations/**
- .github/workflows/db-migration-test.yml

concurrency:
group: db-migration-test-${{ github.ref }}
Expand All @@ -14,25 +15,15 @@ concurrency:
jobs:
db-migration-test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version:
- "3.10"

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
- name: Setup Poetry and Python
uses: ./.github/actions/setup-poetry
with:
python-version: ${{ matrix.python-version }}
cache-dependency-path: |
api/pyproject.toml
api/poetry.lock
- name: Install Poetry
uses: abatilo/actions-poetry@v3
poetry-lockfile: api/poetry.lock

- name: Install dependencies
run: poetry install -C api
Expand All @@ -43,7 +34,7 @@ jobs:
cp middleware.env.example middleware.env
- name: Set up Middlewares
uses: hoverkraft-tech/compose-action@v2.0.0
uses: hoverkraft-tech/compose-action@v2.0.2
with:
compose-file: |
docker/docker-compose.middleware.yaml
Expand All @@ -57,6 +48,8 @@ jobs:
cp .env.example .env
- name: Run DB Migration
env:
DEBUG: true
run: |
cd api
poetry run python -m flask upgrade-db
3 changes: 2 additions & 1 deletion .github/workflows/expose_service_ports.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,6 @@ yq eval '.services["pgvecto-rs"].ports += ["5431:5432"]' -i docker/docker-compos
yq eval '.services["elasticsearch"].ports += ["9200:9200"]' -i docker/docker-compose.yaml
yq eval '.services.couchbase-server.ports += ["8091-8096:8091-8096"]' -i docker/docker-compose.yaml
yq eval '.services.couchbase-server.ports += ["11210:11210"]' -i docker/docker-compose.yaml
yq eval '.services.tidb.ports += ["4000:4000"]' -i docker/docker-compose.yaml

echo "Ports exposed for sandbox, weaviate, qdrant, chroma, milvus, pgvector, pgvecto-rs, elasticsearch, couchbase"
echo "Ports exposed for sandbox, weaviate, tidb, qdrant, chroma, milvus, pgvector, pgvecto-rs, elasticsearch, couchbase"
25 changes: 10 additions & 15 deletions .github/workflows/style.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,34 +22,29 @@ jobs:
id: changed-files
uses: tj-actions/changed-files@v45
with:
files: api/**

- name: Set up Python
uses: actions/setup-python@v5
if: steps.changed-files.outputs.any_changed == 'true'
with:
python-version: '3.10'
files: |
api/**
.github/workflows/style.yml
- name: Install Poetry
- name: Setup Poetry and Python
if: steps.changed-files.outputs.any_changed == 'true'
uses: abatilo/actions-poetry@v3
uses: ./.github/actions/setup-poetry

- name: Python dependencies
- name: Install dependencies
if: steps.changed-files.outputs.any_changed == 'true'
run: poetry install -C api --only lint

- name: Ruff check
if: steps.changed-files.outputs.any_changed == 'true'
run: poetry run -C api ruff check ./api
run: |
poetry run -C api ruff --version
poetry run -C api ruff check ./api
poetry run -C api ruff format --check ./api
- name: Dotenv check
if: steps.changed-files.outputs.any_changed == 'true'
run: poetry run -C api dotenv-linter ./api/.env.example ./web/.env.example

- name: Ruff formatter check
if: steps.changed-files.outputs.any_changed == 'true'
run: poetry run -C api ruff format --check ./api

- name: Lint hints
if: failure()
run: echo "Please run 'dev/reformat' to fix the fixable linting errors."
Expand Down
73 changes: 73 additions & 0 deletions .github/workflows/vdb-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
name: Run VDB Tests

on:
pull_request:
branches:
- main
paths:
- api/core/rag/datasource/**
- docker/**
- .github/workflows/vdb-tests.yml
- api/poetry.lock
- api/pyproject.toml

concurrency:
group: vdb-tests-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

jobs:
test:
name: VDB Tests
runs-on: ubuntu-latest
strategy:
matrix:
python-version:
- "3.11"
- "3.12"

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Setup Poetry and Python ${{ matrix.python-version }}
uses: ./.github/actions/setup-poetry
with:
python-version: ${{ matrix.python-version }}
poetry-lockfile: api/poetry.lock

- name: Check Poetry lockfile
run: |
poetry check -C api --lock
poetry show -C api
- name: Install dependencies
run: poetry install -C api --with dev

- name: Set up dotenvs
run: |
cp docker/.env.example docker/.env
cp docker/middleware.env.example docker/middleware.env
- name: Expose Service Ports
run: sh .github/workflows/expose_service_ports.sh

- name: Set up Vector Stores (TiDB, Weaviate, Qdrant, PGVector, Milvus, PgVecto-RS, Chroma, MyScale, ElasticSearch, Couchbase)
uses: hoverkraft-tech/compose-action@v2.0.2
with:
compose-file: |
docker/docker-compose.yaml
services: |
weaviate
qdrant
couchbase-server
etcd
minio
milvus-standalone
pgvecto-rs
pgvector
chroma
elasticsearch
tidb
- name: Test Vector Stores
run: poetry run -C api bash dev/pytest/pytest_vdb.sh
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,7 @@ docker/volumes/pgvector/data/*
docker/volumes/pgvecto_rs/data/*
docker/volumes/couchbase/*
docker/volumes/oceanbase/*
!docker/volumes/oceanbase/init.d

docker/nginx/conf.d/default.conf
docker/nginx/ssl/*
Expand Down
Loading

0 comments on commit 7e8234c

Please sign in to comment.