Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change ORT nightly python packages' name #22450

Merged
merged 5 commits into from
Oct 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions onnxruntime/python/tools/transformers/machine_info.py
Original file line number Diff line number Diff line change
Expand Up @@ -129,8 +129,6 @@ def get_related_packages(self) -> List[str]:
related_packages = [
"onnxruntime-gpu",
"onnxruntime",
"ort-nightly-gpu",
"ort-nightly",
"onnx",
"transformers",
"protobuf",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -249,11 +249,7 @@ def save_results(results, filename):

installed_packages = pkg_resources.working_set
installed_packages_list = sorted(
[
f"{i.key}=={i.version}"
for i in installed_packages
if i.key in ["ort-nightly-gpu", "ort-nightly", "onnxruntime", "onnxruntime-gpu"]
]
[f"{i.key}=={i.version}" for i in installed_packages if i.key in ["onnxruntime", "onnxruntime-gpu"]]
)

ort_pkg_name = ""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -291,11 +291,7 @@ def save_results(results, filename):

installed_packages = pkg_resources.working_set
installed_packages_list = sorted(
[
f"{i.key}=={i.version}"
for i in installed_packages
if i.key in ["ort-nightly-gpu", "ort-nightly", "onnxruntime", "onnxruntime-gpu"]
]
[f"{i.key}=={i.version}" for i in installed_packages if i.key in ["onnxruntime", "onnxruntime-gpu"]]
)
ort_pkg_name = ""
ort_pkg_version = ""
Expand Down
2 changes: 1 addition & 1 deletion onnxruntime/python/tools/transformers/run_benchmark.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#!/bin/bash
# -------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for
Expand Down Expand Up @@ -66,7 +67,7 @@
# export CUDA_VISIBLE_DEVICES=1

# This script will generate a logs file with a list of commands used in tests.
echo echo "ort=$run_ort torch=$run_torch torch2=$run_torch2 torchscript=$run_torchscript tensorflow=$run_tensorflow gpu_fp32=$run_gpu_fp32 gpu_fp16=$run_gpu_fp16 cpu=$run_cpu optimizer=$use_optimizer batch=$batch_sizes sequence=$sequence_length models=$models_to_test" arm64_bfloat16_fastmath_mode=$arm64_bfloat16_fastmath_mode >> benchmark.log

Check warning on line 70 in onnxruntime/python/tools/transformers/run_benchmark.sh

View workflow job for this annotation

GitHub Actions / Optional Lint

[shellcheck] reported by reviewdog 🐶 run_cpu is referenced but not assigned. Raw Output: ./onnxruntime/python/tools/transformers/run_benchmark.sh:70:167: warning: run_cpu is referenced but not assigned. (ShellCheck.SC2154)

Check warning on line 70 in onnxruntime/python/tools/transformers/run_benchmark.sh

View workflow job for this annotation

GitHub Actions / Optional Lint

[shellcheck] reported by reviewdog 🐶 sequence_length is referenced but not assigned (did you mean 'sequence_lengths'?). Raw Output: ./onnxruntime/python/tools/transformers/run_benchmark.sh:70:229: warning: sequence_length is referenced but not assigned (did you mean 'sequence_lengths'?). (ShellCheck.SC2154)

# Set it to false to skip testing. You can use it to dry run this script with the log file.
run_tests=true
Expand All @@ -91,7 +92,6 @@


if [ "$run_install" = true ] ; then
pip uninstall --yes ort-nightly ort-gpu-nightly
pip uninstall --yes onnxruntime
pip uninstall --yes onnxruntime-gpu
if [ "$run_cpu_fp32" = true ] || [ "$run_cpu_int8" = true ]; then
Expand Down Expand Up @@ -137,8 +137,8 @@
# -------------------------------------------
run_one_test() {
if [ "$run_ort" = true ] ; then
echo python $benchmark_script -m $1 $onnx_export_options $2 $3 $4 >> benchmark.log

Check warning on line 140 in onnxruntime/python/tools/transformers/run_benchmark.sh

View workflow job for this annotation

GitHub Actions / Optional Lint

[shellcheck] reported by reviewdog 🐶 Double quote to prevent globbing and word splitting. Raw Output: ./onnxruntime/python/tools/transformers/run_benchmark.sh:140:19: info: Double quote to prevent globbing and word splitting. (ShellCheck.SC2086)

Check warning on line 140 in onnxruntime/python/tools/transformers/run_benchmark.sh

View workflow job for this annotation

GitHub Actions / Optional Lint

[shellcheck] reported by reviewdog 🐶 Double quote to prevent globbing and word splitting. Raw Output: ./onnxruntime/python/tools/transformers/run_benchmark.sh:140:40: info: Double quote to prevent globbing and word splitting. (ShellCheck.SC2086)

Check warning on line 140 in onnxruntime/python/tools/transformers/run_benchmark.sh

View workflow job for this annotation

GitHub Actions / Optional Lint

[shellcheck] reported by reviewdog 🐶 Double quote to prevent globbing and word splitting. Raw Output: ./onnxruntime/python/tools/transformers/run_benchmark.sh:140:43: info: Double quote to prevent globbing and word splitting. (ShellCheck.SC2086)

Check warning on line 140 in onnxruntime/python/tools/transformers/run_benchmark.sh

View workflow job for this annotation

GitHub Actions / Optional Lint

[shellcheck] reported by reviewdog 🐶 Double quote to prevent globbing and word splitting. Raw Output: ./onnxruntime/python/tools/transformers/run_benchmark.sh:140:64: info: Double quote to prevent globbing and word splitting. (ShellCheck.SC2086)

Check warning on line 140 in onnxruntime/python/tools/transformers/run_benchmark.sh

View workflow job for this annotation

GitHub Actions / Optional Lint

[shellcheck] reported by reviewdog 🐶 Double quote to prevent globbing and word splitting. Raw Output: ./onnxruntime/python/tools/transformers/run_benchmark.sh:140:67: info: Double quote to prevent globbing and word splitting. (ShellCheck.SC2086)

Check warning on line 140 in onnxruntime/python/tools/transformers/run_benchmark.sh

View workflow job for this annotation

GitHub Actions / Optional Lint

[shellcheck] reported by reviewdog 🐶 Double quote to prevent globbing and word splitting. Raw Output: ./onnxruntime/python/tools/transformers/run_benchmark.sh:140:70: info: Double quote to prevent globbing and word splitting. (ShellCheck.SC2086)
echo python $benchmark_script -m $1 $benchmark_options $2 $3 $4 -i $input_counts >> benchmark.log

Check warning on line 141 in onnxruntime/python/tools/transformers/run_benchmark.sh

View workflow job for this annotation

GitHub Actions / Optional Lint

[shellcheck] reported by reviewdog 🐶 Double quote to prevent globbing and word splitting. Raw Output: ./onnxruntime/python/tools/transformers/run_benchmark.sh:141:19: info: Double quote to prevent globbing and word splitting. (ShellCheck.SC2086)

Check warning on line 141 in onnxruntime/python/tools/transformers/run_benchmark.sh

View workflow job for this annotation

GitHub Actions / Optional Lint

[shellcheck] reported by reviewdog 🐶 Double quote to prevent globbing and word splitting. Raw Output: ./onnxruntime/python/tools/transformers/run_benchmark.sh:141:40: info: Double quote to prevent globbing and word splitting. (ShellCheck.SC2086)
if [ "$run_tests" = true ] ; then
python $benchmark_script -m $1 $onnx_export_options $2 $3 $4
python $benchmark_script -m $1 $benchmark_options $2 $3 $4 -i $input_counts
Expand Down
5 changes: 2 additions & 3 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,6 @@ def parse_arg_remove_string(argv, arg_name_equal):
# Any combination of the following arguments can be applied

if parse_arg_remove_boolean(sys.argv, "--nightly_build"):
package_name = "ort-nightly"
nightly_build = True

wheel_name_suffix = parse_arg_remove_string(sys.argv, "--wheel_name_suffix=")
Expand Down Expand Up @@ -89,10 +88,10 @@ def parse_arg_remove_string(argv, arg_name_equal):
pass
elif parse_arg_remove_boolean(sys.argv, "--use_qnn"):
is_qnn = True
package_name = "onnxruntime-qnn" if not nightly_build else "ort-nightly-qnn"
package_name = "onnxruntime-qnn"

if is_rocm or is_migraphx:
package_name = "onnxruntime-rocm" if not nightly_build else "ort-rocm-nightly"
package_name = "onnxruntime-rocm"

# PEP 513 defined manylinux1_x86_64 and manylinux1_i686
# PEP 571 defined manylinux2010_x86_64 and manylinux2010_i686
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,7 @@ jobs:

- ${{ if eq(parameters.EnablePython, true) }}:
- powershell: |
python3 -m pip uninstall -y ort-nightly-gpu ort-nightly onnxruntime onnxruntime-gpu onnxruntime-training onnxruntime-directml ort-nightly-directml -qq
python3 -m pip uninstall -y onnxruntime onnxruntime-gpu onnxruntime-training onnxruntime-directml -qq
Get-ChildItem -Path dist/*.whl | foreach {pip --disable-pip-version-check install --upgrade $_.fullname}

workingDirectory: '$(Build.BinariesDirectory)\${{ parameters.BuildConfig }}\${{ parameters.BuildConfig }}'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ stages:
**/*.dll
- powershell: |
python -m pip uninstall -y ort-nightly-gpu ort-nightly onnxruntime onnxruntime-gpu -qq
python -m pip uninstall -y onnxruntime onnxruntime-gpu -qq
Get-ChildItem -Path $(Build.ArtifactStagingDirectory)/*.whl | foreach {pip --disable-pip-version-check install --force-reinstall --upgrade $_.fullname tabulate}
python -m pip install protobuf==3.18.1
Remove-Item -Recurse -Force onnxruntime
Expand Down Expand Up @@ -334,7 +334,7 @@ stages:
rm -rf $(Build.BinariesDirectory)/Release/onnxruntime $(Build.BinariesDirectory)/Release/pybind11
sudo rm -f /build /onnxruntime_src
sudo ln -s $(Build.SourcesDirectory) /onnxruntime_src
python3 -m pip uninstall -y ort-nightly-gpu ort-nightly onnxruntime onnxruntime-gpu onnxruntime-training onnxruntime-directml ort-nightly-directml -qq
python3 -m pip uninstall -y onnxruntime onnxruntime-gpu onnxruntime-training onnxruntime-directml -qq
cp $(Build.SourcesDirectory)/tools/ci_build/github/linux/docker/scripts/manylinux/requirements.txt $(Build.BinariesDirectory)/requirements.txt
# Test ORT with the latest ONNX release.
sed -i "s/git+http:\/\/github\.com\/onnx\/onnx.*/onnx/" $(Build.BinariesDirectory)/requirements.txt
Expand Down Expand Up @@ -497,7 +497,7 @@ stages:
**/*.dll
- powershell: |
python -m pip uninstall -y ort-nightly-gpu ort-nightly onnxruntime onnxruntime-gpu -qq
python -m pip uninstall -y onnxruntime onnxruntime-gpu -qq
Get-ChildItem -Path $(Build.ArtifactStagingDirectory)/*.whl | foreach {pip --disable-pip-version-check install --upgrade $_.fullname tabulate}
Remove-Item -Recurse -Force onnxruntime
python onnx_backend_test_series.py
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ stages:
continueOnError: true

- powershell: |
python -m pip uninstall -y ort-nightly-gpu ort-nightly onnxruntime onnxruntime-gpu -qq
python -m pip uninstall -y onnxruntime onnxruntime-gpu -qq
Get-ChildItem -Path $(Build.ArtifactStagingDirectory)/*.whl | foreach {pip --disable-pip-version-check install --upgrade $_.fullname tabulate}
Remove-Item -Recurse -Force onnxruntime
if ("$(ExtraParam)" -contains "--use_azure") {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -259,7 +259,7 @@ stages:
arguments: -cpu_arch x64 -install_prefix $(Build.BinariesDirectory)\${{ parameters.cmake_build_type }}\installed -build_config ${{ parameters.cmake_build_type }}

- powershell: |
python -m pip uninstall -y ort-nightly-gpu ort-nightly onnxruntime onnxruntime-gpu -qq
python -m pip uninstall -y onnxruntime onnxruntime-gpu -qq
Get-ChildItem -Path $(Build.ArtifactStagingDirectory)/*cp${{ replace(parameters.PYTHON_VERSION,'.','') }}*.whl | foreach {pip --disable-pip-version-check install --upgrade $_.fullname tabulate}
mkdir -p $(Agent.TempDirectory)\ort_test_data
Copy-Item -Path $(Build.sourcesDirectory)/onnxruntime/test/python/onnx_backend_test_series.py -Destination $(Agent.TempDirectory)\ort_test_data
Expand Down
Loading