Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci][python-package] add testing for the shape of raw score predictions #6765

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

StrikerRUS
Copy link
Collaborator

Refer to #6753.

Copy link
Collaborator

@jameslamb jameslamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you for adding this

@jameslamb
Copy link
Collaborator

@shiyu1994 can you please check if something has changed with our Azure DevOps account?

Those jobs are not running:

Screenshot 2024-12-22 at 9 41 26 PM

https://dev.azure.com/lightgbm-ci/lightgbm-ci/_build?definitionId=1

Screenshot 2024-12-22 at 9 42 47 PM

@shiyu1994
Copy link
Collaborator

I see the jobs on Azure DevOps are running for other PRs. Maybe we can retrigger it for this PR?

@StrikerRUS
Copy link
Collaborator Author

Trying to close-reopen for Azure restart.

@StrikerRUS StrikerRUS closed this Dec 24, 2024
@StrikerRUS StrikerRUS reopened this Dec 24, 2024
@jameslamb
Copy link
Collaborator

Excellent, thanks @shiyu1994 !

I do see jobs being picked up again there 🎉

Now we have a separate problem... all of the CUDA builds are failing like this at the "initialize containers" step:

  /usr/bin/docker ps --all --quiet --no-trunc --filter "label=df5bb3"
  bdd49dc44096c9f53785df313ab685de5547d00283d302b3c456f911d93f5196
  1085ca4cd1d18c5a6e8a4c772637b210cc7dd56e98a69abcce1976704fe9f5af
  /usr/bin/docker rm --force bdd49dc44096c9f53785df313ab685de5547d00283d302b3c456f911d93f5196
  Error: The operation was canceled.

(build link)

I just manually triggered a new run from master, restarting the docker daemon there.

image

build link: https://github.com/microsoft/LightGBM/actions/runs/12485778092

Hopefully that'll help.

@jameslamb
Copy link
Collaborator

:/ that did not help: https://github.com/microsoft/LightGBM/actions/runs/12485778092/job/34845059021

@shiyu1994 could you try fully rebooting the CUDA runner machine, then triggering a new CI run by clicking "Run workflow" at https://github.com/microsoft/LightGBM/actions/workflows/cuda.yml?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants