Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NEW: Add libnvjpeg2k and libnvtiff #28142

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

carterbox
Copy link
Member

@carterbox carterbox commented Nov 7, 2024

Checklist

  • Title of this PR is meaningful: e.g. "Adding my_nifty_package", not "updated meta.yaml".
  • License file is packaged (see here for an example).
  • Source is from official source.
  • Package does not vendor other packages. (If a package uses the source of another package, they should be separate packages or the licenses of all packages need to be packaged).
  • If static libraries are linked in, the license of the static library is packaged.
  • Package does not ship static libraries. If static libraries are needed, follow CFEP-18.
  • Build number is 0.
  • A tarball (url) rather than a repo (e.g. git_url) is used in your recipe (see here for more details).
  • GitHub users listed in the maintainer section have posted a comment confirming they are willing to be listed there.
  • When in trouble, please check our knowledge base documentation before pinging a team.

Closes #28216
Closes #28217

Copy link

github-actions bot commented Nov 7, 2024

Hi! This is the staged-recipes linter and your PR looks excellent! 🚀

@conda-forge-admin
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/nvjpeg2k/meta.yaml) and found it was in an excellent condition.

recipes/nvjpeg2k/meta.yaml Outdated Show resolved Hide resolved
ln -s ${PREFIX}/${targetsDir}/$j ${PREFIX}/$j

if [[ $j =~ \.so\. ]]; then
patchelf --set-rpath '$ORIGIN' --force-rpath ${PREFIX}/${targetsDir}/$j
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@carterbox
Copy link
Member Author

Packaging libnvjpeg2k0-0.8.0.38-hbc370b7_0
number of files: 4
Skipping binary relocation logic
Making absolute symlink relative (lib/libnvjpeg2k.so.0.8.0.38 -> /home/conda/staged-recipes/build_artifacts/libnvjpeg2k-split_1731453510442/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/targets/x86_64-linux/lib/libnvjpeg2k.so.0.8.0.38 :-> ../targets/x86_64-linux/lib/libnvjpeg2k.so.0.8.0.38)
Making absolute symlink relative (lib/libnvjpeg2k.so.0 -> /home/conda/staged-recipes/build_artifacts/libnvjpeg2k-split_1731453510442/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/targets/x86_64-linux/lib/libnvjpeg2k.so.0 :-> ../targets/x86_64-linux/lib/libnvjpeg2k.so.0.8.0.38)
   INFO: sysroot: '/home/conda/staged-recipes/build_artifacts/libnvjpeg2k-split_1731453510442/_build_env/x86_64-conda-linux-gnu/sysroot/' files: '['usr/share/zoneinfo', 'usr/share/locale/zh_TW/LC_MESSAGES/libc.mo', 'usr/share/locale/zh_CN/LC_MESSAGES/libc.mo', 'usr/share/locale/vi/LC_MESSAGES/libc.mo']'
   INFO (libnvjpeg2k0,targets/x86_64-linux/lib/libnvjpeg2k.so.0.8.0.38): Needed DSO x86_64-conda-linux-gnu/sysroot/lib64/librt.so.1 found in CDT/compiler package conda-forge/noarch::sysroot_linux-64==2.17=h4a8ded7_18
   INFO (libnvjpeg2k0,targets/x86_64-linux/lib/libnvjpeg2k.so.0.8.0.38): Needed DSO x86_64-conda-linux-gnu/sysroot/lib64/libpthread.so.0 found in CDT/compiler package conda-forge/noarch::sysroot_linux-64==2.17=h4a8ded7_18
   INFO (libnvjpeg2k0,targets/x86_64-linux/lib/libnvjpeg2k.so.0.8.0.38): Needed DSO x86_64-conda-linux-gnu/sysroot/lib64/libdl.so.2 found in CDT/compiler package conda-forge/noarch::sysroot_linux-64==2.17=h4a8ded7_18
   INFO (libnvjpeg2k0,targets/x86_64-linux/lib/libnvjpeg2k.so.0.8.0.38): Needed DSO x86_64-conda-linux-gnu/sysroot/lib64/libm.so.6 found in CDT/compiler package conda-forge/noarch::sysroot_linux-64==2.17=h4a8ded7_18
  ERROR (libnvjpeg2k0,targets/x86_64-linux/lib/libnvjpeg2k.so.0.8.0.38): $RPATH/libgcc_s.so.1 not found in packages, sysroot(s) nor the missing_dso_whitelist.
.. is this binary repackaging?
   INFO (libnvjpeg2k0,targets/x86_64-linux/lib/libnvjpeg2k.so.0.8.0.38): Needed DSO x86_64-conda-linux-gnu/sysroot/lib64/libc.so.6 found in CDT/compiler package conda-forge/noarch::sysroot_linux-64==2.17=h4a8ded7_18
   INFO (libnvjpeg2k0,targets/x86_64-linux/lib/libnvjpeg2k.so.0.8.0.38): Needed DSO x86_64-conda-linux-gnu/sysroot/lib64/ld-linux-x86-64.so.2 found in CDT/compiler package conda-forge/noarch::sysroot_linux-64==2.17=h4a8ded7_18
WARNING (libnvjpeg2k0): dso library package conda-forge/linux-64::libstdcxx==14.2.0=hc0a3c3a_1 in requirements/run but it is not used (i.e. it is overdepending or perhaps statically linked? If that is what you want then add it to `build/ignore_run_exports`)

@jakirkham
Copy link
Member

Thanks Daniel! 🙏

Likely we need the same RPATH fixes that we have added for the CTK: conda-forge/cuda-feedstock#10

Perhaps the changes from this PR ( conda-forge/libnvjpeg-feedstock#8 ) would be a good starting point for fixing the issues in libraries

If we have executables that need RPATH fixes, we can discuss those as well. Sometimes these differ depending on the layout

@carterbox
Copy link
Member Author

I used the nvjpeg-feedstock as a template to create this recipe. Those fixes had already been merged before I started this recipe, so they are already incorporated.

The same overlinking errors occur and are suppressed in the libnjpeg feedstock. This is why I have added the conda-forge.yml to this recipe which supressed that error. Unfortunately, staged-recipes is not a perfect replica of the end feedstock, so it is ignored for now.

Packaging libnvjpeg
Packaging libnvjpeg-12.3.3.54-h97fd463_0
number of files: 4
Skipping binary relocation logic
Making absolute symlink relative (lib/libnvjpeg.so.12 -> /home/conda/feedstock_root/build_artifacts/libnvjpeg_1731689908939/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/targets/x86_64-linux/lib/libnvjpeg.so.12 :-> ../targets/x86_64-linux/lib/libnvjpeg.so.12.3.3.54)
Making absolute symlink relative (lib/libnvjpeg.so.12.3.3.54 -> /home/conda/feedstock_root/build_artifacts/libnvjpeg_1731689908939/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/targets/x86_64-linux/lib/libnvjpeg.so.12.3.3.54 :-> ../targets/x86_64-linux/lib/libnvjpeg.so.12.3.3.54)
   INFO: sysroot: '/home/conda/feedstock_root/build_artifacts/libnvjpeg_1731689908939/_build_env/x86_64-conda-linux-gnu/sysroot/' files: '['usr/share/zoneinfo', 'usr/share/locale/zh_TW/LC_MESSAGES/libc.mo', 'usr/share/locale/zh_CN/LC_MESSAGES/libc.mo', 'usr/share/locale/vi/LC_MESSAGES/libc.mo']'
   INFO (libnvjpeg,targets/x86_64-linux/lib/libnvjpeg.so.12.3.3.54): Needed DSO x86_64-conda-linux-gnu/sysroot/lib64/librt.so.1 found in CDT/compiler package conda-forge/noarch::sysroot_linux-64==2.17=h4a8ded7_18
   INFO (libnvjpeg,targets/x86_64-linux/lib/libnvjpeg.so.12.3.3.54): Needed DSO x86_64-conda-linux-gnu/sysroot/lib64/libpthread.so.0 found in CDT/compiler package conda-forge/noarch::sysroot_linux-64==2.17=h4a8ded7_18
   INFO (libnvjpeg,targets/x86_64-linux/lib/libnvjpeg.so.12.3.3.54): Needed DSO x86_64-conda-linux-gnu/sysroot/lib64/libdl.so.2 found in CDT/compiler package conda-forge/noarch::sysroot_linux-64==2.17=h4a8ded7_18
   INFO (libnvjpeg,targets/x86_64-linux/lib/libnvjpeg.so.12.3.3.54): Needed DSO x86_64-conda-linux-gnu/sysroot/lib64/libm.so.6 found in CDT/compiler package conda-forge/noarch::sysroot_linux-64==2.17=h4a8ded7_18
  ERROR (libnvjpeg,targets/x86_64-linux/lib/libnvjpeg.so.12.3.3.54): $RPATH/libgcc_s.so.1 not found in packages, sysroot(s) nor the missing_dso_whitelist.
.. is this binary repackaging?
   INFO (libnvjpeg,targets/x86_64-linux/lib/libnvjpeg.so.12.3.3.54): Needed DSO x86_64-conda-linux-gnu/sysroot/lib64/libc.so.6 found in CDT/compiler package conda-forge/noarch::sysroot_linux-64==2.17=h4a8ded7_18
   INFO (libnvjpeg,targets/x86_64-linux/lib/libnvjpeg.so.12.3.3.54): Needed DSO x86_64-conda-linux-gnu/sysroot/lib64/ld-linux-x86-64.so.2 found in CDT/compiler package conda-forge/noarch::sysroot_linux-64==2.17=h4a8ded7_18
WARNING (libnvjpeg): dso library package conda-forge/linux-64::libstdcxx==14.2.0=hc0a3c3a_1 in requirements/run but it is not used (i.e. it is overdepending or perhaps statically linked? If that is what you want then add it to `build/ignore_run_exports`)

@conda-forge-admin
Copy link
Contributor

conda-forge-admin commented Nov 15, 2024

Hi! This is the friendly automated conda-forge-linting service.

I failed to even lint the recipe, probably because of a conda-smithy bug 😢. This likely indicates a problem in your meta.yaml, though. To get a traceback to help figure out what's going on, install conda-smithy and run conda smithy recipe-lint --conda-forge . from the recipe directory.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/11864333049.

@carterbox carterbox changed the title NEW: Add libnvjpeg2k NEW: Add libnvjpeg2k and libnvtiff Nov 15, 2024
@carterbox
Copy link
Member Author

@adibbley, please review

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is using the targetsDir layout necessary for these packages? This is generally only used by packages included in cuda. The rpms/debs for these libraries install to system standard locations, such as:

$ rpm -qpl libnvjpeg2k0-cuda-12-0.8.0.38-1.x86_64.rpm
warning: libnvjpeg2k0-cuda-12-0.8.0.38-1.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID d42d0685: NOKEY
/usr/lib64/libnvjpeg2k
/usr/lib64/libnvjpeg2k/12
/usr/lib64/libnvjpeg2k/12/libnvjpeg2k.so.0
/usr/lib64/libnvjpeg2k/12/libnvjpeg2k.so.0.8.0.38
/usr/share/licenses/libnvjpeg2k0-cuda-12
/usr/share/licenses/libnvjpeg2k0-cuda-12/LICENSE

Note: libnvjpeg2k/12 subdir is used as those formats support side by side installs of the cuda variants, which is not needed here.

@jakirkham
Copy link
Member

jakirkham commented Nov 20, 2024

When you have a moment, could you please add a conda-forge.yml in each recipe directory like so?

os_version:
  linux_64: cos7
  linux_aarch64: cos7

It looks like conda-smithy includes this during feedstock generation. So this should get pulled into the final feedstocks at the end

This will help up track which OS version these binaries support and help us update to new OS versions/images when ready

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Package request: nvtiff Package request: libnvjpeg2k
4 participants