Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

USE_LOCAL_PYTHON is incompatible with ya SDK and leads to failure #21

Open
Krock21 opened this issue Sep 12, 2024 · 0 comments
Open

USE_LOCAL_PYTHON is incompatible with ya SDK and leads to failure #21

Krock21 opened this issue Sep 12, 2024 · 0 comments

Comments

@Krock21
Copy link

Krock21 commented Sep 12, 2024

ya make -DUSE_LOCAL_PYTHON=yes leads to compile failure because ya SDK is not compatible with local python SDK:

------- [CC] {pic, FAILED} $(S)/ydb/library/yql/udfs/common/python/bindings/py_dict.cpp
command /root/.ya/tools/v4/6495238978/bin/clang++ --target=x86_64-linux-gnu --sysroot=/root/.ya/tools/v4/243881345 -B/root/.ya/tools/v4/243881345/usr/bin -c -o /root/.ya/build/build_root/sr1c/0000d2/ydb/library/yql/udfs/common/python/bindings/py_dict.cpp.py3.pic.o -I/root/.ya/build/build_root/sr1c/0000d2 -I/ydb -I/ydb/contrib/libs/linux-headers -I/ydb/contrib/libs/linux-headers/_nf -I/ydb/contrib/libs/cxxsupp/libcxx/include -I/ydb/contrib/libs/cxxsupp/libcxxrt/include -I/ydb/contrib/libs/clang16-rt/include -I/ydb/contrib/libs/zlib/include -I/ydb/contrib/libs/double-conversion -I/ydb/contrib/libs/libc_compat/include/readpassphrase -I/ydb/contrib/libs/libc_compat/reallocarray -I/ydb/contrib/libs/libc_compat/random -I/ydb/contrib/libs/protobuf/src -I/ydb/contrib/restricted/abseil-cpp -I/ydb/contrib/libs/sparsehash/src -I/ydb/contrib/libs/brotli/include -I/ydb/contrib/libs/snappy/include -I/root/.ya/build/build_root/sr1c/0000d2/library/cpp/build_info -pipe -m64 -g -ggnu-pubnames -fexceptions -fno-common -ffunction-sections -fdata-sections -fuse-init-array -fcolor-diagnostics -faligned-allocation -fdebug-default-version=4 -fstack-protector -Wno-everything -DARCADIA_ROOT=/ydb -DARCADIA_BUILD_ROOT=/root/.ya/build/build_root/sr1c/0000d2 -D_THREAD_SAFE -D_PTHREADS -D_REENTRANT -D_LARGEFILE_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -D_YNDX_LIBUNWIND_ENABLE_EXCEPTION_BACKTRACE -D__LONG_LONG_SUPPORTED -DSSE_ENABLED=1 -DSSE3_ENABLED=1 -DSSSE3_ENABLED=1 -DSSE41_ENABLED=1 -DSSE42_ENABLED=1 -DPOPCNT_ENABLED=1 -DCX16_ENABLED=1 -fno-omit-frame-pointer -D_libunwind_ -DLIBCXX_BUILDING_LIBCXXRT -UNDEBUG -fPIC -Wno-array-parameter -Wno-deprecate-lax-vec-conv-all -Wno-unqualified-std-cast-call -Wno-unused-but-set-parameter -Wno-implicit-function-declaration -Wno-int-conversion -Wno-incompatible-function-pointer-types -Wno-address-of-packed-member -DCATBOOST_OPENSOURCE=yes -msse2 -msse3 -mssse3 -msse4.1 -msse4.2 -mpopcnt -mcx16 -I/usr/include/python3.8 -I/usr/include/python3.8 -UNDEBUG -fPIC -Wno-array-parameter -Wno-deprecate-lax-vec-conv-all -Wno-unqualified-std-cast-call -Wno-unused-but-set-parameter -Wno-implicit-function-declaration -Wno-int-conversion -Wno-incompatible-function-pointer-types -Wno-address-of-packed-member -std=c++20 -Wno-everything -DUDF_ABI_VERSION_MAJOR=2 -DUDF_ABI_VERSION_MINOR=27 -DUDF_ABI_VERSION_PATCH=0 -nostdinc++ -DCATBOOST_OPENSOURCE=yes -nostdinc++ /ydb/ydb/library/yql/udfs/common/python/bindings/py_dict.cpp failed with exit code 1 in /root/.ya/build/build_root/sr1c/0000d2
In file included from /ydb/ydb/library/yql/udfs/common/python/bindings/py_dict.cpp:1:
In file included from /ydb/ydb/library/yql/udfs/common/python/bindings/py_dict.h:3:
In file included from /ydb/ydb/library/yql/udfs/common/python/bindings/py_ptr.h:3:
In file included from /usr/include/python3.8/Python.h:8:
/usr/include/python3.8/pyconfig.h:3:12: fatal error: 'x86_64-linux-gnu/python3.8/pyconfig.h' file not found
#  include <x86_64-linux-gnu/python3.8/pyconfig.h>
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
Failed

How to reproduce:

  1. Checkout ydb-platform-ydb to $HOME/ydb
  2. Run the following command: docker run --rm -it -v $HOME/ydb:/ydb ubuntu:focal /bin/bash -c "apt update && apt install -y python3-dev python3.8-dev; /ydb/ya make -DUSE_ARCADIA_PYTHON=no -DUSE_LOCAL_PYTHON -DPYTHON_CONFIG=python3.8-config -DPYTHON_BIN=python3.8 /ydb/ydb/library/yql/udfs/common/python/system_python/python3_8"

We had to implement a workaround because of it: https://github.com/ytsaurus/ytsaurus/pull/793/files#diff-fce526c9dfb5ba1a3e79350a2715244f3414cf4363ee61a8b049d81151cf82e9R29-R41

One other workaround is to use -DOS_SDK=local, but seems like it may do more harm than benefit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant