DeepSpeed windows install errors #6673
Comments
|
need change code in deepspeed/init.py file from . import ops
from . import module_injectto from deepspeed import ops
from deepspeed import module_injectthen def is_nfs_path(path):
# Normalize the path to get the absolute path
path = os.path.abspath(path)
if platform.system() == "Windows":
# 在 Windows 上不执行 df 命令，直接返回 False
return False
else:
# Use the 'df' command to find the file system type for the given path
try:
output = subprocess.check_output(['df', '-T', path], encoding='utf-8')
except subprocess.CalledProcessError:
return False # Command failed
# Process the output of 'df -T' to check for 'nfs' in the filesystem type column
lines = output.strip().split('\n')
if len(lines) > 1: # The first line is headers
fs_type = lines[1].split()[1].lower() # File system type is the second column
return 'nfs' in fs_type
return Falsethen new error then # def put(self, table):
# if self.file_path:
# assert self.lock_path is not None
# with FileLock(self.lock_path):
# with open(self.file_path + ".tmp", 'wb') as handle:
# pickle.dump(table, handle)
# os.rename(self.file_path + ".tmp", self.file_path)
def put(self, table):
if self.file_path:
assert self.lock_path is not None
with FileLock(self.lock_path):
with open(self.file_path + ".tmp", 'wb') as handle:
pickle.dump(table, handle)
if not os.path.exists(self.file_path):
os.rename(self.file_path + ".tmp", self.file_path)
else:
os.remove(self.file_path + ".tmp") # 删除临时文件then run .\build_win.bat then add cl.exe to env path @loadams |
|
@xiezhipeng-git - For the reason why installing the whl is failing, can you please share the python version that you are using as well? The reason you are getting the "unsupported on this platform" error is because the whl must be built with a specific python/cuda, and if you aren't using the versions we used to build it, you will get this error. |
loadams
changed the title
|
@loadams |
|
@xiezhipeng-git - are you trying to install the whl or install from source? Since your original post mentions installing the whl. If you want to do that, the whl is built with python 3.11 only, this means you need switch to python 3.11 to use it. We are working on getting more python whls published, but for now we only have one supporting 3.11, that's why you got this error. Switching to python 3.11 will resolve this.
|
|
why python 3.10 is not supported @loadams ? it is still massively used even kaggle google colab still python 3.10 |
|
@FurkanGozukara - python 3.10 is supported for building from source. We haven't published a Windows whl with that version yet since we need to support a large matrix of cuda/torch/python versions but we hope to publish a 3.10 whl soon. |
It might be supported, but it does not build torch==2.3.1+cu118 |
|
Note - the link error happens because a cygwin installation can interfere with MSVC which can be seen by doing "where.exe link" and checking if there is a non MSVC result on top. The fix should be to modify path such that MSVC comes before cygwin/conda. Also, in order to compile this on Windows you will need the correct MSVC + CUDA + PyTorch. |
|
Hi @rsxdalv - that's correct, the correct MSVC + CUDA + PyTorch is very important. And having cygwin installed makes sense that could interfere. Were you able to get it to build after you modified the path to MSVC to be first? |
|
Yes
…On Fri, Nov 15, 2024, 10:41 PM Logan Adams ***@***.***> wrote:
Hi @rsxdalv <https://github.com/rsxdalv> - that's correct, the correct
MSVC + CUDA + PyTorch is very important. And having cygwin installed makes
sense that could interfere. Were you able to get it to build after you
modified the path to MSVC to be first?
—
Reply to this email directly, view it on GitHub
<#6673 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABTRXI7VHIP4IEOGH2OID2L2AZL7XAVCNFSM6AAAAABQVPSWM6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINZZHA4DOOJXHE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
os:windows cpu:13900ks and gpu:4090 torch:2.5.0-cuda
first try pip install with windows whl
get
ERROR: deepspeed-0.15.0-cp311-cp311-win_amd64.whl is not a supported wheel on this platform.
then try installl with source code
@loadams @jomayeri
Originally posted by @xiezhipeng-git in #6625
The text was updated successfully, but these errors were encountered: