Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[移动云盘][139云盘]PUT方法上传多个文件时出现空指针或者内存地址错误nil pointer dereference #7262

Open
4 tasks done
jibwf opened this issue Sep 25, 2024 · 10 comments
Labels
bug Something isn't working

Comments

@jibwf
Copy link

jibwf commented Sep 25, 2024

Please make sure of the following things

  • I have read the documentation.
    我已经阅读了文档

  • I'm sure there are no duplicate issues or discussions.
    我确定没有重复的issue或讨论。

  • I'm sure it's due to AList and not something else(such as Network ,Dependencies or Operational).
    我确定是AList的问题,而不是其他原因(例如网络依赖操作)。

  • I'm sure this issue is not fixed in the latest version.
    我确定这个问题在最新版本中没有被修复。

AList Version / AList 版本

v3.37.4

Driver used / 使用的存储驱动

中国移动云盘

Describe the bug / 问题描述

批量上传文件时出现panic. debug log已经采集作为日志附件上传了。

�[31m2024/09/25 09:28:27 [Recovery] 2024/09/25 - 09:28:27 panic recovered:
PUT /mobile/p96f17e2785019958977bb8445983c075-s1e3e9ab4e386d28f12d HTTP/1.1
Host: 192.168.1.2:5246
Accept-Encoding: gzip
Authorization: *
Content-Length: 24880957
Content-Md5: 3h5q/xzn7Xccv4glHJUsnA==
Content-Type: application/x-kopia
User-Agent: MinIO (linux; amd64) minio-go/v7.0.69
X-Amz-Content-Sha256: STREAMING-AWS4-HMAC-SHA256-PAYLOAD
X-Amz-Date: 20240925T012807Z
X-Amz-Decoded-Content-Length: 24846672

runtime error: invalid memory address or nil pointer dereference
/opt/hostedtoolcache/go/1.23.1/x64/src/runtime/panic.go:262 (0x477fb8)

Reproduction / 复现链接

如果需要可以单独提供

Config / 配置

{
"force": false,
"site_url": "",
"cdn": "",
"jwt_secret": "mNIstpFmBVEFR07R",
"token_expires_in": 48,
"database": {
"type": "sqlite3",
"host": "",
"port": 0,
"user": "",
"password": "",
"name": "",
"db_file": "data/data.db",
"table_prefix": "x_",
"ssl_mode": "",
"dsn": ""
},
"meilisearch": {
"host": "http://localhost:7700/",
"api_key": "",
"index_prefix": ""
},
"scheme": {
"address": "0.0.0.0",
"http_port": 5244,
"https_port": -1,
"force_https": false,
"cert_file": "",
"key_file": "",
"unix_file": "",
"unix_file_perm": ""
},
"temp_dir": "data/temp",
"bleve_dir": "data/bleve",
"dist_dir": "",
"log": {
"enable": true,
"name": "data/log/log.log",
"max_size": 10,
"max_backups": 5,
"max_age": 28,
"compress": false
},
"delayed_start": 0,
"max_connections": 0,
"tls_insecure_skip_verify": true,
"tasks": {
"download": {
"workers": 5,
"max_retry": 1,
"task_persistant": true
},
"transfer": {
"workers": 5,
"max_retry": 2,
"task_persistant": true
},
"upload": {
"workers": 5,
"max_retry": 0,
"task_persistant": false
},
"copy": {
"workers": 5,
"max_retry": 2,
"task_persistant": true
}
},
"cors": {
"allow_origins": [
""
],
"allow_methods": [
""
],
"allow_headers": [
"*"
]
},
"s3": {
"enable": true,
"port": 5246,
"ssl": false
}
}

Logs / 日志

�[31m2024/09/25 09:28:27 [Recovery] 2024/09/25 - 09:28:27 panic recovered:
PUT /mobile/p96f17e2785019958977bb8445983c075-s1e3e9ab4e386d28f12d HTTP/1.1
Host: 192.168.1.2:5246
Accept-Encoding: gzip
Authorization: *
Content-Length: 24880957
Content-Md5: 3h5q/xzn7Xccv4glHJUsnA==
Content-Type: application/x-kopia
User-Agent: MinIO (linux; amd64) minio-go/v7.0.69
X-Amz-Content-Sha256: STREAMING-AWS4-HMAC-SHA256-PAYLOAD
X-Amz-Date: 20240925T012807Z
X-Amz-Decoded-Content-Length: 24846672
log-2024-09-25T01-30-27.705.log.zip

@jibwf jibwf added the bug Something isn't working label Sep 25, 2024
@pongfcnkl
Copy link

试试添加本机存储后,用复制功能上传

@jibwf
Copy link
Author

jibwf commented Sep 25, 2024

试试添加本机存储后,用复制功能上传

您的意思是先把文件放到本机存储,然后复制到网盘?

@pongfcnkl
Copy link

试试添加本机存储后,用复制功能上传

您的意思是先把文件放到本机存储,然后复制到网盘?

1 添加本地存储
2 复制

@jibwf
Copy link
Author

jibwf commented Sep 25, 2024

试试添加本机存储后,用复制功能上传

您的意思是先把文件放到本机存储,然后复制到网盘?

1 添加本地存储 2 复制

复制的方式上传没有遇到panic的问题. 不过这样操作,比较难自动化备份. 所以这个问题和139网盘驱动有关吗?还是和主程序fs.go 方法有关系?

之前panic时候的traceback.
�[31m2024/09/25 20:26:30 [Recovery] 2024/09/25 - 20:26:30 panic recovered:
runtime error: invalid memory address or nil pointer dereference
/opt/hostedtoolcache/go/1.23.1/x64/src/runtime/panic.go:262 (0x477fb8)
/opt/hostedtoolcache/go/1.23.1/x64/src/runtime/signal_unix.go:900 (0x477f88)
/opt/hostedtoolcache/go/1.23.1/x64/src/internal/bytealg/equal_amd64.s:114 (0x405b76)
/home/runner/work/alist/alist/internal/op/fs.go:222 (0x9882b4)
/home/runner/work/alist/alist/internal/op/fs.go:231 (0x9888b2)
/home/runner/work/alist/alist/internal/op/fs.go:518 (0x98bfaa)
/home/runner/work/alist/alist/internal/fs/put.go:70 (0xe33fb0)
/home/runner/work/alist/alist/internal/fs/fs.go:97 (0xe321ae)
/home/runner/work/alist/alist/server/s3/backend.go:325 (0x1ccd32a)
/home/runner/go/pkg/mod/github.com/alist-org/gofakes3@v0.0.7/gofakes3.go:728 (0x1cb9032)
/home/runner/go/pkg/mod/github.com/alist-org/gofakes3@v0.0.7/routing.go:88 (0x1cc135a)
/home/runner/go/pkg/mod/github.com/alist-org/gofakes3@v0.0.7/routing.go:61 (0x1cc1152)
/opt/hostedtoolcache/go/1.23.1/x64/src/net/http/server.go:2220 (0x790908)
/home/runner/go/pkg/mod/github.com/alist-org/gofakes3@v0.0.7/cors.go:45 (0x1cb23d7)
/home/runner/go/pkg/mod/github.com/alist-org/gofakes3@v0.0.7/gofakes3.go:163 (0x1cb3d53)
/opt/hostedtoolcache/go/1.23.1/x64/src/net/http/server.go:2220 (0x790908)
/home/runner/go/pkg/mod/github.com/alist-org/gofakes3@v0.0.7/gofakes3.go:144 (0x1cb36f2)
/opt/hostedtoolcache/go/1.23.1/x64/src/net/http/server.go:2220 (0x790908)
/home/runner/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/utils.go:49 (0x1cf792a)
/home/runner/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 (0xe1f7ee)
/home/runner/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/recovery.go:102 (0xe1f7db)
/home/runner/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 (0xe1e924)
/home/runner/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/logger.go:249 (0xe1e90b)
/home/runner/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 (0xe1dd11)
/home/runner/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/gin.go:633 (0xe1d780)
/home/runner/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/gin.go:589 (0xe1d2b1)
/opt/hostedtoolcache/go/1.23.1/x64/src/net/http/server.go:3210 (0x7afd8d)
/opt/hostedtoolcache/go/1.23.1/x64/src/net/http/server.go:2092 (0x78f2af)
/opt/hostedtoolcache/go/1.23.1/x64/src/runtime/asm_amd64.s:1700 (0x47e5a0)
�[0m
[GIN] 2024/09/25 - 20:26:30 | 500 | 51.071551543s | 172.17.0.1 | PUT "/mobile/pbce9e795bfb82a679892633a906fa188-sefdfd621c4f20b3412d"
2024/09/25 20:26:36 INFO worker execute task worker=1 task=IozDyXhs2SrEMXpLiJhSh

@jibwf
Copy link
Author

jibwf commented Sep 25, 2024

我限制了同时上传2个,也还是会panic

@pongfcnkl
Copy link

pongfcnkl commented Sep 25, 2024

试试添加本机存储后,用复制功能上传

您的意思是先把文件放到本机存储,然后复制到网盘?

1 添加本地存储 2 复制

复制的方式上传没有遇到panic的问题. 不过这样操作,比较难自动化备份. 所以这个问题和139网盘驱动有关吗?还是和主程序fs.go 方法有关系?

之前panic时候的traceback. �[31m2024/09/25 20:26:30 [Recovery] 2024/09/25 - 20:26:30 panic recovered: runtime error: invalid memory address or nil pointer dereference /opt/hostedtoolcache/go/1.23.1/x64/src/runtime/panic.go:262 (0x477fb8) /opt/hostedtoolcache/go/1.23.1/x64/src/runtime/signal_unix.go:900 (0x477f88) /opt/hostedtoolcache/go/1.23.1/x64/src/internal/bytealg/equal_amd64.s:114 (0x405b76) /home/runner/work/alist/alist/internal/op/fs.go:222 (0x9882b4) /home/runner/work/alist/alist/internal/op/fs.go:231 (0x9888b2) /home/runner/work/alist/alist/internal/op/fs.go:518 (0x98bfaa) /home/runner/work/alist/alist/internal/fs/put.go:70 (0xe33fb0) /home/runner/work/alist/alist/internal/fs/fs.go:97 (0xe321ae) /home/runner/work/alist/alist/server/s3/backend.go:325 (0x1ccd32a) /home/runner/go/pkg/mod/github.com/alist-org/gofakes3@v0.0.7/gofakes3.go:728 (0x1cb9032) /home/runner/go/pkg/mod/github.com/alist-org/gofakes3@v0.0.7/routing.go:88 (0x1cc135a) /home/runner/go/pkg/mod/github.com/alist-org/gofakes3@v0.0.7/routing.go:61 (0x1cc1152) /opt/hostedtoolcache/go/1.23.1/x64/src/net/http/server.go:2220 (0x790908) /home/runner/go/pkg/mod/github.com/alist-org/gofakes3@v0.0.7/cors.go:45 (0x1cb23d7) /home/runner/go/pkg/mod/github.com/alist-org/gofakes3@v0.0.7/gofakes3.go:163 (0x1cb3d53) /opt/hostedtoolcache/go/1.23.1/x64/src/net/http/server.go:2220 (0x790908) /home/runner/go/pkg/mod/github.com/alist-org/gofakes3@v0.0.7/gofakes3.go:144 (0x1cb36f2) /opt/hostedtoolcache/go/1.23.1/x64/src/net/http/server.go:2220 (0x790908) /home/runner/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/utils.go:49 (0x1cf792a) /home/runner/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 (0xe1f7ee) /home/runner/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/recovery.go:102 (0xe1f7db) /home/runner/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 (0xe1e924) /home/runner/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/logger.go:249 (0xe1e90b) /home/runner/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 (0xe1dd11) /home/runner/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/gin.go:633 (0xe1d780) /home/runner/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/gin.go:589 (0xe1d2b1) /opt/hostedtoolcache/go/1.23.1/x64/src/net/http/server.go:3210 (0x7afd8d) /opt/hostedtoolcache/go/1.23.1/x64/src/net/http/server.go:2092 (0x78f2af) /opt/hostedtoolcache/go/1.23.1/x64/src/runtime/asm_amd64.s:1700 (0x47e5a0) �[0m [GIN] 2024/09/25 - 20:26:30 | 500 | 51.071551543s | 172.17.0.1 | PUT "/mobile/pbce9e795bfb82a679892633a906fa188-sefdfd621c4f20b3412d" 2024/09/25 20:26:36 INFO worker execute task worker=1 task=IozDyXhs2SrEMXpLiJhSh

你可以用rclone进行自动化操作,无非就是把put换成命令。
rclone用相同的配置名称就是复制了,比如rclone -P copy alist:本地 alist:139 ...

除非你的文件和alist不在一个地方,否则建议走复制,如果不在一个地方,可以考虑搭建在有文件的设备上,然后在走复制。

@jibwf
Copy link
Author

jibwf commented Sep 25, 2024

明白了感谢。之前考虑过用rclone,但是我这个应用是kopia备份,它对rclone的支持也是实验性的,所以就没用rclone,想减少一层.
希望有大佬能看看修复一下

@pongfcnkl
Copy link

明白了感谢。之前考虑过用rclone,但是我这个应用是kopia备份,它对rclone的支持也是实验性的,所以就没用rclone,想减少一层. 希望有大佬能看看修复一下

其实rclone的同名称复制对alist来说才是减少一层。
另外可以先用kopia备份到本地,然后在用rclone复制到云盘

@jibwf
Copy link
Author

jibwf commented Sep 25, 2024

@pongfcnkl
其实rclone的同名称复制对alist来说才是减少一层。 能不能详细说一下,我之前理解rclone就是通过webdav方法连接alist使用网盘和我直接用alist webdav方法有区别吗?

@pongfcnkl
Copy link

pongfcnkl commented Sep 25, 2024

@pongfcnkl 其实rclone的同名称复制对alist来说才是减少一层。 能不能详细说一下,我之前理解rclone就是通过webdav方法连接alist使用网盘和我直接用alist webdav方法有区别吗?

意思是用相同的webdav配置名称进行copy复制,这样就和你网页上复制一样了
下面这个相当于网页上的复制
rclone -P copy alist:本机存储路径 alist:139 ...
下面这个就和你平常用webdav没区别了
rclone -P copy 本地路径 alist:139 ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants