Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ceph nano crashes when downloading large file #127

Open
nduytg opened this issue Jun 15, 2020 · 1 comment
Open

Ceph nano crashes when downloading large file #127

nduytg opened this issue Jun 15, 2020 · 1 comment

Comments

@nduytg
Copy link

nduytg commented Jun 15, 2020

Hi guys,

I'm encountering an issue. Whenever I download large file (in my case, an 2.5GB video), ceph nano will crash.

I'm using huge flavor now and It works well with small files and text files

How to reproduce this

Start Ceph nano

./cn cluster start my-first-cluster -f huge

Use mc client to set the bucket publicly accessible

mc policy set public mybucket/mybucket

Then download the files from browser directly => Ceph nano crashes

Docker logs

++/opt/ceph-container/bin/docker_exec.sh:149: exec(): ceph --cluster ceph -w
 * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
  cluster:
    id:     6ddca59e-d870-4dea-8260-3530542530d5
    health: HEALTH_OK
 
  services:
    mon: 1 daemons, quorum ceph-nano-my-first-cluster-faa32aebf00b (age 3s)
    mgr: ceph-nano-my-first-cluster-faa32aebf00b(active, since 5m)
    osd: 1 osds: 1 up (since 5m), 1 in (since 4d)
    rgw: 1 daemon active (ceph-nano-my-first-cluster-faa32aebf00b)
 
  task status:
 
  data:
    pools:   9 pools, 209 pgs
    objects: 930 objects, 2.5 GiB
    usage:   3.6 GiB used, 96 GiB / 100 GiB avail
    pgs:     209 active+clean
 
  io:
    client:   26 MiB/s rd, 6 op/s rd, 0 op/s wr
 

2020-06-15T07:52:37.030096+0000 mon.ceph-nano-my-first-cluster-faa32aebf00b [INF] Active manager daemon ceph-nano-my-first-cluster-faa32aebf00b restarted
2020-06-15T07:52:37.042020+0000 mon.ceph-nano-my-first-cluster-faa32aebf00b [INF] Activating manager daemon ceph-nano-my-first-cluster-faa32aebf00b
2020-06-15T07:52:37.123684+0000 mon.ceph-nano-my-first-cluster-faa32aebf00b [INF] Manager daemon ceph-nano-my-first-cluster-faa32aebf00b is now available
+++/opt/ceph-container/bin/docker_exec.sh:1: exec(): _chld
+++/opt/ceph-container/bin/docker_exec.sh:117: _chld(): teardown SIGCHLD 0
+++/opt/ceph-container/bin/docker_exec.sh:34: teardown(): trap - SIGINT SIGILL SIGABRT SIGFPE SIGSEGV SIGTERM SIGBUS SIGCHLD SIGKILL ERR
+++/opt/ceph-container/bin/docker_exec.sh:39: teardown(): local signal_name=SIGCHLD
+++/opt/ceph-container/bin/docker_exec.sh:40: teardown(): local exit_code=0
+++/opt/ceph-container/bin/docker_exec.sh:41: teardown(): echo 'teardown: managing teardown after SIGCHLD'
teardown: managing teardown after SIGCHLD
+++/opt/ceph-container/bin/docker_exec.sh:45: teardown(): '[' SIGCHLD = SIGTERM ']'
+++/opt/ceph-container/bin/docker_exec.sh:52: teardown(): '[' -e /proc/395 ']'
+++/opt/ceph-container/bin/docker_exec.sh:52: teardown(): echo 'teardown: Sending SIGTERM to PID 395'
+++/opt/ceph-container/bin/docker_exec.sh:52: teardown(): kill -TERM 395
teardown: Sending SIGTERM to PID 395
/opt/ceph-container/bin/docker_exec.sh: line 32: warning: run_pending_traps: bad value in trap_list[17]: 0x55f1c1e917b0
+++/opt/ceph-container/bin/docker_exec.sh:54: teardown(): echo -n 'teardown: Waiting PID 395 to terminate '
+++/opt/ceph-container/bin/docker_exec.sh:55: teardown(): local MAX_TIMEOUT=50
+++/opt/ceph-container/bin/docker_exec.sh:56: teardown(): local timeout=0
+++/opt/ceph-container/bin/docker_exec.sh:57: teardown(): '[' -e /proc/395 ']'
+++/opt/ceph-container/bin/docker_exec.sh:68: teardown(): echo
+++/opt/ceph-container/bin/docker_exec.sh:69: teardown(): echo 'teardown: Process 395 is terminated'
+++/opt/ceph-container/bin/docker_exec.sh:71: teardown(): [[ SIGCHLD =~ SIGTERM|SIGCHLD ]]
teardown: Waiting PID 395 to terminate 
teardown: Process 395 is terminated
+++/opt/ceph-container/bin/docker_exec.sh:73: teardown(): declare -F sigterm_cleanup_post
+++/opt/ceph-container/bin/docker_exec.sh:79: teardown(): do_stayalive
+++/opt/ceph-container/bin/docker_exec.sh:7: do_stayalive(): '[' -z '' ']'
+++/opt/ceph-container/bin/docker_exec.sh:8: do_stayalive(): return
teardown: Bye Bye, container will die with return code 0
+++/opt/ceph-container/bin/docker_exec.sh:81: teardown(): echo 'teardown: Bye Bye, container will die with return code 0'
+++/opt/ceph-container/bin/docker_exec.sh:82: teardown(): [[ 0 -ne 0 ]]
+++/opt/ceph-container/bin/docker_exec.sh:85: teardown(): exit 0
@nduytg
Copy link
Author

nduytg commented Jun 15, 2020

I tested again with 600mb file, crashed half-way too

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant