Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cyrcular-calling package error while calling graphs in nanopore sequencing data #20

Open
swcho-HYBigLab opened this issue Aug 19, 2024 · 6 comments
Open

cyrcular-calling package error while calling graphs in nanopore sequencing data #20

swcho-HYBigLab opened this issue Aug 19, 2024 · 6 comments

Comments

@swcho-HYBigLab
Copy link

Hello, I'm a graduated student studying bioinformatics in South Korea.
I'm trying to do detect extrachromosomal DNA (ecDNA) in nanopore long-read targeted sequencing data.

I followed the workflow you described here.

I have tried two times of that workflow, but had same error in my job.

Workflow defines that rule get_genome is eligible for caching between workflows (use the --cache argument to enable this).
Workflow defines that rule genome_faidx is eligible for caching between workflows (use the --cache argument to enable this).
Workflow defines that rule minimap2_index is eligible for caching between workflows (use the --cache argument to enable this).
Workflow defines that rule download_regulatory_annotation is eligible for caching between workflows (use the --cache argument to enable this).
Workflow defines that rule download_repeatmasker_annotation is eligible for caching between workflows (use the --cache argument to enable this).
Workflow defines that rule download_gene_annotation is eligible for caching between workflows (use the --cache argument to enable this).
Assuming unrestricted shared filesystem usage.
Building DAG of jobs...
Your conda installation is not configured to use strict channel priorities. This is however crucial for having robust and correct environments (for details, see https://conda-forge.org/docs/user/tipsandtricks.html). Please consider to configure strict priorities by executing 'conda config --set channel_priority strict'.
Creating conda environment https://raw.githubusercontent.com/snakemake-workflows/cyrcular-calling/v2.1.0/workflow/envs/python.yaml...
Downloading and installing remote packages.
Environment for https://raw.githubusercontent.com/snakemake-workflows/cyrcular-calling/v2.1.0/workflow/envs/python.yaml created (location: .snakemake/conda/f73c94fe37d96f51957feb7810eab8cf_)
Creating conda environment https://github.com/snakemake/snakemake-wrappers/raw/v1.25.0/bio/bcftools/concat/environment.yaml...
Downloading and installing remote packages.
Environment for https://github.com/snakemake/snakemake-wrappers/raw/v1.25.0/bio/bcftools/concat/environment.yaml created (location: .snakemake/conda/cf33438f5c7c9884abf85d29285efe04_)
Creating conda environment https://raw.githubusercontent.com/snakemake-workflows/cyrcular-calling/v2.1.0/workflow/envs/varlociraptor.yaml...
Downloading and installing remote packages.
Environment for https://raw.githubusercontent.com/snakemake-workflows/cyrcular-calling/v2.1.0/workflow/envs/varlociraptor.yaml created (location: .snakemake/conda/2eef35d24a59f83df9961f5f70ab7240_)
Creating conda environment https://raw.githubusercontent.com/snakemake-workflows/cyrcular-calling/v2.1.0/workflow/envs/graphviz.yaml...
Downloading and installing remote packages.
Environment for https://raw.githubusercontent.com/snakemake-workflows/cyrcular-calling/v2.1.0/workflow/envs/graphviz.yaml created (location: .snakemake/conda/359230e4a02075ecdb5b70d4ee000c2b_)
Creating conda environment https://raw.githubusercontent.com/snakemake-workflows/cyrcular-calling/v2.1.0/workflow/envs/bcftools.yaml...
Downloading and installing remote packages.
Environment for https://raw.githubusercontent.com/snakemake-workflows/cyrcular-calling/v2.1.0/workflow/envs/bcftools.yaml created (location: .snakemake/conda/e9f1db15c0d3c38f4c28473ae5cdc670_)
Creating conda environment https://github.com/snakemake/snakemake-wrappers/raw/v1.25.0/bio/minimap2/aligner/environment.yaml...
Downloading and installing remote packages.
Environment for https://github.com/snakemake/snakemake-wrappers/raw/v1.25.0/bio/minimap2/aligner/environment.yaml created (location: .snakemake/conda/760088e00c13c025db1dc43a805800c8_)
Creating conda environment https://github.com/snakemake/snakemake-wrappers/raw/v1.25.0/bio/reference/ensembl-annotation/environment.yaml...
Downloading and installing remote packages.
Environment for https://github.com/snakemake/snakemake-wrappers/raw/v1.25.0/bio/reference/ensembl-annotation/environment.yaml created (location: .snakemake/conda/0fc8bd2d8acee9db91dcc38a4fd1bf12_)
Creating conda environment https://github.com/snakemake/snakemake-wrappers/raw/v1.25.0/bio/minimap2/index/environment.yaml...
Downloading and installing remote packages.
Environment for https://github.com/snakemake/snakemake-wrappers/raw/v1.25.0/bio/minimap2/index/environment.yaml created (location: .snakemake/conda/3609edbf7386a95c393e536852539e9f_)
Creating conda environment https://github.com/snakemake/snakemake-wrappers/raw/v1.25.0/bio/samtools/faidx/environment.yaml...
Downloading and installing remote packages.
Environment for https://github.com/snakemake/snakemake-wrappers/raw/v1.25.0/bio/samtools/faidx/environment.yaml created (location: .snakemake/conda/86ba35beb3d191069d6472b09bda5d32_)
Creating conda environment https://raw.githubusercontent.com/snakemake-workflows/cyrcular-calling/v2.1.0/workflow/envs/cyrcular.yaml...
Downloading and installing remote packages.
Environment for https://raw.githubusercontent.com/snakemake-workflows/cyrcular-calling/v2.1.0/workflow/envs/cyrcular.yaml created (location: .snakemake/conda/28015d0ac6596dc33c1e8186e4f0b356_)
Creating conda environment https://raw.githubusercontent.com/snakemake-workflows/cyrcular-calling/v2.1.0/workflow/envs/datavzrd.yaml...
Downloading and installing remote packages.
Environment for https://raw.githubusercontent.com/snakemake-workflows/cyrcular-calling/v2.1.0/workflow/envs/datavzrd.yaml created (location: .snakemake/conda/ba21bc8117aee787fe0a59947a7e5953_)
Creating conda environment https://raw.githubusercontent.com/snakemake-workflows/cyrcular-calling/v2.1.0/workflow/envs/pandas.yaml...
Downloading and installing remote packages.
Environment for https://raw.githubusercontent.com/snakemake-workflows/cyrcular-calling/v2.1.0/workflow/envs/pandas.yaml created (location: .snakemake/conda/4b444ac939a51643b8ca3c1a7874e512_)
Creating conda environment https://raw.githubusercontent.com/snakemake-workflows/cyrcular-calling/v2.1.0/workflow/envs/pigz.yaml...
Downloading and installing remote packages.
Environment for https://raw.githubusercontent.com/snakemake-workflows/cyrcular-calling/v2.1.0/workflow/envs/pigz.yaml created (location: .snakemake/conda/853ad1c780ed78c680a7892915d39095_)
Creating conda environment https://raw.githubusercontent.com/snakemake-workflows/cyrcular-calling/v2.1.0/workflow/envs/rbt.yaml...
Downloading and installing remote packages.
Environment for https://raw.githubusercontent.com/snakemake-workflows/cyrcular-calling/v2.1.0/workflow/envs/rbt.yaml created (location: .snakemake/conda/28dc58deed4445cc74232de79a311c81_)
Creating conda environment https://github.com/snakemake/snakemake-wrappers/raw/v1.25.0/bio/samtools/index/environment.yaml...
Downloading and installing remote packages.
Environment for https://github.com/snakemake/snakemake-wrappers/raw/v1.25.0/bio/samtools/index/environment.yaml created (location: .snakemake/conda/ca4357ab5cb510b9fda99ce03b644e13_)
Creating conda environment https://raw.githubusercontent.com/snakemake-workflows/cyrcular-calling/v2.1.0/workflow/envs/wget.yaml...
Downloading and installing remote packages.
Environment for https://raw.githubusercontent.com/snakemake-workflows/cyrcular-calling/v2.1.0/workflow/envs/wget.yaml created (location: .snakemake/conda/76d042288def0f75ec9e4fc0e00390ce_)
Using shell: /usr/bin/bash
Provided cores: 40
Rules claiming more threads will be scaled down.
Job stats:
job                                   count
----------------------------------  -------
all                                       1
bcf_index                                12
bcftools_concat                           1
bcftools_sort                            12
circle_bnds                               1
circle_coverage_plot                      1
circle_graph_plots                        1
copy_graph_plots_for_datavzrd             4
copy_qc_plots_for_datavzrd                4
cyrcular_annotate_graph                   1
cyrcular_generate_tables                  1
datavzrd_circle_calls                     1
download_gene_annotation                  1
download_regulatory_annotation            1
download_repeatmasker_annotation          1
filter_overview_table                     1
filter_varlociraptor                      1
genome_faidx                              1
get_bcf_header                            1
get_genome                                1
merge_fastqs                              1
minimap2_bam                              1
minimap2_index                            1
reheader_filtered_bcf                     1
render_datavzrd_config                    1
samtools_index                            1
scatter_candidates                        1
sort_bcf_header                           1
sort_bnd_bcfs                             1
varlociraptor_alignment_properties        1
varlociraptor_call                       12
varlociraptor_preprocess                 12
total                                    82

Select jobs to execute...
Execute 5 jobs...

[Mon Aug 19 16:08:19 2024]
localrule download_regulatory_annotation:
    output: resources/regulatory_annotation.gff3.gz
    log: logs/download_regulatory_annotation.log
    jobid: 14
    reason: Missing output files: resources/regulatory_annotation.gff3.gz
    resources: tmpdir=/tmp

Activating conda environment: .snakemake/conda/76d042288def0f75ec9e4fc0e00390ce_

[Mon Aug 19 16:08:19 2024]
localrule get_genome:
    output: resources/homo_sapiens.GRCh38.107.fasta
    log: logs/get-genome.log
    jobid: 5
    reason: Missing output files: resources/homo_sapiens.GRCh38.107.fasta
    resources: tmpdir=/tmp

[Mon Aug 19 16:08:19 2024]
localrule download_gene_annotation:
    output: resources/gene_annotation.gff3.gz
    log: logs/download_gene_annotation.log
    jobid: 13
    reason: Missing output files: resources/gene_annotation.gff3.gz
    resources: tmpdir=/tmp
[Mon Aug 19 16:08:19 2024]
localrule download_repeatmasker_annotation:
    output: resources/repeat_masker.fa.out.gz
    log: logs/download_repeatmasker_annotation.log
    jobid: 15
    reason: Missing output files: resources/repeat_masker.fa.out.gz
    resources: tmpdir=/tmp

Activating conda environment: .snakemake/conda/76d042288def0f75ec9e4fc0e00390ce_

[Mon Aug 19 16:08:19 2024]
localrule merge_fastqs:
    input: /home/swcho/1_ecDNA/3_data_for_WGS/nanopore_targeted/S23-10097_targeted_long_read_ERBB2.trimmed_Q7.fastq.gz
    output: results/calling/merged/S23-10097_rmdup_single.fastq.gz
    log: logs/merge-fastqs/S23-10097_rmdup_single.log
    jobid: 10
    reason: Missing output files: results/calling/merged/S23-10097_rmdup_single.fastq.gz
    wildcards: sample=S23-10097_rmdup, read=single
    resources: tmpdir=/tmp

Activating conda environment: .snakemake/conda/853ad1c780ed78c680a7892915d39095_
Activating conda environment: .snakemake/conda/0fc8bd2d8acee9db91dcc38a4fd1bf12_
Activating conda environment: .snakemake/conda/0fc8bd2d8acee9db91dcc38a4fd1bf12_
[Mon Aug 19 16:09:05 2024]
Finished job 13.
1 of 82 steps (1%) done
[Mon Aug 19 16:09:20 2024]
Finished job 14.
2 of 82 steps (2%) done
[Mon Aug 19 16:09:38 2024]
Finished job 15.
3 of 82 steps (4%) done
[Mon Aug 19 16:14:29 2024]
Finished job 10.
4 of 82 steps (5%) done
[Mon Aug 19 16:21:23 2024]
Finished job 5.
5 of 82 steps (6%) done
Select jobs to execute...
Execute 2 jobs...

[Mon Aug 19 16:21:24 2024]
localrule genome_faidx:
    input: resources/homo_sapiens.GRCh38.107.fasta
    output: resources/homo_sapiens.GRCh38.107.fasta.fai
    log: logs/genome-faidx.log
    jobid: 12
    reason: Missing output files: resources/homo_sapiens.GRCh38.107.fasta.fai; Input files updated by another job: resources/homo_sapiens.GRCh38.107.fasta
    resources: tmpdir=/tmp

[Mon Aug 19 16:21:24 2024]
localrule minimap2_index:
    input: resources/homo_sapiens.GRCh38.107.fasta
    output: resources/homo_sapiens.GRCh38.107.mmi
    log: logs/minimap2_index/genome.log
    jobid: 9
    reason: Missing output files: resources/homo_sapiens.GRCh38.107.mmi; Input files updated by another job: resources/homo_sapiens.GRCh38.107.fasta
    threads: 3
    resources: tmpdir=/tmp

Activating conda environment: .snakemake/conda/86ba35beb3d191069d6472b09bda5d32_
Activating conda environment: .snakemake/conda/3609edbf7386a95c393e536852539e9f_
Activating conda environment: .snakemake/conda/86ba35beb3d191069d6472b09bda5d32_
[Mon Aug 19 16:21:49 2024]
Finished job 12.
6 of 82 steps (7%) done
[Mon Aug 19 16:23:11 2024]
Finished job 9.
7 of 82 steps (9%) done
Select jobs to execute...
Execute 1 jobs...

[Mon Aug 19 16:23:11 2024]
localrule minimap2_bam:
    input: resources/homo_sapiens.GRCh38.107.mmi, results/calling/merged/S23-10097_rmdup_single.fastq.gz
    output: results/calling/mapping/S23-10097_rmdup.bam
    log: logs/minimap2/S23-10097_rmdup.log
    jobid: 8
    benchmark: benchmarks/minimap2/S23-10097_rmdup.txt
    reason: Missing output files: results/calling/mapping/S23-10097_rmdup.bam; Input files updated by another job: results/calling/merged/S23-10097_rmdup_single.fastq.gz, resources/homo_sapiens.GRCh38.107.mmi
    wildcards: sample=S23-10097_rmdup
    threads: 20
    resources: tmpdir=/tmp

Activating conda environment: .snakemake/conda/760088e00c13c025db1dc43a805800c8_
Activating conda environment: .snakemake/conda/760088e00c13c025db1dc43a805800c8_
[Mon Aug 19 17:24:56 2024]
Finished job 8.
8 of 82 steps (10%) done
Select jobs to execute...
Execute 2 jobs...

[Mon Aug 19 17:24:56 2024]
localrule varlociraptor_alignment_properties:
    input: resources/homo_sapiens.GRCh38.107.fasta, resources/homo_sapiens.GRCh38.107.fasta.fai, results/calling/mapping/S23-10097_rmdup.bam
    output: results/calling/alignment-properties/S23-10097_rmdup.json
    log: logs/varlociraptor/estimate-alignment-properties/S23-10097_rmdup.log
    jobid: 24
    reason: Missing output files: results/calling/alignment-properties/S23-10097_rmdup.json; Input files updated by another job: results/calling/mapping/S23-10097_rmdup.bam, resources/homo_sapiens.GRCh38.107.fasta, resources/homo_sapiens.GRCh38.107.fasta.fai
    wildcards: sample=S23-10097_rmdup
    resources: tmpdir=/tmp

Activating conda environment: .snakemake/conda/2eef35d24a59f83df9961f5f70ab7240_

[Mon Aug 19 17:24:56 2024]
localrule samtools_index:
    input: results/calling/mapping/S23-10097_rmdup.bam
    output: results/calling/mapping/S23-10097_rmdup.bam.bai
    log: logs/samtools_index/S23-10097_rmdup.log
    jobid: 11
    benchmark: benchmarks/samtools_index/S23-10097_rmdup.txt
    reason: Missing output files: results/calling/mapping/S23-10097_rmdup.bam.bai; Input files updated by another job: results/calling/mapping/S23-10097_rmdup.bam
    wildcards: sample=S23-10097_rmdup
    threads: 12
    resources: tmpdir=/tmp

Activating conda environment: .snakemake/conda/ca4357ab5cb510b9fda99ce03b644e13_
[Mon Aug 19 17:25:44 2024]
Finished job 11.
9 of 82 steps (11%) done
Select jobs to execute...
Execute 1 jobs...

[Mon Aug 19 17:25:44 2024]
localrule circle_bnds:
    input: results/calling/mapping/S23-10097_rmdup.bam, results/calling/mapping/S23-10097_rmdup.bam.bai, resources/homo_sapiens.GRCh38.107.fasta, resources/homo_sapiens.GRCh38.107.fasta.fai
    output: results/calling/candidates/S23-10097_rmdup.bcf, results/calling/graphs/S23-10097_rmdup.graph, results/calling/graphs/S23-10097_rmdup
    log: logs/cyrcular/S23-10097_rmdup.log
    jobid: 7
    benchmark: benchmarks/cyrcular/S23-10097_rmdup.txt
    reason: Missing output files: results/calling/candidates/S23-10097_rmdup.bcf, results/calling/graphs/S23-10097_rmdup.graph, results/calling/graphs/S23-10097_rmdup; Input files updated by another job: results/calling/mapping/S23-10097_rmdup.bam, resources/homo_sapiens.GRCh38.107.fasta, results/calling/mapping/S23-10097_rmdup.bam.bai, resources/homo_sapiens.GRCh38.107.fasta.fai
    wildcards: sample=S23-10097_rmdup
    threads: 4
    resources: tmpdir=/tmp

Activating conda environment: .snakemake/conda/28015d0ac6596dc33c1e8186e4f0b356_
[Mon Aug 19 17:25:46 2024]
Error in rule circle_bnds:
    jobid: 7
    input: results/calling/mapping/S23-10097_rmdup.bam, results/calling/mapping/S23-10097_rmdup.bam.bai, resources/homo_sapiens.GRCh38.107.fasta, resources/homo_sapiens.GRCh38.107.fasta.fai
    output: results/calling/candidates/S23-10097_rmdup.bcf, results/calling/graphs/S23-10097_rmdup.graph, results/calling/graphs/S23-10097_rmdup
    log: logs/cyrcular/S23-10097_rmdup.log (check log file(s) for error details)
    conda-env: /lustre/export/home/swcho/1_ecDNA/3_data_for_WGS/nanopore_targeted/cyrcular-calling/.snakemake/conda/28015d0ac6596dc33c1e8186e4f0b356_
    threads: 12
    resources: tmpdir=/tmp

Activating conda environment: .snakemake/conda/ca4357ab5cb510b9fda99ce03b644e13_
[Mon Aug 19 17:25:44 2024]
Finished job 11.
9 of 82 steps (11%) done
Select jobs to execute...
Execute 1 jobs...

[Mon Aug 19 17:25:44 2024]
localrule circle_bnds:
    input: results/calling/mapping/S23-10097_rmdup.bam, results/calling/mapping/S23-10097_rmdup.bam.bai, resources/homo_sapiens.GRCh38.107.fasta, resources/homo_sapiens.GRCh38.107.fasta.fai
    output: results/calling/candidates/S23-10097_rmdup.bcf, results/calling/graphs/S23-10097_rmdup.graph, results/calling/graphs/S23-10097_rmdup
    log: logs/cyrcular/S23-10097_rmdup.log
    jobid: 7
    benchmark: benchmarks/cyrcular/S23-10097_rmdup.txt
    reason: Missing output files: results/calling/candidates/S23-10097_rmdup.bcf, results/calling/graphs/S23-10097_rmdup.graph, results/calling/graphs/S23-10097_rmdup; Input files updated by another job: results/calling/mapping/S23-10097_rmdup.bam, resources/homo_sapiens.GRCh38.107.fasta, results/calling/mapping/S23-10097_rmdup.bam.bai, resources/homo_sapiens.GRCh38.107.fasta.fai
    wildcards: sample=S23-10097_rmdup
    threads: 4
    resources: tmpdir=/tmp

Activating conda environment: .snakemake/conda/28015d0ac6596dc33c1e8186e4f0b356_
[Mon Aug 19 17:25:46 2024]
Error in rule circle_bnds:
    jobid: 7
    input: results/calling/mapping/S23-10097_rmdup.bam, results/calling/mapping/S23-10097_rmdup.bam.bai, resources/homo_sapiens.GRCh38.107.fasta, resources/homo_sapiens.GRCh38.107.fasta.fai
    output: results/calling/candidates/S23-10097_rmdup.bcf, results/calling/graphs/S23-10097_rmdup.graph, results/calling/graphs/S23-10097_rmdup
    log: logs/cyrcular/S23-10097_rmdup.log (check log file(s) for error details)
    conda-env: /lustre/export/home/swcho/1_ecDNA/3_data_for_WGS/nanopore_targeted/cyrcular-calling/.snakemake/conda/28015d0ac6596dc33c1e8186e4f0b356_
    shell:
        cyrcular        graph        breakends        results/calling/mapping/S23-10097_rmdup.bam        --reference resources/homo_sapiens.GRCh38.107.fasta        --min-read-depth 2        --min-split-reads 5        --max-paths-per-component 15        --max-deletion-length 10000        -t 4        --output results/calling/candidates/S23-10097_rmdup.bcf        --graph results/calling/graphs/S23-10097_rmdup.graph        --dot results/calling/graphs/S23-10097_rmdup        2> logs/cyrcular/S23-10097_rmdup.log
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

[Mon Aug 19 17:40:26 2024]
Finished job 24.
10 of 82 steps (12%) done
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2024-08-19T155240.245712.snakemake.log
WorkflowError:
At least one job did not complete successfully.

This is my standard log above

/usr/bin/bash: cyrcular: command not found

and this is the error log which place in logs/cyrcular/

I think it seems like the cyrcular-calling packages isn't installing properly from your github.
Please check my logs and it will be grateful if you answer my question asap.

Thank you.
@tedil
Copy link
Collaborator

tedil commented Aug 19, 2024

Hm yes, perhaps the post-deploy.sh script for cyrcular.yaml does not correctly install the binaries (in the right place). I will have a look, thank you for the report!

@tedil
Copy link
Collaborator

tedil commented Aug 19, 2024

Try again with the newest release of this workflow. If you run into any other issues, please let me know :)

@swcho-HYBigLab
Copy link
Author

Thank you! I will try the release, and let you know as soon as the results come out

@swcho-HYBigLab
Copy link
Author

This is my running command
snakemake --cores 40 --use-conda

I checked the cyrcular was correctly installed, but unfortunately I got the same error :(

[Tue Aug 20 15:32:42 2024]
localrule circle_bnds:
    input: results/calling/mapping/S23-10097.bam, results/calling/mapping/S23-10097.bam.bai, resources/homo_sapiens.GRCh38.107.fasta, resources/homo_sapiens.GRCh38.107.fasta.fai
    output: results/calling/candidates/S23-10097.bcf, results/calling/graphs/S23-10097.graph, results/calling/graphs/S23-10097
    log: logs/cyrcular/S23-10097.log
    jobid: 7
    benchmark: benchmarks/cyrcular/S23-10097.txt
    reason: Missing output files: results/calling/graphs/S23-10097.graph, results/calling/graphs/S23-10097, results/calling/candidates/S23-10097.bcf
    wildcards: sample=S23-10097
    threads: 4
    resources: tmpdir=/tmp

Activating conda environment: .snakemake/conda/91799390de7870b6300fcef12d4e036b_
/usr/bin/bash: line 1: 135306 Killed                  cyrcular graph breakends results/calling/mapping/S23-10097.bam --reference resources/homo_sapiens.GRCh38.107.fasta --min-read-depth 2 --min-split-reads 5 --max-paths-per-component 15 --max-deletion-length 10000 -t 4 --output results/calling/candidates/S23-10097.bcf --graph results/calling/graphs/S23-10097.graph --dot results/calling/graphs/S23-10097 2> logs/cyrcular/S23-10097.log
[Tue Aug 20 17:24:05 2024]
Error in rule circle_bnds:
    jobid: 7
    input: results/calling/mapping/S23-10097.bam, results/calling/mapping/S23-10097.bam.bai, resources/homo_sapiens.GRCh38.107.fasta, resources/homo_sapiens.GRCh38.107.fasta.fai
    output: results/calling/candidates/S23-10097.bcf, results/calling/graphs/S23-10097.graph, results/calling/graphs/S23-10097
    log: logs/cyrcular/S23-10097.log (check log file(s) for error details)
    conda-env: /lustre/export/home/swcho/1_ecDNA/3_data_for_WGS/nanopore_targeted/cyrcular-calling/.snakemake/conda/91799390de7870b6300fcef12d4e036b_
    shell:
        cyrcular        graph        breakends        results/calling/mapping/S23-10097.bam        --reference resources/homo_sapiens.GRCh38.107.fasta        --min-read-depth 2        --min-split-reads 5        --max-paths-per-component 15        --max-deletion-length 10000        -t 4        --output results/calling/candidates/S23-10097.bcf        --graph results/calling/graphs/S23-10097.graph        --dot results/calling/graphs/S23-10097        2> logs/cyrcular/S23-10097.log
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2024-08-20T152938.659126.snakemake.log
WorkflowError:
At least one job did not complete successfully.

And the logs/cyrcular is saved as

mean read depth: 8.3784
Calculating coverage took: 144.033059079s
Found 10574441 split reads
Building split read info
Building split-read info took: 207.717488729s
Step 1: Determining breakpoints (by coverage)
1.338594207s
Step 2: Insert breakpoint nodes
(number of breakpoints: 1953283)
474.314899ms
Step 3: Inserting edges between neighbouring breakpoints
6.720517866s
Step 4: Inserting edges between split-read nodes
5844.616390635s
Step 5: Pruning graph, first removing certain edges, then removing nodes with no edges left (or not part of a circle)
Number of components (pre-pruning): 249646
Number of components (after edge removal): 1040163
Number of components (after node removal #1): 945908
Number of components (after node removal #2): 6994
19.591672393s
Generating breakend events from graph
Number of components: 6994

It seems like there's an error in the process itself (maybe my running command),
I wonder if the allocation of too many cores could be the problem

@tedil
Copy link
Collaborator

tedil commented Aug 20, 2024

It seems like the process is killed, I suspect memory limits. How much memory do you have available / did you request (if on a cluster system or similar)?
(Judging from the cyrcular log, it is almost done, too.)

@swcho-HYBigLab
Copy link
Author

I'm not sure that I'm answering your question correctly,
but I ran the command with --cores 40 on the cluster below

node6: [CPU] 43.2/96 (45.0%) [MEM] 39G/754G (5.2%)

I don't think the process was killed because the cluster ran out of memory or cores,
Maybe should I do it as workflow --cores all without specifying the number of cores?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tedil @swcho-HYBigLab