added Coreograph #49

RobJY · 2024-07-25T21:24:55Z

PR checklist

github-actions · 2024-07-25T21:26:28Z

`nf-core lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit 2d07fad

+| ✅ 177 tests passed       |+
#| ❔   1 tests were ignored |#
!| ❗  25 tests had warnings |!

❗ Test warnings:

nextflow_config - Config manifest.version should end in dev: 2.0.0
readme - README contains the placeholder zenodo.XXXXXXX. This should be replaced with the zenodo doi (after the first release).
pipeline_todos - TODO string in README.md: TODO nf-core:
pipeline_todos - TODO string in README.md: Include a figure that guides the user through the major workflow steps. Many nf-core
pipeline_todos - TODO string in README.md: Fill in short bullet-pointed list of the default steps in the pipeline
pipeline_todos - TODO string in README.md: Describe the minimum required steps to execute the pipeline, e.g. how to prepare samplesheets.
pipeline_todos - TODO string in README.md: update the following command to include all required parameters for a minimal example
pipeline_todos - TODO string in README.md: If applicable, make list of people who have also contributed
pipeline_todos - TODO string in README.md: Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file.
pipeline_todos - TODO string in README.md: Add bibliography of tools and data used in your pipeline
pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
pipeline_todos - TODO string in awsfulltest.yml: You can customise AWS full pipeline tests as required
pipeline_todos - TODO string in ci.yml: You can customise CI pipeline run tests as required
pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
pipeline_todos - TODO string in output.md: Write this documentation describing your workflow's output
pipeline_todos - TODO string in usage.md: Add documentation about anything specific to running your pipeline. For general topics, please point to (and add to) the main nf-core website.
pipeline_todos - TODO string in test_full.config: Specify the paths to your full test data ( on nf-core/test-datasets or directly in repositories, e.g. SRA)
pipeline_todos - TODO string in test_full.config: Give any required params for the test so that command line flags are not needed
pipeline_todos - TODO string in base.config: Check the defaults for all processes
pipeline_todos - TODO string in base.config: Customise requirements for specific processes.
pipeline_todos - TODO string in test.config: Specify the paths to your test data on nf-core/test-datasets
pipeline_todos - TODO string in test.config: Give any required params for the test so that command line flags are not needed
schema_lint - Parameter input not found in schema

❔ Tests ignored:

nextflow_config - Config variable ignored: params.input

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-mcmicro_logo_light.png
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-mcmicro_logo_light.png
files_exist - File found: docs/images/nf-core-mcmicro_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: conf/igenomes.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: modules.json
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: docs/images/nf-core-mcmicro_logo.png
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/NfcoreTemplate.groovy
files_exist - File not found check: lib/Utils.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: lib/WorkflowMain.groovy
files_exist - File not found check: lib/WorkflowMcmicro.groovy
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: Singularity
files_exist - File not found check: lib/nfcore_external_java_deps.jar
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.validationShowHiddenParams
nextflow_config - Config variable found: params.validationSchemaIgnoreParams
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
nextflow_config - nextflow.config contains configuration profile test
nextflow_config - Config default value correct: params.custom_config_version= master
nextflow_config - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Config default value correct: params.max_cpus= 16
nextflow_config - Config default value correct: params.max_memory= 128.GB
nextflow_config - Config default value correct: params.max_time= 240.h
nextflow_config - Config default value correct: params.publish_dir_mode= copy
nextflow_config - Config default value correct: params.max_multiqc_email_size= 25.MB
nextflow_config - Config default value correct: params.validate_params= true
nextflow_config - Config default value correct: params.pipelines_testdata_base_path= https://raw.githubusercontent.com/nf-core/test-datasets/
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/CONTRIBUTING.md matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - .github/workflows/linting.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - assets/nf-core-mcmicro_logo_light.png matches the template
files_unchanged - docs/images/nf-core-mcmicro_logo_light.png matches the template
files_unchanged - docs/images/nf-core-mcmicro_logo_dark.png matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version badge matched config. Badge: 23.04.0, Config: 23.04.0
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (146 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: release-announcements.yml
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: download_pipeline.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: branch.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - assets/multiqc_config.yml found and not ignored.
multiqc_config - assets/multiqc_config.yml contains report_section_order
multiqc_config - assets/multiqc_config.yml contains export_plots
multiqc_config - assets/multiqc_config.yml contains report_comment
multiqc_config - assets/multiqc_config.yml follows the ordering scheme of the minimally required plugins.
multiqc_config - assets/multiqc_config.yml contains a matching 'report_comment'.
multiqc_config - assets/multiqc_config.yml contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
base_config - conf/base.config found and not ignored.
modules_config - conf/modules.config found and not ignored.
modules_config - MULTIQC found in conf/modules.config and Nextflow scripts.
modules_config - BASICPY found in conf/modules.config and Nextflow scripts.
modules_config - ASHLAR found in conf/modules.config and Nextflow scripts.
modules_config - DEEPCELL_MESMER found in conf/modules.config and Nextflow scripts.
modules_config - CELLPOSE found in conf/modules.config and Nextflow scripts.
modules_config - MCQUANT found in conf/modules.config and Nextflow scripts.
modules_config - COREOGRAPH found in conf/modules.config and Nextflow scripts.
modules_config - SCIMAP_MCMICRO found in conf/modules.config and Nextflow scripts.
nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 2.14.1

Run details

nf-core/tools version 2.14.1
Run at 2024-08-05 18:48:37

jmuhlich

I am still running the tests locally to verify the snapshots are stable, but here are some changes to work through in the meantime.

Also I found that on a 4-core machine the nf-test run failed as coreograph is tagged process_medium which requests 6 cores. Still working out the best way to address that.

assets/markers_1_coreograph.csv

assets/samplesheet_1_row_sample_cycle_coreograph.csv

conf/modules.config

nextflow_schema.json

tests/main.nf.test

workflows/mcmicro.nf

tests/main.nf.test

conf/modules.config

RobJY · 2024-08-01T21:21:57Z

Hmm, my check for pixel_size when tma_dearray is true doesn't seem to be catching when I remove pixel_size from our test case. Looking into this now.

I ran a manual test and it seems to work correctly.
The error message when running with nf-test makes it look like it's getting further than the validation though. Please let me know if you think this is a problem.

RobJY · 2024-08-01T21:23:14Z

Should I change the publish directory for the coreograph module to be output/tma_dearray/coreograph?

…d in PR

nextflow_schema.json

subworkflows/local/utils_nfcore_mcmicro_pipeline/main.nf

jmuhlich · 2024-08-05T18:32:44Z

Should I change the publish directory for the coreograph module to be output/tma_dearray/coreograph?

No, I think it should just be tma_dearray. Could you change it to that?

In general, I think the publish directories should only contain the tool name as a sub-directory when we support multiple simultaneous implementations for that tool or in downstream results (so just segmentation at the moment) but that's something we can adjust in another PR. I was unable to find any guidance on this in the nf-core documentation. Have you seen anything?

RobJY · 2024-08-05T18:43:00Z

Yes, I'll make the change to tma_dearray.

That sounds like a good plan. I wasn't sure if we'd be adding other modules besides coreograph for performing dearray, but you're right, we can always change that in a later PR.

RobJY · 2024-08-05T18:43:05Z

Yes, I'll make the change to tma_dearray.

That sounds like a good plan. I wasn't sure if we'd be adding other modules besides coreograph for performing dearray, but you're right, we can always change that in a later PR.

RobJY added 8 commits July 11, 2024 18:07

basic idea seems to be working; needs testing and clean-up

e4cc708

added coreograph test

b137c5e

removed conditional run of mcquant as suggeted in Slack

793ac76

changed coreograph input parameter to tma_dearray as suggested

4976b7a

fixed a couple of coreograph references that I had missed

192a47c

merged in updated dev; looks good, but needs more testing

52d009d

changed coreograph test to use input file from test data repo

53d8f4c

added missing coreograph module; thanks Jeremy!

792970d

RobJY requested a review from jmuhlich July 25, 2024 21:24

jmuhlich reviewed Jul 30, 2024

View reviewed changes

RobJY added 3 commits August 1, 2024 10:41

removed marker and sample sheets used for local testing

917c5c2

fixed tma_dearray description with suggestion from PR

71e7b7c

cleaned up coreograph map with suggestion from PR

cf32e80

jmuhlich requested changes Aug 1, 2024

View reviewed changes

tests/main.nf.test Outdated Show resolved Hide resolved

conf/modules.config Outdated Show resolved Hide resolved

added parameter pixel_size to compute downsampleFactor

b8829f4

RobJY added 2 commits August 1, 2024 17:55

changed coreograph test of mcquant output to summarizeCsv as suggeste…

d828fb8

…d in PR

added default pixel_size to config

b404d68

jmuhlich requested changes Aug 2, 2024

View reviewed changes

nextflow_schema.json Outdated Show resolved Hide resolved

subworkflows/local/utils_nfcore_mcmicro_pipeline/main.nf Outdated Show resolved Hide resolved

subworkflows/local/utils_nfcore_mcmicro_pipeline/main.nf Outdated Show resolved Hide resolved

RobJY added 2 commits August 2, 2024 17:33

updated validation and schema text as suggested in PR

1719c17

added validation test as suggested in PR

a62308b

moved coreograph output to outdir/tma_dearray as suggested in PR

2d07fad

jmuhlich approved these changes Aug 12, 2024

View reviewed changes

jmuhlich merged commit 48f7b23 into nf-core:dev Aug 12, 2024
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

added Coreograph #49

added Coreograph #49

RobJY commented Jul 25, 2024

github-actions bot commented Jul 25, 2024 •

edited

Loading

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

jmuhlich left a comment

RobJY commented Aug 1, 2024 •

edited

Loading

RobJY commented Aug 1, 2024

jmuhlich commented Aug 5, 2024

RobJY commented Aug 5, 2024

RobJY commented Aug 5, 2024

added Coreograph #49

added Coreograph #49

Conversation

RobJY commented Jul 25, 2024

PR checklist

github-actions bot commented Jul 25, 2024 • edited Loading

nf-core lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

jmuhlich left a comment

Choose a reason for hiding this comment

RobJY commented Aug 1, 2024 • edited Loading

RobJY commented Aug 1, 2024

jmuhlich commented Aug 5, 2024

RobJY commented Aug 5, 2024

RobJY commented Aug 5, 2024

github-actions bot commented Jul 25, 2024 •

edited

Loading

`nf-core lint` overall result: Passed ✅ ⚠️

RobJY commented Aug 1, 2024 •

edited

Loading