Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added multiple sequencing methods #47

Merged
merged 19 commits into from
Jul 19, 2024
Merged

Added multiple sequencing methods #47

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
6e1692c
almost there; everything working except multisample input
RobJY Jul 8, 2024
6223584
works, but needs clean-up
RobJY Jul 9, 2024
806b734
clean-up
RobJY Jul 9, 2024
81a2bea
added multiple segmentation test
RobJY Jul 9, 2024
0c9967d
set segmentation default to mesmer to pass test suite
RobJY Jul 9, 2024
88b883b
added infrastructure for cellpose models
RobJY Jul 10, 2024
1978913
fixed lint error; added cellpose_model param to config
RobJY Jul 10, 2024
bd36782
typo fix
RobJY Jul 12, 2024
729e2ee
added regex for segmentation input as suggested in PR
RobJY Jul 15, 2024
60167ff
added fix for non-root container user from PR suggestion
RobJY Jul 15, 2024
e7fd5c1
removed segmentation condition and replaced with ext.when as suggeste…
RobJY Jul 15, 2024
faf3ef9
added named vars in mutiMaps as suggested in PR
RobJY Jul 15, 2024
b8da482
cleaned-up adding segmenter to meta
RobJY Jul 15, 2024
145ac24
cleaned up MCQUANT workflow command with the \suggestion from the PR
RobJY Jul 15, 2024
6ee95ee
forgot to add cellpose.diff when I made the patch
RobJY Jul 15, 2024
9e43853
ran prettier to fix formatting
RobJY Jul 15, 2024
f2141bb
fixed duplicate ch_versions
RobJY Jul 18, 2024
76d8ba4
cleaned up segmentation commands from suggestion in PR
RobJY Jul 18, 2024
9d187b3
changed cellpose test to not use predefined model
RobJY Jul 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion conf/modules.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,15 +47,24 @@ process {
withName: "DEEPCELL_MESMER" {
ext.prefix = { "mask_${meta.id}" }
ext.args = '--image-mpp=0.215 --nuclear-channel 0 --compartment nuclear'
ext.when = {params.segmentation && params.segmentation.split(',').contains('mesmer')}
publishDir = [
path: { "${params.outdir}/segmentation/deepcell_mesmer" },
mode: params.publish_dir_mode,
]
}

withName: "CELLPOSE" {
ext.when = {params.segmentation && params.segmentation.split(',').contains('cellpose')}
publishDir = [
path: { "${params.outdir}/segmentation/cellpose" },
mode: params.publish_dir_mode,
]
}

withName: MCQUANT {
publishDir = [
path: { "${params.outdir}/quantification/mcquant" },
path: { "${params.outdir}/quantification/mcquant/${meta2.segmenter}" },
mode: params.publish_dir_mode,
]
}
Expand Down
6 changes: 6 additions & 0 deletions modules.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,12 @@
"cellpose": {
"branch": "master",
"git_sha": "716ef3019b66772a817b417078edce2f7b337858",
"installed_by": ["modules"],
"patch": "modules/nf-core/cellpose/cellpose.diff"
},
"coreograph": {
"branch": "master",
"git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
"installed_by": ["modules"]
},
"deepcell/mesmer": {
Expand Down
15 changes: 15 additions & 0 deletions modules/nf-core/cellpose/cellpose.diff
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions modules/nf-core/cellpose/main.nf
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions nextflow.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ params {
input_sample = null
input_cycle = null
marker_sheet = null
segmentation = 'mesmer'
cellpose_model = []

// Illumination correction
illumination = null
Expand Down
9 changes: 9 additions & 0 deletions nextflow_schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,15 @@
"type": "string",
"enum": ["basicpy"],
"description": "input that defines type of illumination correction to be performed"
},
"segmentation": {
RobJY marked this conversation as resolved.
Show resolved Hide resolved
"type": "string",
"pattern": "^((cellpose|mesmer)?,?)*(?<!,)$",
"description": "comma separated list of segmentation modules to run. options are ['mesmer','cellpose']"
},
"cellpose_model": {
"type": "string",
"description": "optional model file for cellpose segmentation"
}
}
},
Expand Down
5 changes: 5 additions & 0 deletions subworkflows/local/utils_nfcore_mcmicro_pipeline/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,11 @@ def validateInputParameters() {
} else if(!params.input_sample && !params.input_cycle) {
error "You must specify either input_sample or input_cycle."
}

if (params.cellpose_model && !segmentation_list.contains('cellpose')) {
error "You can only provide a cellpose model if you have selected cellpose as one of your segmentation methods"
}

}

//
Expand Down
145 changes: 136 additions & 9 deletions tests/main.nf.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ nextflow_workflow {
test("cycle: no illumination correction") {

when {
params {
segmentation = "mesmer"
}
workflow {
"""
input[0] = Channel.of(
Expand Down Expand Up @@ -37,7 +40,7 @@ nextflow_workflow {
assert snapshot (
path("$outputDir/registration/ashlar/TEST1.ome.tif"),
path("$outputDir/segmentation/deepcell_mesmer/mask_TEST1.tif"),
CsvUtils.roundAndHashCsv("$outputDir/quantification/mcquant/TEST1_mask_TEST1.csv"),
CsvUtils.roundAndHashCsv("$outputDir/quantification/mcquant/mesmer/TEST1_mask_TEST1.csv"),
).match()
},
{ assert workflow.success }
Expand All @@ -51,6 +54,7 @@ nextflow_workflow {
when {
params {
illumination = "basicpy"
segmentation = "mesmer"
}
workflow {
"""
Expand Down Expand Up @@ -80,7 +84,7 @@ nextflow_workflow {
assert snapshot (
ImageUtils.getImageMetadata("$outputDir/registration/ashlar/TEST1.ome.tif"),
ImageUtils.getImageMetadata("$outputDir/segmentation/deepcell_mesmer/mask_TEST1.tif"),
CsvUtils.summarizeCsv("$outputDir/quantification/mcquant/TEST1_mask_TEST1.csv"),
CsvUtils.summarizeCsv("$outputDir/quantification/mcquant/mesmer/TEST1_mask_TEST1.csv"),
ImageUtils.getImageMetadata("$outputDir/illumination_correction/basicpy/cycif-tonsil-cycle1.ome-dfp.tiff"),
ImageUtils.getImageMetadata("$outputDir/illumination_correction/basicpy/cycif-tonsil-cycle1.ome-ffp.tiff")
).match()
Expand All @@ -95,6 +99,7 @@ nextflow_workflow {
when {
params {
illumination = "manual"
segmentation = "mesmer"
}
workflow {
"""
Expand Down Expand Up @@ -124,7 +129,7 @@ nextflow_workflow {
assert snapshot (
path("$outputDir/registration/ashlar/TEST1.ome.tif"),
path("$outputDir/segmentation/deepcell_mesmer/mask_TEST1.tif"),
CsvUtils.roundAndHashCsv("$outputDir/quantification/mcquant/TEST1_mask_TEST1.csv")
CsvUtils.roundAndHashCsv("$outputDir/quantification/mcquant/mesmer/TEST1_mask_TEST1.csv")
).match()
},
{ assert workflow.success }
Expand All @@ -135,6 +140,9 @@ nextflow_workflow {
test("cycle: multiple file ashlar input no correction") {

when {
params {
segmentation = "mesmer"
}
workflow {
"""
input[0] = Channel.of(
Expand Down Expand Up @@ -183,7 +191,7 @@ nextflow_workflow {
assert snapshot (
path("$outputDir/registration/ashlar/cycif-tonsil.ome.tif"),
path("$outputDir/segmentation/deepcell_mesmer/mask_cycif-tonsil.tif"),
CsvUtils.roundAndHashCsv("$outputDir/quantification/mcquant/cycif-tonsil_mask_cycif-tonsil.csv")
CsvUtils.roundAndHashCsv("$outputDir/quantification/mcquant/mesmer/cycif-tonsil_mask_cycif-tonsil.csv")
).match()
},
{ assert workflow.success }
Expand All @@ -194,6 +202,76 @@ nextflow_workflow {
test("cycle: multiple file ashlar input with multiple samples no correction") {

when {
params {
segmentation = "mesmer"
}
workflow {
"""
input[0] = Channel.of(
[
[id:"cycif-tonsil", cycle_number:1, channel_count:4],
"https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/imaging/ome-tiff/cycif-tonsil-cycle1.ome.tif",
[],
[],
],
[
[id:"cycif-tonsil", cycle_number:2, channel_count:4],
"https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/imaging/ome-tiff/cycif-tonsil-cycle2.ome.tif",
[],
[],
],
[
[id:"cycif-tonsil2", cycle_number:1, channel_count:4],
"https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/imaging/ome-tiff/cycif-tonsil-cycle2.ome.tif",
[],
[],
],
[
[id:"cycif-tonsil2", cycle_number:2, channel_count:4],
"https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/imaging/ome-tiff/cycif-tonsil-cycle3.ome.tif",
[],
[],
],
)
input[1] = Channel.of(
[
[1,1,'DNA_6',[],[],[]],
[2,1,'ELANE',[],[],[]],
[3,1,'CD57',[],[],[]],
[4,1,'CD45',[],[],[]],
[5,1,'DNA_7',[],[],[]],
[6,1,'ELANE7',[],[],[]],
[7,1,'CD577',[],[],[]],
[8,1,'CD457',[],[],[]],
],
)
"""
}
}

then {
assertAll (
{
assert snapshot (
path("$outputDir/registration/ashlar/cycif-tonsil.ome.tif"),
path("$outputDir/registration/ashlar/cycif-tonsil2.ome.tif"),
path("$outputDir/segmentation/deepcell_mesmer/mask_cycif-tonsil.tif"),
path("$outputDir/segmentation/deepcell_mesmer/mask_cycif-tonsil2.tif"),
CsvUtils.roundAndHashCsv("$outputDir/quantification/mcquant/mesmer/cycif-tonsil_mask_cycif-tonsil.csv"),
CsvUtils.roundAndHashCsv("$outputDir/quantification/mcquant/mesmer/cycif-tonsil2_mask_cycif-tonsil2.csv"),
).match()
},
{ assert workflow.success }
)
}
},

test("cycle: multiple file ashlar input with multiple samples no correction, multiple segmentation") {

when {
params {
segmentation = "mesmer,cellpose"
}
workflow {
"""
input[0] = Channel.of(
Expand Down Expand Up @@ -246,8 +324,12 @@ nextflow_workflow {
path("$outputDir/registration/ashlar/cycif-tonsil2.ome.tif"),
path("$outputDir/segmentation/deepcell_mesmer/mask_cycif-tonsil.tif"),
path("$outputDir/segmentation/deepcell_mesmer/mask_cycif-tonsil2.tif"),
CsvUtils.roundAndHashCsv("$outputDir/quantification/mcquant/cycif-tonsil_mask_cycif-tonsil.csv"),
CsvUtils.roundAndHashCsv("$outputDir/quantification/mcquant/cycif-tonsil2_mask_cycif-tonsil2.csv"),
path("$outputDir/segmentation/cellpose/cycif-tonsil.ome_cp_masks.tif"),
path("$outputDir/segmentation/cellpose/cycif-tonsil2.ome_cp_masks.tif"),
CsvUtils.roundAndHashCsv("$outputDir/quantification/mcquant/mesmer/cycif-tonsil_mask_cycif-tonsil.csv"),
CsvUtils.roundAndHashCsv("$outputDir/quantification/mcquant/mesmer/cycif-tonsil2_mask_cycif-tonsil2.csv"),
CsvUtils.roundAndHashCsv("$outputDir/quantification/mcquant/cellpose/cycif-tonsil_cycif-tonsil.csv"),
CsvUtils.roundAndHashCsv("$outputDir/quantification/mcquant/cellpose/cycif-tonsil2_cycif-tonsil2.csv"),
).match()
},
{ assert workflow.success }
Expand All @@ -260,6 +342,7 @@ nextflow_workflow {
when {
params {
illumination = "basicpy"
segmentation = "mesmer"
}
workflow {
"""
Expand Down Expand Up @@ -309,7 +392,7 @@ nextflow_workflow {
assert snapshot (
ImageUtils.getImageMetadata("$outputDir/registration/ashlar/cycif-tonsil.ome.tif"),
ImageUtils.getImageMetadata("$outputDir/segmentation/deepcell_mesmer/mask_cycif-tonsil.tif"),
CsvUtils.summarizeCsv("$outputDir/quantification/mcquant/cycif-tonsil_mask_cycif-tonsil.csv"),
CsvUtils.summarizeCsv("$outputDir/quantification/mcquant/mesmer/cycif-tonsil_mask_cycif-tonsil.csv"),
ImageUtils.getImageMetadata("$outputDir/illumination_correction/basicpy/cycif-tonsil-cycle1.ome-dfp.tiff"),
ImageUtils.getImageMetadata("$outputDir/illumination_correction/basicpy/cycif-tonsil-cycle1.ome-ffp.tiff"),
ImageUtils.getImageMetadata("$outputDir/illumination_correction/basicpy/cycif-tonsil-cycle2.ome-dfp.tiff"),
Expand All @@ -328,6 +411,7 @@ nextflow_workflow {
when {
params {
illumination = "basicpy"
segmentation = "mesmer"
}
workflow {
"""
Expand Down Expand Up @@ -381,8 +465,8 @@ nextflow_workflow {
ImageUtils.getImageMetadata("$outputDir/registration/ashlar/cycif-tonsil2.ome.tif"),
ImageUtils.getImageMetadata("$outputDir/segmentation/deepcell_mesmer/mask_cycif-tonsil.tif"),
path("$outputDir/segmentation/deepcell_mesmer/mask_cycif-tonsil2.tif"),
CsvUtils.summarizeCsv("$outputDir/quantification/mcquant/cycif-tonsil_mask_cycif-tonsil.csv"),
CsvUtils.summarizeCsv("$outputDir/quantification/mcquant/cycif-tonsil2_mask_cycif-tonsil2.csv"),
CsvUtils.summarizeCsv("$outputDir/quantification/mcquant/mesmer/cycif-tonsil_mask_cycif-tonsil.csv"),
CsvUtils.summarizeCsv("$outputDir/quantification/mcquant/mesmer/cycif-tonsil2_mask_cycif-tonsil2.csv"),
ImageUtils.getImageMetadata("$outputDir/illumination_correction/basicpy/cycif-tonsil-cycle1.ome-dfp.tiff"),
ImageUtils.getImageMetadata("$outputDir/illumination_correction/basicpy/cycif-tonsil-cycle1.ome-ffp.tiff"),
ImageUtils.getImageMetadata("$outputDir/illumination_correction/basicpy/cycif-tonsil-cycle2.ome-dfp.tiff"),
Expand All @@ -396,5 +480,48 @@ nextflow_workflow {
}
},

test("cycle: no illumination correction, cellpose segmentation") {

when {
params {
segmentation = "cellpose"
}
workflow {
"""
input[0] = Channel.of(
[
[id:"TEST1", cycle_number:1, channel_count:4],
"https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/imaging/ome-tiff/cycif-tonsil-cycle1.ome.tif",
[],
[],
],
)
input[1] = Channel.of(
[
[1,1,'DNA_6',[],[],[]],
[2,1,'ELANE',[],[],[]],
[3,1,'CD57',[],[],[]],
[4,1,'CD45',[],[],[]],
],
)
"""
}
}

then {
assertAll (
{
assert snapshot (
path("$outputDir/registration/ashlar/TEST1.ome.tif"),
path("$outputDir/segmentation/cellpose/TEST1.ome_cp_masks.tif"),
CsvUtils.roundAndHashCsv("$outputDir/quantification/mcquant/cellpose/TEST1_TEST1.csv"),
).match()
},
{ assert workflow.success }
)
}

},

]
}
Loading