readfish targets default parameter value of throttle & readfish stats

[jennieli421]

1. For readfish targets, I'm confused by what is the default value of throttle. In [_cli_args.py], there is a line DEFAULT_THROTTLE = 0.4. but in [targets.py], the value is set to 0.1 in init: throttle: float = 0.1. Which one is the value being used? Which should I change if I want to tweak this value?

2. Please let me know if there is detailed documentation on using readfish stats as I was not able to find useful descriptions. After running readfish stats with input of a fastq directory and toml (split into control and analysis region), I got 6 different fastq.gz files. For analysis and control, each has three types proceed, stop_receiving, unblock. Does that mean all these reads are truncated into chunks sent for decision-making? If I only want to know which reads are eventually proceed or unblocked, what should I do?

[jennieli421]

Apologies for a mistake. I thought that throttle is equivalent to break_reads_after_seconds in the older version. the chunk size is what I want to tweak. Please guide me on how to do that. Thanks

[alexomics]

For 1:

The throttle that will be used is 0.4 seconds as it is overridden by the command line using the --throttle option. We recommend that this be set to the same value as the break_reads_after_seconds value; this value is an internal setting for MinKNOW and must be set in the sequencing configuration script which will be called something like: sequencing_PRO114HD_DNA_e8_2_400K.toml.

For 2: @Adoni5 should be able to help.

[jennieli421]

A third question regarding target position file format. In the old version we used txt file. Now it was suggested to use csv. But when I test using a txt file it still seems to work. Is there any issue with input of txt file?

[alexomics]

That is just us choosing to be a bit more specific in the terminology that we use. The txt and csv formats are the same.

[jennieli421]

I see! Is there any difference in terms of mapping speed if I use txt vs bed?

[alexomics]

No, they’re converted to the same structure under the hood so they should perform exactly the same

[jennieli421]

Follow up regarding how to split and config a control region. To ensure that control is not enriching for reads, I set all types of result "stop_receiving". Is this necessary? Should I provide the same targets as in analysis or leave it empty?

[[regions]]
name = "control"
control = true
min_chunks = 1
max_chunks = 2 
targets = [] 
single_on = "stop_receiving"  
multi_on = "stop_receiving"   
single_off = "stop_receiving"        
multi_off = "stop_receiving"      
no_seq = "stop_receiving"            
no_map = "stop_receiving"           
above_max_chunks = "stop_receiving"
below_min_chunks = "stop_receiving"  

[alexomics]

Control regions are always sent a stop_receiving action, but we do currently require that all the result types are populated, so this is exactly how I would set this up.

[Adoni5]

2. Please let me know if there is detailed documentation on using readfish stats as I was not able to find useful descriptions.

Assuming you have seen - https://looselab.github.io/readfish/readfish.console.html#module-readfish.entry_points.stats

There is a some missing documentation on this page, will get that sorted for the next release!

After running readfish stats with input of a fastq directory and toml (split into control and analysis region), I got 6 different fastq.gz files. For analysis and control, each has three types proceed, stop_receiving, unblock. Does that mean all these reads are truncated into chunks sent for decision-making?

We don't currently truncate the reads when we do stats, we align the full read.

If I only want to know which reads are eventually proceed or unblocked, what should I do?

The files are named as such - all reads that would have been proceed or unblock (for each condition) are found in *.proceed.fastq.gz and *.unblock.fastq.gz respectively. I'm not sure what you want mean when you say that you want to know which reads are assigned to each decision beyond suggesting you look in the file?

[Adoni5]

We still output the unblocked_read_ids.txt during an experiment - so you should be able to use that!
Reads that would be sent an unblock and reads that are sent a stop_receiving are demultiplexed into the actual file. Assuming you used the output of a readfish experiment, you can use analysis_unblock.fastq.gz for unblocked reads, and analysis_stop_receiving.fastq.gz + control_stop_receiving.fastq.gz (as all control reads are stop_receiving).

Any reads which are deemed proceed would most likely end up as unblocked as they would hit the exceed_max_chunks behaviour, which is by default unblock

[jennieli421]

To make sure I understand, in the demultiplex files, a read id is allowed to appear multiple times in those files. Eg., if first proceed then stop_receiving, it would show up in both fastq.gz.

[Adoni5]

No - we only make one decision per read in this case. So if a read is falls into the proceed category according to the rules in your toml file, that is the file it will end up in.

[jennieli421]

but what if based on the first chunk a read is unmapped and proceed, and then the second chunk is mapped, won't it be in the stop_receiving file?

[jennieli421]

Additionally, I notice that if I use a toml withno_op for aligner, readfish stats will run into an error and cannot split up the fastq files.

[mattloose]

but what if based on the first chunk a read is unmapped and proceed, and then the second chunk is mapped, won't it be in the stop_receiving file?

No - because when you run the readfish stats script it is analysing the read as a whole and telling you what it should do based on the logic in your toml file for the whole read.

The only place that chunks come into play is during a live run when you are sequencing. In those cases, chunks will be assigned but the last decision is the one that is recorded.

Please ask new unrelated questions in new issues as this helps other users find answers to similar questions.

[mattloose]

Additionally, I notice that if I use a toml withno_op for aligner, readfish stats will run into an error and cannot split up the fastq files.

Again - this is a new question so please open a new issue.

In this case, readfish stats should probably give an error. However, it doesn't really make sense to run readfish stats with a no_op aligner.

The no_op aligner doesn't align your data so there is no way for readfish stats to determine the logic of the underlying decisions made about the read.

You should run readfish stats with the same toml file used for the readfish experiment you wish to analyse.