Skip to content
This repository has been archived by the owner on Jun 2, 2023. It is now read-only.

Higher number of observations threshold to compute metrics #214

Open
jds485 opened this issue Mar 21, 2023 · 1 comment
Open

Higher number of observations threshold to compute metrics #214

jds485 opened this issue Mar 21, 2023 · 1 comment

Comments

@jds485
Copy link
Member

jds485 commented Mar 21, 2023

Code location in evaluate.py

Originally posted by @janetrbarclay in #211 (comment)

I think it's probably worth getting some broader input on this. A higher threshold makes sense for the top / bottom 10% metrics, but maybe not the others?

This change would mostly affect the reach and reach_month metrics (and the new biweekly metric and maybe the year metric). I don't really know the extent of the effects so did a quick data summary below.

There are 907 reach - partition pairs in the observation dataset. 192 have <= 10 temps (and therefore currently would have no metrics calculated) Increasing the cutoff to 20 would leave 256 reaches without metrics.

There are 7685 month - reach - partition pairings. 40% (3624) of them have <= 10 temps. Increasing the cutoff to 20 would leave 52% (4025) without metrics.

@jds485
Copy link
Member Author

jds485 commented Mar 21, 2023

We could make the minimum number of observations a function parameter.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant