Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Edge case in Mitotic Branching Correctness #157

Open
lmanan opened this issue Aug 26, 2024 · 2 comments
Open

Edge case in Mitotic Branching Correctness #157

lmanan opened this issue Aug 26, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@lmanan
Copy link

lmanan commented Aug 26, 2024

For certain datasets, where there are no ground truth divisions and an employed tracker model also predicts no divisions, in such a scenario, the number of true positive divisions would be equal to 0, false positive divisions would be equal to 0 and false negative divisions would be equal to 0.

In this scenario, reporting the mitotic branching correctness measure equal to zero (see here) might be misleading (?). Maybe a warning can be raised or np.nan can be returned (similar to how it is done for evaluation of F1 score here, see documentation for zero_division parameter), so that it is excluded from any further evaluation.

@lmanan lmanan added the bug Something isn't working label Aug 26, 2024
@DragaDoncila
Copy link
Collaborator

@lmanan thanks for raising this issue! I agree that reporting a 0 is misleading.

I would lean towards both a warning and a NaN, as the warning is only useful at runtime while the NaN could be written to file and persist even if the user wasn't looking at the terminal output.

I would probably vote against making it a settable argument like they have in the F1 score for scikit-learn, because I think it's additional complexity without much benefit. @cmalinmayor @msschwartz21 curious on your thoughts?

@msschwartz21
Copy link
Collaborator

@DragaDoncila I agree with everything you proposed 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: No status
Development

No branches or pull requests

3 participants