Penalty for wrong ticks

Penalty for wrong ticks

par Sarah Nicole Antille,
Nombre de réponses : 4



Good afternoon,

I'm wondering how exactly the penalty for wrong ticks work. As you can see below I ticked 3 answers, two of which were the two correct ones. I got 0 point out of 2 so should I assume that it is +1 for a correct answer and -2 for a incorrect one? Or is it simply 0 point when we tick a wrong one?

Thanks in advance,

Sarah

En réponse à Sarah Nicole Antille

Penalty for wrong ticks

par Matthieu Le Cauchois,
Good afternoon,

Would it not make sense to compute the std on precision, recall and F1 metrics over k-fold cross validation? Or do we assume here that the system is trained and we calculate metrics on the whole test set at once, hence no variability?

Thank you
En réponse à Matthieu Le Cauchois

Re: Penalty for wrong ticks

par Jean-Cédric Chappelier,

For the same question, you mean?

Sure it would make sense but to evaluate... ...the evaluation itself. Standard deviation is a way to evaluate whether the score of the evaluation of the system makes sense (is relevant) or not. Standard deviation is NOT the score to evaluate the system itself.

En réponse à Jean-Cédric Chappelier

Penalty for wrong ticks

par Matthieu Le Cauchois,
Yes sorry, for the same question, thank you! I figured in the sense that std would give confidence information over the generalization ability of the system on the different validation subsets. But I understand the point that in fact it is not directly an evaluation score of the system itself.