Penalty for wrong ticks

Penalty for wrong ticks

by Sarah Nicole Antille -
Number of replies: 4



Good afternoon,

I'm wondering how exactly the penalty for wrong ticks work. As you can see below I ticked 3 answers, two of which were the two correct ones. I got 0 point out of 2 so should I assume that it is +1 for a correct answer and -2 for a incorrect one? Or is it simply 0 point when we tick a wrong one?

Thanks in advance,

Sarah

In reply to Sarah Nicole Antille

Re: Penalty for wrong ticks

by Jean-Cédric Chappelier -

For that one we decided that Cohen's kappa and standard deviation, which are completely out of scope, shall fully kill the question.

In reply to Sarah Nicole Antille

Penalty for wrong ticks

by Matthieu Le Cauchois -
Good afternoon,

Would it not make sense to compute the std on precision, recall and F1 metrics over k-fold cross validation? Or do we assume here that the system is trained and we calculate metrics on the whole test set at once, hence no variability?

Thank you
In reply to Matthieu Le Cauchois

Re: Penalty for wrong ticks

by Jean-Cédric Chappelier -

For the same question, you mean?

Sure it would make sense but to evaluate... ...the evaluation itself. Standard deviation is a way to evaluate whether the score of the evaluation of the system makes sense (is relevant) or not. Standard deviation is NOT the score to evaluate the system itself.

In reply to Jean-Cédric Chappelier

Penalty for wrong ticks

by Matthieu Le Cauchois -
Yes sorry, for the same question, thank you! I figured in the sense that std would give confidence information over the generalization ability of the system on the different validation subsets. But I understand the point that in fact it is not directly an evaluation score of the system itself.