Hello,
Would it be possible to have a detailed solution for quizz 4 (all questions that are not multiple choice and where there's some calculation involved)?
Thank you
there are a bit too many (including all randomization).
I can address that in the QAS (see https://moodlearchive.epfl.ch/2022-2023/mod/forum/discuss.php?d=85338) or here if you have a more precise (focussed) question.
I can address that in the QAS (see https://moodlearchive.epfl.ch/2022-2023/mod/forum/discuss.php?d=85338) or here if you have a more precise (focussed) question.
I mean, if possible of course, it would be nice to have the solution of 1 of the different versions, but like the whole methodology so it can easily be applied on the other versions.
for what question more precisely? (I won't do it for all)
Then for 4 and 16 please
For question 4:
+ product of P(w|class) times class prior (don't forget the prior!)
+ estimates of the P(w|class) by ML: count w in class divided by nb of words in class
For your version of that question:
+ estimates: Space prior = 3/5 Animal prior = 2/6
likelihoods: Space: Space:4/10, Mouse: 1/10, Cat: 1/10, Dog: 4/10
Animal: Space:2/10, Mouse: 4/10, Cat: 3/10, Dog: 1/10
+ inference: P(D6,Space)
4*4*1*1*3=48
P(D6,Animal) 2*2*3*4*2=96
P(D7,Space) 1*1*4*4*3=48
P(D7,Animal) 3*3*2*1*2=36
+ product of P(w|class) times class prior (don't forget the prior!)
+ estimates of the P(w|class) by ML: count w in class divided by nb of words in class
For your version of that question:
+ estimates: Space prior = 3/5 Animal prior = 2/6
likelihoods: Space: Space:4/10, Mouse: 1/10, Cat: 1/10, Dog: 4/10
Animal: Space:2/10, Mouse: 4/10, Cat: 3/10, Dog: 1/10
+ inference: P(D6,Space)
4*4*1*1*3=48P(D6,Animal)
2*2*3*4*2=96P(D7,Space)
1*1*4*4*3=48P(D7,Animal)
3*3*2*1*2=36For question 16:
first of all attention is a weighted average of the encoder hidden states, so whenever you have the same components in the encoder hidden states, then the result has the same component as well.
So in your version the answer is already [10, 10, ?]
To compute the last component, you have to compute the weights of the average, which are the softmax of the dot-product of encoder by decoder hidden states. In your case the second encoder is only 0.1 higher than the first, thus the second dot-product is simply 0.1*6 higher than the first, which is 140+50+15=250 (thus second is 250.6).
Here the softtmay is of dim 2 so you only have to compute 1 value (well, actually you were provided with some software to do the computation, but it's not really needed): exp(250) / [ exp(250) + exp(250.6) ] = 0.35, thus second is 1-0.35=0.65
And thus the final component you are looking for is 0.35*2.5 + 0.65*2.6 = 2.565
first of all attention is a weighted average of the encoder hidden states, so whenever you have the same components in the encoder hidden states, then the result has the same component as well.
So in your version the answer is already [10, 10, ?]
To compute the last component, you have to compute the weights of the average, which are the softmax of the dot-product of encoder by decoder hidden states. In your case the second encoder is only 0.1 higher than the first, thus the second dot-product is simply 0.1*6 higher than the first, which is 140+50+15=250 (thus second is 250.6).
Here the softtmay is of dim 2 so you only have to compute 1 value (well, actually you were provided with some software to do the computation, but it's not really needed): exp(250) / [ exp(250) + exp(250.6) ] = 0.35, thus second is 1-0.35=0.65
And thus the final component you are looking for is 0.35*2.5 + 0.65*2.6 = 2.565
