Set 3, Ex 5

Set 3, Ex 5

par Plouton Grammatikos,
Number of replies: 1

Hi,

The solution of ex. 5c of the third week's set claims that when  a=a' , we update only one weight per step. However, state  s'  might fall in a different box than state s. If, for example, state s is in box k and state s' is in box j, then according to the answer to question 5b, shouldn't we update both  w^a_k and w^a_j ? In one case \Phi(s-s_k) \neq0 , while in the other  \Phi(s'-s_j) \neq0 .
In reply to Plouton Grammatikos

Re: Set 3, Ex 5

par Bernd Albert Illing,

Hi Plouton,

Thanks for pointing this out! I think that's a very good question and I don't see a reason why you shouldn't be right. 

At least in the general case that s and s' are in different boxes, we should update two weights, even for the same action a = a' .

I'll verify with the Professor and come back to you in case there is something I missed.

Best regards,

Bernd