MP3: How do you sample from the softmax distribution?

MP3: How do you sample from the softmax distribution?

by Thibault Serge Mario Porssut -
Number of replies: 2

Hi,

I m sorry for my question but how do you sample from the softmax distribution?

Because for the moment I pick the action with the highest probability.....

Thank in advance for your help

Thibault

In reply to Thibault Serge Mario Porssut

Re: MP3: How do you sample from the softmax distribution?

by Olivier Jérémie Cloux -

(not a TA, so won't give too much details)
So here, you have a list of actions (say, [0,1,2,3]) with each a given probability (say, [0.5,0.2,0.1,0.2]).

You want to pick an action not completely deterministic (what you say you do, pick one with highest probability), neither full random (otherwise... well training is useless). Your job to understand why (and also it's one of the question).

You want to pick one at random, with "weights"  (so action 0 with probability 0.5 has a higher chance to be picked).

This is the idea. Now in practice, the situation I described above is similar to throwing a biased die. Gonna help you a little, that's called a multinomial distribution. Research on that and on numpy should be enough.

Hope that helps, without giving away too much :)

Best of luck

O. Cloux