Hi all,
Having a softmax output is sufficient indeed to consider the output as a probability distribution over words.
The readout can be whatever (non-recurrent) processing of the recurrent unit activities. With the proper dimension and activation function, such an additional layer can improve the performance of your model. But can also decrease the performance if the dimension and activations are badly chosen.
You are free to experience with this readout layer as you like. If you want to try without it, it is fine as well.
Hope it helps!
Best,
