Hi,
I have a question about 5.c. I have no idea what the heatmap of Q-values looks like because Q-value is a function of both states and actions. But the y-axis is only the actions selected. I am confused about what the visualization looks like.
Thanks in advance,
Yiyang