In all the project, when the performance questions arises, the answer is that we should have "reasonable performance". My problem is that given the stochastic nature of the outcomes, when can we consider we reached sufficient performance.
For instance my agents learns up until episode 2500 a roches a score of 150, but in the following episodes the score decreases ( guess he tired of playing..) and the score drops to -200.
A little after the score rises again and in the end the score is ~100 on episode 3000.
In this kind of configuration did we reach "reasonable" ?
This behaviour happens a lot and as the last steps of the project privilege exploration which can make even worse.
