Hello,
I realize that a notation in HW4 might not be crystal clear to you. This notation "will be" explained only in Lecture 5 (the thing is, I prepared Lecture 5 a month ago: this is one of the disadvantages of these prerecorded classes: you lose track of what you already said while preparing exercises...).
The notation in question is . Here,
stands actually for the
-th row of the matrix
: it is the distribution
of the Markov chain at time
, given that the chain started in state
at time 0 (i.e.
).
All the best,
Olivier