Performance Evaluation is often the critical part in evaluating the results of a research project. Many of us are familiar with simulations, but it is often difficult to address questions like
- I want to estimate a probability of error and I see no error in n experiments: what can I say about the error probability ?
- I would like to characterize the fairness of my protocol. Should I use Jain's Fairness Index or the Lorenz Curve Gap ?
- should I eliminate the beginning of the simulation in order to wait until the system stabilizes ?
- I would like to fit an explanatory model to my data, I was told to use least squares for that; is that the right thing to do ? Why ?
- I simulate a random way point model but the average speed in my simulation is not as expected. What happened ?
- the reviewers of my paper complained that I did not provide confidence intervals. What is that ? How do I get them ?
- how do I analyze the power consumption of my system ?
- I would like to fit a distribution to the flow sizes that I measured but all my measurements are truncated to a maximum value; how do I account for the truncation~?
- what is Palm calculus and why should I know about it ?
These and other questions are the topic of this lecture.