M2R Scientific Methodology and Performance Evaluation

The coordinator for these lectures is Arnaud Legrand. The lecturers are Jean-Marc Vincent and Arnaud Legrand .

Lectures take place on Thursday afternoon from 3:45PM to 5:15PM, generally in Amphi H of the Ensimag (H building).

Here is a map of the campus for those who do not know where the rooms are.

The planning with lecture rooms is available at the ADE website (look for PDS then for Scientific Methodology).

The aim of this course is to provide the fundamental basis for sound scientific methodology of performance evaluation of computer systems. This lecture emphasize on methodological aspects of measurement and on the statistics needed to analyze computer systems. We first sensibilize the audience to the importance of experiment and analysis reproducibility in particular in computer science. Then we present tools that help answering the analysis problem and may also reveal useful for managing the experimental process through notebooks. The audience will be given the basis of probabilities and statistics required to develop sound experiment designs. Unlike some other lectures, our goal is not to provide analysis recipes that people can readily apply but to make the audience truly understand some simple statistical tools on which they can build further.

Here are links to the previous editions of this lecture: 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017.

See https://github.com/alegrand/SMPE for details of the lectures and additional resources. You can also check the last edition.

1. 5 October 2017: A. Legrand Reproducible research
   • Here is the Pad you may use to collaborate.
   • Here is a poll. Please fill it as soon as you have a few minutes but do not waste your time doing it during the lecture.
2. 12 October 2017: A. Legrand Litterate programming
3. 26 October 2017: A. Legrand R crash course
4. 9 November 2017: J.M. Vincent Descriptive statistics of univariate data
5. 15 November 2017: J.M. Vincent Data presentation
6. 23 November 2017: J.M. Vincent Introduction to probabilities and statistics, confidence interval
7. 30 November 2017: J.M. Vincent Correlation/causation
8. 7 December 2017: A. Legrand Linear regression
9. 14 December 2017: A. Legrand Linear regression
10. 21 December 2017: A. Legrand Design of experiments
11. 11 January 2017: A. Legrand Homework presentations
12. 15 January 2017 (Monday!!!): J.M. Vincent A bit of epistemology…
    • Homework: Read the presentation of Karl Popper Science: Conjectures and Refutations.

    • Question: This text has been written before Computer Science was funded. Explain in at most one page, how Popper's theory could apply in the context of Computer Science research now.

      Etudiants Intervenants (accès restreint) Salles Equipements Enseignements Autres Autres (accès restreint)

As explained during the lectures, now that we have covered the whole spectrum of experimental study (preliminary analysis and visualization, modeling, experiment planning, corresponding analysis), I want you to put this into practice. The best way for this is to work in small groups, pick a topic of your choice and just report your activity and possibly conclude. The topic and the conclusions you will reach are of little importance. What's important is that you do it in a clean way and allow others to look into your work.

Here is a pad (https://pad.inria.fr/p/RiWGRqRW2lnZqdUJ) where you will indicate for each group:

• the names of the members
• the topic in a few words
• and the URL where we can access your work (github, rstudio, …).

The deadline is set to January 19th so that we have enough time to read your work. Note that I have added in this pad a few links that some students returned me a few weeks ago. This is what they did in 5-6 hours max so you should have way enough time to do something pretty clean.