M2R Scientific Methodology and Performance Evaluation

The coordinator for these lectures is Jean-Marc Vincent . The lecturers are Jean-Marc Vincent and Arnaud Legrand .

This year, this lecture is part of the Embedded Systems lecture.

Lectures take place on Wednesday afternoon from 2PM to 5PM, generally in H104.

Here is a map of the campus for those who do not know where the rooms are.

The planning with lecture rooms is available at the ADE website (look for PDES and then for Embedded Systems).

The aim of this course is to provide the fundamental basis for sound scientific methodology of performance evaluation of computer systems. This lecture emphasize on methodological aspects of measurement and on the statistics needed to analyze computer systems. We first sensibilize the audience to the importance of experiment and analysis reproducibility in particular in computer science. Then we present tools that help answering the analysis problem and may also reveal useful for managing the experimental process through notebooks. The audience will be given the basis of probabilities and statistics required to develop sound experiment designs. Unlike some other lectures, our goal is not to provide analysis recipes that people can readily apply but to make the audience truly understand some simple statistical tools on which they can build further.

Here are links to the previous editions of this lecture: 2011-2012, 2012-2013, 2013-2014, 2014-2015.

See https://github.com/alegrand/SMPE for details of the lectures and additional resources.

• 7 October 2015 (14:00 - 17:00): Arnaud Legrand, H104 Introduction to reproducible research and litterate programming.
• 14 October 2015 (8:00 - 11:00): Jean-Marc Vincent, H202 Data visualizationin in action with R.
• 18 November 2015 (14:00 - 17:00): Arnaud Legrand, H104 Introduction to probabilities and statistics.
• 25 November 2015 (14:00 - 17:00): Arnaud Legrand, D213 The linear model.
• 16 December 2015 (14:00 - 17:00): Arnaud Legrand, H104 Design of Experiments
• 13 January 2016 (14:00 - 17:00): Arnaud Legrand and Jean-Marc Vincent, H104 Mini-projects' defense.
  • Here are a few slides on Measuring on a computer system (part 2).
  • The goal was to try to "reproduce" and extend a bit the study proposed in https://github.com/alegrand/M2R-ParallelQuicksort. Several students simply forked the project and commited their work. The presentation was completely free, the goal of this session being mostly to generate discussions and make the student realize that many of them were reaching different conclusions and that conducting experiments and presenting results in a satisfying way is quite hard:
    • Cyril Six
    • Alia Hajjar
    • Raphaël Jackse et …
    • Rémy Boutonnet
    • Jad Darrous (HTML report)
    • Julien Stephan
    • Steven Quinito Masnado
    • Philippe Virouleau