Controlled Experiments in Machine Learning

Last Updated on August 8, 2019

Systematic experimentation is a key part of applied machine learning.

Given the complexity of machine learning methods, they resist formal analysis methods. Therefore, we must learn about the behavior of algorithms on our specific problems empirically. We do this using controlled experiments.

In this tutorial, you will discover the important role that controlled experiments play in applied machine learning.

After completing this tutorial, you will know:

• The need for systematic discovery via controlled experiments.
• The need to repeat experiments in order to control for the sources of variance.
• Examples of experiments performed in machine learning and the challenge and opportunity they represent.

Kick-start your project with my new book Statistics for Machine Learning, including step-by-step tutorials and the Python source code files for all examples.

Let’s get started.

Controlled Experiments in Machine Learning
Photo by Mike Baird, some rights reserved.

Tutorial Overview

This tutorial is divided into 3 parts; they are:

1. Systematic Experimentation
2. Controlling For Variance
3. Experiments in Machine Learning