The Model Performance Mismatch Problem (and what to do about it)

What To Do If Model Test Results Are Worse than Training.

The procedure when evaluating machine learning models is to fit and evaluate them on training data, then verify that the model has good skill on a held-back test dataset.

Often, you will get a very promising performance when evaluating the model on the training dataset and poor performance when evaluating the model on the test set.

In this post, you will discover techniques and issues to consider when you encounter this common problem.

After reading this post, you will know:

• The problem of model performance mismatch that may occur when evaluating machine learning algorithms.
• The causes of overfitting, under-representative data samples, and stochastic algorithms.
• Ways to harden your test harness to avoid the problem in the first place.

This post was based on a reader question; thanks! Keep the questions coming!

Let’s get started.

Overview

This post is
To finish reading, please visit source site