How to Control Neural Network Model Capacity With Nodes and Layers

Last Updated on August 25, 2020

The capacity of a deep learning neural network model controls the scope of the types of mapping functions that it is able to learn.

A model with too little capacity cannot learn the training dataset meaning it will underfit, whereas a model with too much capacity may memorize the training dataset, meaning it will overfit or may get stuck or lost during the optimization process.

The capacity of a neural network model is defined by configuring the number of nodes and the number of layers.

In this tutorial, you will discover how to control the capacity of a neural network model and how capacity impacts what a model is capable of learning.

After completing this tutorial, you will know:

• Neural network model capacity is controlled both by the number of nodes and the number of layers in the model.
• A model with a single hidden layer and sufficient number of nodes has the capability of learning any mapping function, but the chosen learning algorithm may or may not be able to realize this capability.
• Increasing the number of layers provides a short-cut to increasing the capacity of the model with fewer resources,
  To finish reading, please visit source site