Machine Learning

How to Develop Word-Based Neural Language Models in Python with Keras

Last Updated on September 3, 2020

Language modeling involves predicting the next word in a sequence given the sequence of words already present.

A language model is a key element in many natural language processing models such as machine translation and speech recognition. The choice of how the language model is framed must match how the language model is intended to be used.

In this tutorial, you will discover how the framing of a language model affects the skill of the model when generating short sequences from a nursery rhyme.

After completing this tutorial, you will know:

  • The challenge of developing a good framing of a word-based language model for a given application.
  • How to develop one-word, two-word, and line-based framings for word-based language models.
  • How to generate sequences using a fit language model.

Kick-start your project with my new book Deep Learning for Natural Language Processing, including step-by-step tutorials and the Python source code files for all examples.

Let’s get started.