Last Updated on August 28, 2020 It can be difficult when starting out on a new predictive modeling project with neural networks. There is so much to configure, and no clear idea where to start. It is important to be systematic. You can break bad assumptions and quickly hone in on configurations that work and areas for further investigation likely to payoff. In this tutorial, you will discover how to use exploratory configuration of multilayer perceptron (MLP) neural networks to […]

Last Updated on August 28, 2020 Long Short-Term Memory (LSTM) models are a type of recurrent neural network capable of learning sequences of observations. This may make them a network well suited to time series forecasting. An issue with LSTMs is that they can easily overfit training data, reducing their predictive skill. Dropout is a regularization method where input and recurrent connections to LSTM units are probabilistically excluded from activation and weight updates while training a network. This has the […]

Last Updated on August 28, 2020 Long Short-Term Memory (LSTM) models are a recurrent neural network capable of learning sequences of observations. This may make them a network well suited to time series forecasting. An issue with LSTMs is that they can easily overfit training data, reducing their predictive skill. Weight regularization is a technique for imposing constraints (such as L1 or L2) on the weights within LSTM nodes. This has the effect of reducing overfitting and improving model performance. […]

Last Updated on August 28, 2020 The Long Short-Term Memory network or LSTM is a recurrent neural network that can learn and forecast long sequences. A benefit of LSTMs in addition to learning long sequences is that they can learn to make a one-shot multi-step forecast which may be useful for time series forecasting. A difficulty with LSTMs is that they can be tricky to configure and it can require a lot of preparation to get the data in the […]

Last Updated on August 5, 2019 Recurrent neural networks are a type of neural network that add the explicit handling of order in input observations. This capability suggests that the promise of recurrent neural networks is to learn the temporal context of input sequences in order to make better predictions. That is, that the suite of lagged observations required to make a prediction no longer must be diagnosed and specified as in traditional time series forecasting, or even forecasting with […]

Last Updated on August 5, 2019 Long Short-Term Memory (LSTM) is a type of recurrent neural network that can learn the order dependence between items in a sequence. LSTMs have the promise of being able to learn the context required to make predictions in time series forecasting problems, rather than having this context pre-specified and fixed. Given the promise, there is some doubt as to whether LSTMs are appropriate for time series forecasting. In this post, we will look at […]

Last Updated on August 28, 2020 It is common to have missing observations from sequence data. Data may be corrupt or unavailable, but it is also possible that your data has variable length sequences by definition. Those sequences with fewer timesteps may be considered to have missing values. In this tutorial, you will discover how you can handle data with missing values for sequence prediction problems in Python with the Keras deep learning library. After completing this tutorial, you will […]

Last Updated on August 5, 2019 The data for your sequence prediction problem probably needs to be scaled when training a neural network, such as a Long Short-Term Memory recurrent neural network. When a network is fit on unscaled data that has a range of values (e.g. quantities in the 10s to 100s) it is possible for large inputs to slow down the learning and convergence of your network and in some cases prevent the network from effectively learning your […]

Last Updated on June 23, 2020 Time series datasets may contain trends and seasonality, which may need to be removed prior to modeling. Trends can result in a varying mean over time, whereas seasonality can result in a changing variance over time, both which define a time series as being non-stationary. Stationary datasets are those that have a stable mean and variance, and are in turn much easier to model. Differencing is a popular and widely used data transform for […]

Last Updated on August 5, 2019 3 Strategies to Design Experiments and Manage Complexity onYour Predictive Modeling Problem. It is difficult to get started on a new time series forecasting project. Given years of data, it can take days or weeks to fit a deep learning model. How do you get started exactly? For some practitioners, this can lead to paralysis and even procrastination at the very beginning of a project. In others, it can result in being caught in […]