Forecasting Many Time Series (Using NO For-Loops)
Written by Matt Dancho on July 19, 2021
Spending too much time on making iterative forecasts? I’m super excited to introduce the new panel data forecasting functionality in
modeltime. It’s perfect for forecasting many time series at once without for-loops saving you time ⏱️ and aggravation 😞. Just say NO to for-loops for forecasting.
Fitting many time series can be an expensive process. The most widely-accepted technique is to iteratively run an ARIMA model on each time series in a for-loop.
Times are changing. Organizations now need 1000’s of forecasts. Think 1000s of customers, products, and complex hierarchical data.
In this tutorial:
We’ll explain new techniques involving Global Models and Panel Data for dealing with many time series.
We’ll then provide an introductory tutorial using
modeltime0.7.0 new features (now available on CRAN 🎉) for modeling time series as Panel Data to make forecasts without for-loops.
If you like what you see, I have an Advanced Time Series Course where you will learn the foundations of the growing Modeltime Ecosystem.
First, here’s a quick overview of the Modeltime Forecasting Ecosystem that unlocks
tidymodels (similar to scikit learn but in R).
What is Modeltime?
A growing ecosystem for tidymodels forecasting
Modeltime is part of a growing ecosystem of forecasting packages. Modeltime integrates
tidymodels for forecasting at scale. The ecosystem contains:
Problem: Forecasting with For-Loops is Not Scalable
Time series data is increasing at an exponential rate. Organization-wide forecasting demands have changed from top-level to bottom-level forecasting, which has increased the number of forecasts that need to be made from the range of 1-100 to the range of 1,000-10,000.
Think of forecasting by customer for an organization that has 10,000 customers. It becomes a challenge to make these forecasts one at a time in an iterative approach. As that organization grows, moving from 10,000 to 100,000 customers, forecasting with an iterative approach is not scalable.
Modeltime has been designed to take a different approach using Panel Data and Global Models (more on these concepts shortly). Using these approaches, we can dramatically increase the scale at which forecasts can be made. Prior limitations in the range of 1,000 to 10,000 forecasts become managable. Beyond is also possible with clustering techniques and making several panel models. We are only limited by RAM, not modeling time.
Before we move on, we need to cover two key concepts:
- Panel Data
- Global Models
What are Panel Data and Global Models?
In it’s simplest form, Panel Data is a time series dataset that has more than one series. Each time series is stacked row-wise (on-top) of each other.
The Panel Data Time Series Format
Traditional modeling techniques like ARIMA can only be used on one time series at a time. The widely accepted forecasting approach is to iterate through each time series producing a unique model and forecast for each time series identifier. The downside with this approach is that it’s expensive when you have many time series. Think of the number of products in a database. As the number of time series approaches the range of 1000-10,000, the iterative approach becomes unscalable as for-loops run endlessly and errors can grind your analysis to a hault.
Problem: 1000 ARIMA Models Needed for 1000 Time Series
Global Models are alternatives to the iterative approach. A Global Model is a single model that forecasts all time series at once. Global Models are highly scalable, which solves the problem of 1-10,000 time series. An example is an XGBoost Model, which can determine relationships for all 1000 time series panels with a single model. This is great: No For-Loops!
Solution: A Single XGBOOST Model can Model 1000 Time Series leaves you waiting for hours ⏳...
The downside is that an global model approach can be less accurate than the iterative approach. To improve accuracy, feature engineering and localized model selection by time series identifier become critical to large-scale forecasting success. If interested, I teach proven feature engineering techniques in my Time Series Forecasting Course.
Say No to For-Loops
If you’re tired of waiting for ARIMA models to finish, then maybe it’s time to say NO to for-loops and give
modeltime a try.
Forecast using Global Models and Panel Data with Modeltime for a 1000X Speed-up
While Modeltime can perform iterative modeling, Modeltime excels at forecasting at scale without For-Loops using:
Global Modeling: Global model Machine Learning and Deep Learning strategies using the Modeltime Ecosystem (e.g.
Panel Data: Tidy data that is easy to work with if you are familiar with the
Feature Engineering: Developing calendar features, lagged features, and other time-based, window-based, and sequence-based features using
Multi-Forecast Visualization: Visualizing multiple local time series forecasts at once.
Global and Localized Accuracy Reporting: Generating out-of-sample accuracy both globally and at a local level by time series identifier (available in
Global and Localized Confidence Intervals Reporting: Generating out-of-sample confidence intervals both globally and at a local level by time series identifier (available in
Once you learn these concepts, you can achieve speed-ups of 1000X or more. We’ll showcase several of these features in our tutorial on forecasting many time series without for-loops.
Tutorial on Forecasting Many Time Series (Without For-Loops)
We’ll cover a short tutorial on Forecasting Many Time Series (Without For-Loops).
First, load the following libraries.
Next, collect the
walmart_sales_weekly dataset. The dataset consists of 1001 observations of revenue generated by a store-department combination on any given week. It contains:
- 7 Time Series Groups denoted by the “ID” column
- The data is structured in Panel Data format
- The time series groups will be modeled with a single Global Model
Visualize the Data
From visualizing, the weekly department revenue patterns emerge. Most of the series have yearly seasonality and long-term trends.
We can split the data into training and testing sets using
time_series_split(). We’ll investigate the last 3-months of the year to test a global model on a 3-month forecast. The message on overlapping dates is to let us know that multiple time series are being processed using the last 3-month window for testing.
Feature Engineering (Recipe)
We can move to preprocessing the data. We will use the
recipes workflow for generating time series features.
This results in 37 derived features for modeling.
We can certainly include more features such as lags and rolling features, which are covered in the High-Performance Time Series Course.
We’ll create an
xgboost workflow by fitting the default xgboost model to our derived features from our in-sample training data set.
We create a Global XGBOOST Model, a single model that forecasts all of our time series
Training the global xgboost model takes approximately 50 milliseconds.
Conversely, an ARIMA model might take several minutes to iterate through possible parameter combinations for each of the 7 time series.
Global modeling is a 1000X speedup.
We’ll step through the modeltime workflow, which is used to test many different models on the time series and organize the entire process.
Create a Modeltime Table
First, we create a Modeltime Table using
modeltime_table(). The Modeltime Table organizes our model(s). We can even add more models if we’d like, and each model will get an ID (.model_id) and description (.model_desc).
Calibrate by ID
Next, we calibrate. Calibration calculates the out of sample residual error.A new feature in
modeltime 0.7.0 is the ability to calibrate by each time series.
Next, we measure the global and local accuracy on the global model.
Global Accuracy is the overall accuracy of the test forecasts, which simply returns an aggregated error without taking into account that there are multiple time series. The default is
modeltime_accuracy(acc_by_id = FALSE), which returns a global model accuracy.
The drawback with the global accuracy is that the model may not perform well on specific time series. By toggling
modeltime_accuracy(acc_by_id = TRUE), we can obtain the Local Accuracy, which is the accuracy that the model has on each of the time series groups. This can be useful for identifying specifically which time series the model does well on (and which it does poorly on). We can then apply model selection logic to select specific global models for specific IDs.
Forecast the Data
The last step we’ll cover is forecasting the test dataset. This is useful to evaluate the model using a sampling of the time series within the panel dataset. In
modeltime 0.7.0, we now have
modeltime_forecast(conf_by_id = TRUE) to allow the confidence intervals (prediction intervals) to be calculated by time series identifier. Note, that the
modeltime_calibrate() must have been performed with an
We just showcased the Modeltime Workflow for Panel Data using a Global XGBOOST Model. But, this is a simple problem. And, there’s a lot more to learning time series:
- Many more algorithms
- Feature Engineering for Time Series
- Machine Learning
- Deep Learning
- Scalable Modeling: 10,000+ time series
Your probably thinking how am I ever going to learn time series forecasting. Here’s the solution that will save you years of struggling.
It gets better
You’ve just scratched the surface, here’s what’s coming…
The Modeltime Ecosystem functionality is much more feature-rich than what we’ve covered here (I couldn’t possibly cover everything in this post). 😀
Here’s what I didn’t cover:
Feature Engineering: We can make this forecast much more accurate by including features from competition-winning strategies
Ensemble Modeling: We can stack models together to make super-learners that stabilize predictions.
Deep Learning: We can use GluonTS Deep Learning for developing high-performance, scalable forecasts.
So how are you ever going to learn time series analysis and forecasting?
You’re probably thinking:
- There’s so much to learn
- My time is precious
- I’ll never learn time series
I have good news that will put those doubts behind you.
You can learn time series analysis and forecasting in hours with my state-of-the-art time series forecasting course. 👇
High-Performance Time Series Course
Become the times series expert in your organization.
My High-Performance Time Series Forecasting in R course is available now. You’ll learn
modeltime plus the most powerful time series forecasting techniques available like GluonTS Deep Learning. Become the times series domain expert in your organization.
You will learn:
- Time Series Foundations - Visualization, Preprocessing, Noise Reduction, & Anomaly Detection
- Feature Engineering using lagged variables & external regressors
- Hyperparameter Tuning - For both sequential and non-sequential models
- Time Series Cross-Validation (TSCV)
- Ensembling Multiple Machine Learning & Univariate Modeling Techniques (Competition Winner)
- Deep Learning with GluonTS (Competition Winner)
- and more.
Have questions about Modeltime?
Make a comment in the chat below. 👇
And, if you plan on using
modeltime for your business, it’s a no-brainer - Join my Time Series Course.