Revolutionizing energy price forecasting with Shapelets

The Case: How to forecast energy pricing using energy generation data

The accurate forecasting of energy prices is crucial for the orientation of the energy market and can guide policymakers and market participants, such as businesses and individuals.

In this use case, we will explore how Shapelets accelerated platform can be used to improve price prediction. We will use historical energy price data together with energy generation data to find out how energy consumption and production are related.

Intro

The electricity market in Spain is regulated by the REE (Red Eléctrica Española) and acts as an intermediary between companies that generate energy and companies that buy this energy to distribute it to the customers.

The price of electricity is calculated using a matching offer. Every day at 16:00 the next day’s energy prices are calculated, and it is calculated by ordering from the lowest to highest the prices at which energy sellers want to sell the energy produced, and by ordering from the highest to lowest the prices at which buyers want to buy the energy.

This process is repeated for each hourly segment of the following day and is managed by Operador del Mercado Ibércio-Polo español (OMIE).

The objective of this DataApp is to try to predict the energy matching price for each of the hours a day.

1. The Challenge

The main challenge solved with this Data App is the construction of a benchmark of anomaly detection algorithms, implemented in different python libraries, in order to find a powerful anomaly detection solution for one of the annotated time series in the Numenta Anomaly Benchmark (NAB).

Data

The data used in this project is a dataset extracted from the OMIE API. This API has 1400 indicators available for analysis, among them information on scheduled power generation, real-time generation, and energy price including intraday market session prices.

Methodology

To solve this use case, we have applied the following methodology.

• Step 1: Step 1 Data loading and anexploratory data analysis (EDA).)
• Step 2: Data processingbycreatingthe target variable.
• Step 3: Data modelling and performance analysis.

We have some initial business knowledge about this data, and we know that price evolution throughout the day has a very similar shape over the different days of the month.

Knowing this, you can use a dimensionality reduction algorithm such as PCS to reduce the dimensionality of the target variable.

You can try adding information about the season of the year or month information, to see if the results improve!

Metrics

For this project, we have used traditional regression error metrics such as MSE, RMSE, MAE and MAPE. The target feature in this use case is a value that will be used by a PCA algorithm to obtain the real price data so that a small error can mean a very high deviation in the real price data.

From the Shapelets team, we have chosen MAE as a reference metric, to focus our efforts on reducing the MAE value to the minimum possible.

Algorithms

We have used 3 algorithms:

Synthesized Resolution

Predicting the price allow companies that purchase energy to adjust their budget more accurately.

Once the data is processed, three algorithms are proposed to predict the energy price which is evaluated using regression error metrics. These models have similar performance, giving some tighter predictions for each day.

This data app implements a system to compare the predictions between them so that the user can choose which model best fits his business vision.

Results

Several interesting results arise from this study:

This Data App implements a system to compare the predictions between them so that the user can choose which model best fits his business vision.
With Shapelets you have all the power of Python and its different packages, so it is as easy as importing and running them!

How does Shapelets help solve this challenge?

This DataApp shows us that it is possible to predict the price of energy using power generation data and machine learning algorithms.

The real goal of this Data App is to show you a starting point to develop this problem. With only 10 KPIs you get a very accurate prediction of the price of energy, so you can improve this approach to get better results. It can serve as inspiration to incorporate it into your project or to compare it with other approaches or ways to solve this project.

Now, what are you waiting for?

If you have any questions or would like some business guidance, you may contact us here