Energy prices forecast

with Shapelets

Carlos Sevilla

07 June 2022 | 6 minutes

The Case: How to forecast energy pricing using energy generation data

The accurate forecasting of energy prices is crucial for the orientation of the energy market and can guide policymakers and market participants, such as businesses and individuals.

In this use case, we will explore how Shapelets accelerated platform can be used to improve price prediction. We will use historical energy price data together with energy generation data to find out how energy consumption and production are related.

In this use case, we will explore how Shapelets accelerated platform can be used to improve price prediction. We will use historical energy price data together with energy generation data to find out how energy consumption and production are related.

 

Note

While this case study focuses on the Spanish energy market, the following approach is applicable to any other international market, as long as large customer databases are available.

Furthermore, while the energy pricing market needs to follow each jurisdiction's rules, the primary objective of this use case is to obtain an understanding of the relationship between energy price data and energy generation data to improve its prediction.

Intro

The electricity market in Spain is regulated by the REE (Red Eléctrica Española) and acts as an intermediary between companies that generate energy and companies that buy this energy to distribute it to the customers.

The price of electricity is calculated using a matching offer. Every day at 16:00 the next day’s energy prices are calculated, and it is calculated by ordering from the lowest to highest the prices at which energy sellers want to sell the energy produced, and by ordering from the highest to lowest the prices at which buyers want to buy the energy.

This process is repeated for each hourly segment of the following day and is managed by Operador del Mercado Ibércio-Polo español (OMIE).

The objective of this DataApp is to try to predict the energy matching price for each of the hours a day.  

The Challenge

The main challenge solved with this Data App is the construction of a benchmark of anomaly detection algorithms, implemented in different python libraries, in order to find a powerful anomaly detection solution for one of the annotated time series in the Numenta Anomaly Benchmark (NAB).

data

The data used in this project is a dataset extracted from the OMIE API. This API has 1400 indicators available for analysis, among them information on scheduled power generation, real-time generation, and energy price including intraday market session prices.

 

grid of grey points background

Methodology

To solve this use case, we have applied the following methodology.

Step 1: Step 1 Data loading and anexploratory data analysis (EDA).)

Step 2: Data processingbycreatingthe target variable.

Step 3: Data modelling and performance analysis.

We have some initial business knowledge about this data, and we know that price evolution throughout the day has a very similar shape over the different days of the month.

Knowing this, you can use a dimensionality reduction algorithm such as PCS to reduce the dimensionality of the target variable.

TODO: You can try adding information about the season of the year or month information, to see if the results improve!

metrics

For this project, we have used traditional regression error metrics such as MSE, RMSE, MAE and MAPE. The target feature in this use case is a value that will be used by a PCA algorithm to obtain the real price data so that a small error can mean a very high deviation in the real price data. From the Shapelets team, we have chosen MAE as a reference metric, to focus our efforts on reducing the MAE value to the minimum possible.

 

algorithms

This problem can be approached from different points of view.

We have used 3 algorithms:

Synthesized Resolution

Predicting the price allow companies that purchase energy to adjust their budget more accurately.

Once the data is processed, three algorithms are proposed to predict the energy price which is evaluated using regression error metrics. These models have similar performance, giving some tighter predictions for each day.

This data app implements a system to compare the predictions between them so that the user can choose which model best fits his business vision.

results

Several interesting results arise from this study:

The first insight that is obtained is based on this study’s results: it is indeed possible to predict the price of energy with a very small error.

We have obtained an accurate energy price prediction with a 1.12 average score in MAE on the target variable.

Additionally, if we look it from a business perspective, there is only a 35.77 € average difference in the pricing prediction.

With only 10 KPIs you get a very accurate prediction of the price of energy, so you can improve this approach to get better results. It can serve as inspiration to incorporate it into your project or to compare it with other approaches or ways to solve this project.

This Data App implements a system to compare the predictions between them so that the user can choose which model best fits his business vision.
With Shapelets you have all the power of Python and its different packages, so it is as easy as importing and running them!

energy-price-forecast-use-case-graph-2
grid of grey points background

How does Shapelets help solve this challenge? 

Data App development in less than 30 min.

Results are ready to be shared with the business stakeholders in a secure way.

Data App ready to go into test for multiple approaches.

This DataApp shows us that it is possible to predict the price of energy using power generation data and machine learning algorithms.

The real goal of this Data App is to show you a starting point to develop this problem. With only 10 KPIs you get a very accurate prediction of the price of energy, so you can improve this approach to get better results. It can serve as inspiration to incorporate it into your project or to compare it with other approaches or ways to solve this project.

Now, what are you waiting for?

If you have any questions or would like some business guidance, you may contact us here

Click here to download our free eBook and give us some feedbacK!
Carlos Sevilla

Carlos Sevilla

Data Scientist

Carlos Sevilla is a Data Scientist at Shapelets. He has experience with time series models and has worked for large international companies in the energy and telecommunications sectors such as Endesa and Telefónica.