ENERGY PRICE FORECAST

WITH SHAPELETS

CATEGORY

Shapelets

CATEGORY

Shapelets

CATEGORY

Shapelets

 CATEGORY

  Use Case

 DATE

  07 June

 TIME

   05 Minutes

The Case: How to forecast energy pricing using energy generation data 

 

The accurate forecasting of energy prices is crucial for the orientation of the energy market and can guide policymakers and market participants, such as businesses and individuals.   

In this use case, we will explore how Shapelets accelerated platform can be used to improve price prediction. We will use historical energy price data together with energy generation data to find out how energy consumption and production are related.  

 

USE CASE

By Carlos Sevilla (Data Scientist Shapelets)

In this use case, we will explore how Shapelets accelerated platform can be used to improve price prediction. We will use historical energy price data together with energy generation data to find out how energy consumption and production are related.

 

Note 

While this case study focuses on the Spanish energy market, the following approach is applicable to any other international market, as long as large customer databases are available. Furthermore, while the energy pricing market needs to follow each jurisdiction’s rules, the primary objective of this use case is to obtain an understanding of the relationship between energy price data and energy generation data to improve its prediction.   

Intro

 

The electricity market in Spain is regulated by the REE (Red Eléctrica Española) and acts as an intermediary between companies that generate energy and companies that buy this energy to distribute it to the customers.  

The price of electricity is calculated using a matching offer. Every day at 16:00 the next day’s energy prices are calculated, and it is calculated by ordering from the lowest to highest the prices at which energy sellers want to sell the energy produced, and by ordering from the highest to lowest the prices at which buyers want to buy the energy.  

This process is repeated for each hourly segment of the following day and is managed by Operador del Mercado Ibércio-Polo español (OMIE). 

The objective of this DataApp is to try to predict the energy matching price for each of the hours a day.  

 

The Challenge

 

The main challenge solved with this DataApp is to find some relationship between energy generation and energy price and discover that relationship using Machine Learning algorithms. Using different python libraries we implemented the best models to solve this problem. 

To get this challenge it is necessary to take into account some problems such as working with datasets that have different frequencies and also present anomalies within the data which make difficult the discovery of patterns in the data. 

Data

 

The data used in this project is a dataset extracted from the OMIE API. This API has 1400 indicators available for analysis, among them information on scheduled power generation, real-time generation, and energy price including intraday market session prices.

Methodology

 

To solve this use case, we have applied the following methodology. 

Step 1 Data loading and anexploratory data analysis (EDA).

Step 2 Data processingbycreatingthe target variable.

Step 3 Data modelling and performance analysis.

We have some initial business knowledge about this data, and we know that price evolution throughout the day has a very similar shape over the different days of the month.  

Knowing this, you can use a dimensionality reduction algorithm such as PCS to reduce the dimensionality of the target variable.  

TODO: You can try adding information about the season of the year or month information, to see if the results improve! 

Metrics

 

For this project, we have used traditional regression error metrics such as MSE, RMSE, MAE and MAPE. The target feature in this use case is a value that will be used by a PCA algorithm to obtain the real price data so that a small error can mean a very high deviation in the real price data. From the Shapelets team, we have chosen MAE as a reference metric, to focus our efforts on reducing the MAE value to the minimum possible 

Algorithms 

 

This problem can be approached from different points of view 

As an autoregressive time series, predicting the value of the principal component of the daily data.  

As a multivariate time series problem, with the different KPIs, we have

As a regression problem 

For this approach, we have treated the problem as a regression problem, in which, with the processing of the previous point, I predict the value of the principal component that we have predicted. The value of the principal component that we have calculated for the target.  

We have used 3 algorithms:  

The RandomForestRegressor algorithm from the package, and is implemented like this

The LightGBM algorithm, from Microsoft.  

The XGBoost algorithm 

This Data App implements a system to compare the predictions between them so that the user can choose which model best fits his business vision.  

With Shapelets you have all the power of Python and its different packages, so it is as easy as importing and running them!  

 

Synthesized Resolution  

 

Predicting the price allow companies that purchase energy to adjust their budget more accurately. 

Once the data is processed, three algorithms are proposed to predict the energy price which is evaluated using regression error metrics. These models have similar performance, giving some tighter predictions for each day.  

This data app implements a system to compare the predictions between them so that the user can choose which model best fits his business vision. 

Results

Several interesting results arise from this study: 

The first insight that is obtained is based on this study’s results: it is indeed possible to predict the price of energy with a very small error.

We have obtained an accurate energy price prediction with a 1.12 average score in MAE on the target variable. 

Additionally, if we look it from a business perspective, there is only a 35.77 € average difference in the pricing prediction 

With only 10 KPIs you get a very accurate prediction of the price of energy, so you can improve this approach to get better results. It can serve as inspiration to incorporate it into your project or to compare it with other approaches or ways to solve this project. 

This Data App implements a system to compare the predictions between them so that the user can choose which model best fits his business vision.  

With Shapelets you have all the power of Python and its different packages, so it is as easy as importing and running them!  

 

How does Shapelets help solve this challenge? 

 

Data App development in less than 30 min.  

Results are ready to be shared with the business stakeholders in a secure way.   

Data App ready to go into test for multiple approaches.   

This DataApp shows us that it is possible to predict the price of energy using power generation data and machine learning algorithms. 
The real goal of this Data App is to show you a starting point to develop this problem. With only 10 KPIs you get a very accurate prediction of the price of energy, so you can improve this approach to get better results. It can serve as inspiration to incorporate it into your project or to compare it with other approaches or ways to solve this project.  

Now, what are you waiting for? 

If you have any questions or would like some business guidance, you may contact us here.