Renewables storage - control the plant controller

I have received a lot of positive feedback on my first post, which of course encouraged me to write the next one. In general, this post and my last post show backtesting results to illustrate the use cases of the here-elaborated models. Few people criticize backtesting as an overrated, assumption sensitive technique, where one can count himself poor or rich. In my opinion that is not really the ambition of backtesting. On the contrary its goal should be stated the other way around: If backtesting shows senseless results on scientific and independent assumptions, we should never try the idea out or better adjust the idea to the results. If backtesting results behave well, there is an unneglectable chance that the idea might work in practice. Carrying out backtesting is good, because it develops a realistic feeling for possible outcomes and potential costs we may be confronted with.

Returning from this excursion, let us remember that we lately showed balancing portfolio effects between solar and wind in order to stabilize revenues achieved from renewables investment over time. Furthermore, we have shown that both and especially wind has not only a seasonal, but also a highly volatile production profile. This can cause problems e.g. when wind blows hard and consequently, we are in a production oversupply situation. In that case we have to turn down the windmill and cannot use its capacity. On the other hand, we have seen that renewables are quite naturally exposed to seasonal production profiles, which result in under-supply situations. Both ways end up with the question of how to deal with renewables energy specific production uncertainty and in my opinion lead to a solution through storage technologies, which serve as a buffer. In regards to storage we have to differ between seasonal long-term and short-term storage technology like batteries. This post is about a seasonal storage solution. We exemplify seasonal storage technology in the German case, where we have wind production under the EEG regime and a very well-established gas network, which serves as the buffering candidate. We will try to connect the wind production with natural gas and end up with the green storage technology, namely the Power-to-Gas (PtG) plants.

Let’s take a step back and shortly reflect what we have done so far. We dived into a practical model and showed its in and out-of-sample performance. We already mentioned some shortcomings of the model concerning volatility and distribution assumptions, especially in regards to the wind production simulation. Thereby we will work on our distribution hypotheses in the first part of this post. Quite naturally this will lead us to a generalized class of enthralling stochastic processes. By the help of these we will build a virtual power to gas plant. Simulations will help us to determine scenarios, when we should convert power to gas in the wind oversupply case and scenarios where we should not do that. In the end an algorithm should sell the gas on spot market or in the EEG regime when conditions in respective price regimes are appealing. The aim of this exercise should again be power plant cashflow / production stabilisation. In addition we have gained exposure to a new market, whose supply and demand mechanism is different from the normal power price market.

Stepping away from the conceptual viewpoint, there are already test power plants available by e.g. Uniper, but there is also a discussion going on whether this technology is fully marketable at its current state. Even though being a storable asset and hence in a way decoupled from the standard power price market, the gas market has proven himself to be rather difficult. Recently prices have shown lack of seasonality and volatility, which is bad for traders and an obstacle for PtG, too. Apart from recent market developments and drawbacks, I think PtG is a great technology as the production itself is CO2 free. Naively speaking one only needs wind power, water and access to a gas network, which in the most cases has already been established. Besides there are political initiatives to promote this technology with a certificate scheme, comparable to ElCerts in Scandinavia. This shift could give the PtG technology another economical boost, similar to the one experienced with renewables in Scandinavia.

German wind production data is freely provided by 50 Hertz. Other good data sources outside Germany are freely provided by Nordpool. Price data is given by Netztransparenz as well as Gaspool. If anyone of you out there, has a good, non-commercial alternative to wunderground, please drop me a mail.

Distribution fitting

Lately we discovered, that the assumptions of normality in the returns were a bit simplified. Hence, we stick to the family of exponential distributions, namely the Gamma and the Normal Inverse Gaussian distribution in order to enhance the quality of our forecasting results. Shortly we will call the first one NIG and the second one Gamma. These distributions should enrich our artificial simulated data so that we have a better fitting between real world and simulated data, especially when it comes to the observed tails.

Parameters necessary for both distributions were estimated by the moments matching method. When it comes to the Gamma distribution, we used a simple trick to displace the log production profile to the positive real numbers including zero. Otherwise the Gamma fit will not work. After successfully carrying out the fit, we transformed Gamma back to the original number range. The whole process should result in the following plots:

Illustration of NIG and Gamma fit

Again, purely optical we see a better fit between model and data in comparison to the former normal distribution hypothesis. Keep an eye on the tails, evidently both distributions behave contrary. The Gamma distribution slightly over-estimates the positive returns, however the NIG likes negative returns more. In order to reduce the complexity and length of this post, we will purely focus on Gamma.

The next step consists in simulating wind power production into the future with our new distributions, but this time we will make it “right”. Our new model will not only better match the tail behaviour of our production cycles, but also will incorporate jumps. We hope that this will be sufficient, so we do not have to model the wind power production volatility itself. The latter in my opinion states a real obstacle, as you really do not have any benchmark for wind production volatility as e.g. in the liquid equity universe the VIX Index for S&P 500 is. As already mentioned, I do not think that we have to do that, so let us introduce Levy processes.

The book is a very good read in regards to Levy processes and most of what follows are simply R implementations of what is rigorously described here.

Simulation with Gamma

Cut a long story short, we would like to simulate a mean reverting process, which is driven by a Gamma distributed Levy process. In general, this kind of models are formulated as follows:

$$\begin{align*} S(t) &= exp(f(t)+X(t)) \\ dX(t) &= -\alpha X(t)dt+dL(t) \\ f(t) &= a_0+\sum_{i=1}^6[a_i cos(2\pi\gamma_it)+b_i sin(2\pi\gamma_i t)] \\ \end{align*}$$

Easy things first, as usual $f(t)$ represents our seasonality function, but what about the ominous $L(t)$. As it is shown in another great paper by Benth, if $L(t)$ is a compound Poisson process with jumps distributed to $b$ and the Poisson process itself distributed to $a$, then the limiting distribution of our $X(t)$ is a Gamma distribution, shortly $\Gamma(a/\alpha, b)$. The parameter $a/\alpha$ is called shape, the parameter $b$ is called rate parameter. Hence in order to carry out some simulations we have to calibrate three parameters to real data. At first $\alpha$ is calculated easily by non-linear least squares of an exponential decreasing function $exp(-\alpha t)$ against $ACF(t)$, which corresponds to the empirical auto-correlation function of the data. Running the calculations, we arrive at $\alpha = 0.5589$ and again visually we recognize a reasonable fit between the empirical autocorrelation and the exponential function.

Fitting alpha to empirical autocorrelation function

After having achieved the estimation of the mean reversion speed, we proceed to rate and shape parameters of the Gamma distribution. These are simply fitted by moment matching and R has well established packages for doing that, so we do not spend time on details here.

After the execution of this calculation we are finally able to present some simulation results. Implementation of the algorithm in R is really straight forward, but you need a good routine for the Poisson process simulation. Nice algorithms are as well presented in the book of Wim Schouwers and in general we us antithetic sampling. As before we have calibrated our data to the years 2016/2017 and then will compare simulated to real data for the time from the beginning of 2018 till the end.

Monte-Carlo simulation of wind farm with gamma distribution

Brightly coloured, shivering lines represent hundred example paths of our Monte-Carlo simulation. The solid black line stands for real production figures of our 50Hertz data. What I like most is, that the fuzzy behaviour of wind production is now well reproduced by the simulation. If you have our old Gaussian approach in mind, which clearly suffered by underestimating the tails in the return distribution as well as the bumpy spikes in the production, we now see a clear fit achieved by the jumpy style of the Levy processes. I would really go as far as to say that one should always use generalized Ornstein-Uhlenbeck processes for simulating production for wind farms. But further evidence for this hypothesis should be given by analysing a lot of data. I really would like to do that and remembered wunderground as a good non-commercial source for wind speed data. But unfortunately they restricted the API to a commercial license. If on of you has a free alternative, please drop me a line.

Power to gas

After this short technical tour, let us return to the case study we promised in our introductory paragraph. We have seen, that there is a clear seasonal pattern and huge variability in our wind production, which often results in periods of harsh over-supply. What if we could convert this oversupply, which often occurs in Winter, somehow in a completely different commodity and sell it in summer. This idea naturally leads us to our power to gas plant.

Though the idea sounds great at first sight, practical implications are rather significant. First thing is, that technical circumstances for the conversion from power to natural gas are still limited. Much research effort has been conducted to improve the efficiency of the conversion, so that KIT working group HELMETH reports that 85% are technically feasible and that they achieved 75% in their testing power plant, which is quite good.

In our case the second problem involves the storage of the generated natural gas. In order to compensate low wind production in summer, one needs a storage facility, which is expensive. In addition, if one generates gas in winter having in mind to sell it in summer one naturally is exposed to long periods of storage costs. I would like to stress that this “problem” is only applicable in our case and there are other ways to embark PtG technology, which e.g. directly put the gas into the network. But in our case we need a storage solution to long for the promised stability in the renewables supply.

Besides all of these obstacles, let’s calculate an optimal strategy for pumping our over-supplied, wind produced renewables power to a natural gas storage device. Then selling certain amounts of gas into the gas network, when we have an undersupply situation in regards to our wind farm. The sketch of our idea is following. Our windpark is regulatory supported and has a daily production volume of approximately 154MWh per day. The EEG regime in 2018 has given wind produced power 6.792 ct/kWh. Obviously the easiest way would be to sell the power under the EEG regime, regardless of the production cycles. Consequently this would result in rather seasonal revenue schemes and periods where on has to put the windmills down due to strong wind with restrained demand. On the other hand we could say, that we do not want to sell all of our produced power in EEG, but convert it to natural gas and sell it later in the gas market. Therefore we introduce a certain threshold, which could e.g. correspond to the minimum revenue which we would like to achieve in 2018, based on what we had achieved in 2017. Everything above the threshold is converted into natural gas. The rest below the threshold is sold under normal EEG conditions. With this threshold we absorb the over-supply spikes and convert them into something what we can use, when the wind mills are in a period of under-supply.

As pointed out before the chemical side of power to gas conversion involves some production losses.Therefore we introduce the conversion efficiency ratio $0.5 \leq \eta \leq 0.85$, where the last value represents the efficiency value, which should be technically possible. By doing so we are able to convert our produced power with a given loss to natural gas. Unfortunately pipe size limitations from windmill to storage and from storage to gas network imply further limitations on the sales process, but more about this later.

Wind power scenarios are simulated with our Levy model developed above. Our aim is an optimal strategy, which walks the thin red line between monetizing power in EEG and converting power to gas, selling the latter on the gas spot market in Summer in order to compensate adverse revenue scenarios from EEG. At this stage it is a good point to state that this approach in a way differs from classical gas storage problems, where the optimization always takes place in respect to price scenarios. Usual storage problems utilise more or less the rule “buy cheap, sell high” and translate future price scenarios in inventory levels of the supply. Our case is one step before, but can of course be enhanced to also respect natural gas price scenarios when injecting or withdrawing gas from a bunker.

Optimisation presented here answers the question of how much power we should optimally convert in gas, given wind power scenarios simulated by our Levy model. The price itself is purely deterministic with EEG sales in Wind power case and natural gas forward prices given by settlement prices of respective futures. In the course of engaging myself with this subject, I came to the conclusion that it is feasible and desirable to enhance this optimisation in a way that also the natural gas price will be simulated. Furthermore we could place our wind park into Norway, where for a good reason also a lot of PtG is going to be installed. In this case we would not only have natural gas price scenarios to be respected, but also a simulation of power spot market should be carried out. I am going to deepen the here-presented analysis in upcoming posts towards this business case in order to show how this approach might work with full market exposure.

Putting the optimisation into a mathematical form, we take inspiration from the wonderful paper of Boogert and de Jong, which uses Least Squares Monte-Carlo approach to tackle the problem. Let us shortly sketch what the idea behind this technique is, by putting ourself into the role of the PtG power plant controller, who himself is controlled by our algorithm. As a first step we will force the mind of our controller into formal decision points $\pi(t,k)$ for a given timepoint $t$ and a Monte-Carlo wind power generation scenario $1\leq k\leq N$. Furthermore, he is dominated by the predefined threshold $L$, which represents the minimum revenue, which he is obliged to achieve throughout 2018. For each Monte-Carlo scenario $k$ the algorithm calculates the revenue from the EEG regime after subtracting variable costs, which e.g. from grid access costs. All simulated revenue scenarios are split into two sets. The first one $v_{wind,L>}(t,k)$ stands for potential revenues generated by the over-supply amount of the windmills. Second one $v_{wind,L\leq}(t,k)$ represents revenue from wind power below the threshold $L$. Now the controller is roughly speaking allowed to perform the following actions $\Delta g$ on the storage with supply level $g(t,k)$ at time $t$ and in wind scenario $k$: If the revenue is below or equal the threshold, he is allowed for one of the following actions $\Delta g$:

1. If gas is available in storage, sell an allowed amount to the gas network and sell total wind power under EEG
2. If gas storage is empty, do nothing, which is equivalent in selling the whole wind power amount in EEG

Hence the situation for 1. and 2. represents the under-supply situation. If we have a look at the real production data, this situation occurs sporadic on certain days in Winter, but on a continuous period in Summer. First allowed action (1.) should straighten this low production cycle with the gas sales.

On the other hand, we have the oversupply situation, where EEG revenues are above the threshold. Then the plant controller is allowed to perform one of the following steps:

1. convert some wind power amount into natural gas and store it in his bunker. Sell the rest of the wind power under EEG
2. do nothing
3. sell some amount of gas on the spot market and sell total wind power amount under EEG.

Honestly the situation is already complex, but in reality circumstances are a lot more subtle, because every action is limited by further constraints.

The chemical process of power to gas conversion is dominated by an efficiency gradient $\eta$ and moreover, there are other physical constraints to the gas supply, which further detail the allowance levels of the controller’s gas actions. The PtG controller can only inject up to 750MWh per day into the gas network and he is only allowed to convert 750 MWh at maximum per day into the gas storage nearby the windmills. Naturally the gas bunker itself has a maximum and minimum supply levels, which cannot be crossed by allowed actions of the plant controller. The Monte Carlo algortihm now proceeds by deploying an incentive function $C(t,k)$, which roughly speaking approximates the future cashflow the controller is going to receive from $t+1$. What would a rational controller do? Obviously he will try to maximise the potential future revenue given the described possible actions. In other words, if the algorithm has calculated all decision points $\pi(t,k)$, he forces the controller to take the decision, which represents the maximum of the decision in terms of potential future cashflows calculated on all actions that are allowed. Furthermore, this maximum decision will be translated into inventory levels, which then represent optimal storage parameters for the PtG plant.

Let us summarise, what we have written in the following decision tree:

Monte-Carlo decision tree, without costs

Let’s spend further words on the continuation function $C(t,k)$.

Having found this formal concept for our optimisation problem we are left with the calculation of the continuation values $C(t,k)$. This is where the least square regression comes into play. The technique says, that we can approximate the revenue at time $t+1$ through basis functions on the logarithm of produced wind power $(log(v_{wind}(t,-)))^i$ with $i=1,2,3$. With ordinary least squares regression, we get parameters $\beta_{t,i}$ for $i=1,2,3$: $logY(t+1,-) \approx 1+\beta_{(t,1)}logv_{wind}(t,-)+\beta_{(t,2)}logv_{wind}(t,-)^2+\beta_{(t,3)}logv_{wind}(t,-)^3$ These betas are now used to estimate the continuation value $C(t,k)$ and result in the so called incentive for the controller at time $t$ in scenario $k$. Technically, as continuation at time $t$ depends on cashflows one would receive at $t+1$, the algorithm in the end moves backwards in time. Hence the first step consists of simulating wind production scenarios on a daily grid $0<...<t<...<T$ with our Levy process and then move backward through the grid, starting with an initial cashflow $Y(T)$ as the basis for our regression algorithm. This last / intial cashflow is given by the minimum revenue threshold $L$, the controller has imposed on himself.

Given our incentive or continuation in each volume scenario, we technically fix timpoint $t$ and wind power scenario $k$. On this time-volume grid point we apply allowed volume actions, which are applicable given current supply level and wind power scenario. Recall that the controller is only allowed to apply allowed actions $\Delta g$, given by the physical storage constraints of the gas bunker and its supply lines. By respecting wind scenario $k$ at time $t$ and the constraints, the algorithm calculates a maximum amount, which can be converted in gas as well as a maximum amount, which can sold into the gas network. As the first is added to storage, the second one is removed from the storage the algortihm determines an upper bound $g_u(t,k)=-max(-\eta v_{wind}(t,k), -750MWh)$ and a lower bound $g_l(t,k)=-max(-g(t,k), -750MWh)$. We discretise this interval $[g_l,g_u]$ into 1000 equally sized pieces and calculate the possible outcome of each of these actions. In the end we take the one which promises the most cashflows given the before calculated continuation value.

Enough theory for now, let us have a look at simulation results. Naively starting the algorithm did not show desired results. In our price setting the controller does not really pump a lot of gas into the storage. In my first runs he was pumping small amounts (15 MWh) of gas into the supply and then instantly sold the total maximum allowed gas amount. Furthermore, in a lot of scenarios he did not do anything as for the short time horizon selling under EEG seemed profitable enough. This seems plausible to me, because due to technical limitations the controller is only allowed to sell small amounts of gas. Hence the gas sales does not give him that much expected compensation for low production / revenue cycles in wind in comparison to standard EEG sales. If the efficiency gradient slumps to levels of 50% the situation even worsens. How do we improve the situation?

First of all we set the efficiency ratio constantly to 75% percent, so that we are in a stable and good conversion setting. Then we penalise early gas sales. That means that the algorithm tells the controller to level up the gas inventory in oversupply seasons, by giving him a sales penalty determined by the expected revenue he is not going to get in Summer. This is brute force on the algorithm, giving the controller an hard gas sales stop in some seasons.

Now the controller constantly pumps about 50MWh per day into the storage and when allowed sales appropriate amounts. Cost of carry is also respected, as we have used a forward curve starting in early 2018 in contango and when time passes by we slope down into the backwardation. Not to mention that the algorithm is highly sensitive to the constraints. If we stick to the line boundaries described in the decision tree, the controller instantly sells the maximum amount possible if the penalty is not around. If we lower this allowed amount from 750MWh to 250MWh the controller will sell gas more smoothly in Summer. Playing around with the algortihm boundaries yields more or less the same optimal inventory levels.

Optimal inventory levels with different optimisation constraints

The algorithm misses its ultimate target. In a perfect setting, the controller should have constantly sold amounts of gas in Summer in order to really compensate low wind production cycles. The controller does its job right, but unfortunately the sold gas amounts are too small to smooth the wind production out. Of course, further incentives on the gas sales like a constant spread on the gas price improve the overall situation, but it is not really enough. In addition, we see that it is not attractive for him to pump further gas into the supply after he has sold it close to empty.

This is a hint, that we really have to respect natural gas price and volume simulation scenarios. Otherwise with a constant gas price, the controller has no real incentive to store gas. But if we simulate the gas price with a lot of different scenarios, the controller will convert wind to gas in low price scenarios and empty the supply in high price scenarios. Translating this principle in a simulation setting, would give the results more versatile outcomes and a more steady sales and pumping flow.

The same holds true solely for the wind farm. In a constant EEG scenario, there is no real incentive to convert a lot of gas in the short term horizon, because EEG revenue is sufficient. Only if we artificially penalise the controller in respective oversupply seasons, he is forced to convert gas into the bunker. Consequently we have to simulate production, power price and natural gas. In my opinion this finding further indicates, that the e.g. Scandinavia turns out to be an attractive region for the PtG technology. However, from a mathematical perspective we will have to respect a lot of different stochastic factors and their correlation, which turns out to be a ambitious task.

Key takeaways and further steps

From a green storage perspective PtG is very interesting, but we have discovered that - at least - in the setting described here it turns out to be a difficult task. There are certainly limitations and still some obstacles to cross, but given an established gas network and power price spot sales like in Scandinavia it should be more attractive. Returning to our thoughts about backtesting in our initial paragraph, we managed to understand that by doing a lot of backtesting.

For my part further investigations with PtG, especially when one would like to form a gas storage contract out of it, are very appealing. After further fine-tuning the algorithm, it might be an interesting and surely a green alternative to the more traditional gas storage contract.