Principal Component Density Estimation for Scenario Generation Using Normalizing Flows

Abstract

Neural networks-based learning of the distribution of non-dispatchable renewable electricity generation from sources such as photovoltaics (PV) and wind as well as load demands has recently gained attention. Normalizing flow density models have performed particularly well in this task due to the training through direct log-likelihood maximization. However, research from the field of image generation has shown that standard normalizing flows can only learn smeared-out versions of manifold distributions and can result in the generation of noisy data. To avoid the generation of time series data with unrealistic noise, we propose a dimensionality-reducing flow layer based on the linear principal component analysis (PCA) that sets up the normalizing flow in a lower-dimensional space. We train the resulting principal component flow (PCF) on data of PV and wind power generation as well as load demand in Germany in the years 2013 to 2015. The results of this investigation show that the PCF preserves critical features of the original distributions, such as the probability density and frequency behavior of the time series. The application of the PCF is, however, not limited to renewable power generation but rather extends to any data set, time series, or otherwise, which can be efficiently reduced using PCA.

Nonlinear Isometric Manifold Learning for Injective Normalizing Flows

“…In this work, we generalize our work on PCAbased dimensionality reduction [10] to nonlinear embeddings by extending our findings to compositions of arbitrary isometric embeddings and normalizing flows. We show theoretically that the PDF described by an injective normalizing flow is invariant to isometric embeddings, in general, rather than just to the PCA, specifically.…”

“…Options on how to mitigate the computational cost are using approximate Jacobian determinants [8], identifying the Riemannian metric [22,23], or learning embeddings of Riemannian manifolds directly [17]. In our previous work [10], we highlighted that the Jacobian determinant of the embedding is always equal to one if the encoding is performed via PCA, i.e., the PDF in Equation ( 4) is invariant to a PCA encoding. In the following, we extend this finding to isometric embeddings, in general.…”

“…For completeness, this section reviews the use of PCA as affine isometric embedding for normalizing flows from our previous work [10]. For PCA, both the encoder g PCA and the decoder f PCA are affine transformations [25]:…”

“…Despite the advantage of likelihood maximization, normalizing flows often result in poor fits of the target probability distributions [8,9]. For instance, our previous work [10] has shown that the application of a standard normalizing flow model to energy time series data results in the sampling of uncharacteristically noisy data. The poor fits are a direct result of the structural setup of the normalizing flow that is ill-posed to describe probability distributions over lower-dimensional mani-folds [8,9].…”

“…In Cramer et al [10], we have addressed the manifold structure of renewable energy time series data via dimensionality reduction using the principal component analysis (PCA), which transforms the data into a latent space without a manifold structure. Our theoretical analysis highlighted that the PDF described by the normalizing flow is invariant to the PCA.…”