Towards a standardized procedure to assess solar forecast accuracy: A new ramp and time alignment metric

Vallance, Loïc; Charbonnier, Bruno; Paul, Nicolas; Dubost, Stéphanie; Blanc, Philippe

doi:10.1016/j.solener.2017.04.064

Cited by 65 publications

(50 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MBE and RMSE are conventional statistical metrics widely used in the SR forecasting field for characterizing and validating forecasting models. These scores depend on several factors such as comparing spatial and temporal resolution of the data, the percentage of clear-sky days in the set of data, and the months used in the comparison, and the limitations of these metrics have been shown in [74]. Although these metrics provide comprehensive global information on forecast errors, it can be misleading and insufficient to characterize the behavior of the forecast.…”

Section: Discussionmentioning

confidence: 99%

Solar Energy Estimations in India Using Remote Sensing Technologies and Validation with Sun Photometers in Urban Areas

et al. 2020

View full text Add to dashboard Cite

Solar radiation ground data is available in poor spatial resolution, which provides an opportunity and demonstrates the necessity to consider solar irradiance modeling based on satellite data. For the first time, solar energy monitoring in near real-time has been performed for India. This study focused on the assessment of solar irradiance from the Indian Solar Irradiance Operational System (INSIOS) using operational cloud and aerosol data from INSAT-3D and Copernicus Atmosphere Monitoring Service (CAMS)-Monitoring Atmospheric Composition Climate (MACC), respectively. Simulations of the global horizontal irradiance (GHI) and direct normal irradiance (DNI) were evaluated for 1 year for India at four Baseline Surface Radiation Network (BSRN) stations located in urban regions. The INSIOS system outputs as per radiative transfer model results presented high accuracy under clear-sky and cloudy conditions for GHI and DNI. DNI was very sensitive to the presence of cloud and aerosols, where even with small optical depths the DNI became zero, and thus it affected the accuracy of simulations under realistic atmospheric conditions. The median BSRN and INSIOS difference was found to vary from −93 to −49 W/m2 for GHI and −103 to −76 W/m2 for DNI under high solar energy potential conditions. Clouds were able to cause an underestimation of 40%, whereas for various aerosol inputs to the model, the overall accuracy was high for both irradiances, with the coefficient of determination being 0.99, whereas the penetration of photovoltaic installation, which exploits GHI, into urban environments (e.g., rooftop) could be effectively supported by the presented methodology, as estimations were reliable during high solar energy potential conditions. The results showed substantially high errors for monsoon season due to increase in cloud coverage that was not well-predicted at satellite and model resolutions.

show abstract

Section: Discussionmentioning

confidence: 99%

Solar Energy Estimations in India Using Remote Sensing Technologies and Validation with Sun Photometers in Urban Areas

et al. 2020

View full text Add to dashboard Cite

show abstract

“…By using the RFs trained as detailed in the previous section, all the samples in the validation dataset were used to estimate the vertical profiles, which were subsequently compared against the vertical profiles from the three data sources. Figure 7 shows the mean T of the estimated vertical profile; it also shows the corresponding mean bias error (MBE) and RMSE [61] compared with the standard deviation (STD) with respect to ERA-Interim, IASI L2 v6, and IGRA, separately and to their average. The average profile was calculated by using the same weight for each of ERA-Interim, IASI L2 v6, and IGRA, in order to obtain a general evaluation of the performance of the algorithm that did not take into account the different size of the three sources of data.…”

Section: Resultsmentioning

confidence: 99%

MiRTaW: An Algorithm for Atmospheric Temperature and Water Vapor Profile Estimation from ATMS Measurements Using a Random Forests Technique

et al. 2018

View full text Add to dashboard Cite

A new algorithm for the estimation of atmospheric temperature (T) and water vapor (WV) vertical profiles in nonprecipitating conditions is presented. The microwave random forest temperature and water vapor (MiRTaW) profiling algorithm is based on the random forest (RF) technique and it uses microwave (MW) sounding from the Advanced Technology Microwave Sounder (ATMS) onboard the Suomi National Polar-orbiting Partnership (SNPP) satellite. Three different data sources were chosen for both training and validation purposes, namely, the ERA-Interim from the European Centre for Medium-Range Weather Forecasts (ECMWF), the Infrared Atmospheric Sounding Interferometer Atmospheric Temperature Water Vapour and Surface Skin Temperature (IASI L2 v6) from the Meteorological Operational satellites of the European Organization for the Exploitation of Meteorological Satellites (EUMETSAT), and the radiosonde observations from the Integrated Global Radiosonde Archive (IGRA). The period from 2012 to 2016 was considered in the training dataset; particular attention was paid to the instance selection procedure, in order to reduce the full training dataset with negligible information loss. The out-of-bag (OOB) error was computed and used to select the optimal RF parameters. Different RFs were trained, one for each vertical level: 32 levels for T (within 10–1000 hPa) and 23 levels for WV (200–1000 hPa). The validation of the MiRTaW profiling algorithm was conducted on a dataset from 2017. The mean bias error (MBE) of T vertical profiles ranges within about (−0.4–0.4) K, while for the WV mixing ratio, the MBE starts at ~0.5 g/kg near the surface and decreases to ~0 g/kg at 200 hPa level, in line with the expectations.

show abstract

“…In other cases, new metrics are proposed to meet a specific objective. This is exemplified by the work of Vallance et al (2017), in which the ability to forecast ramps in irradiance transients is gauged by two new metrics.…”

Section: What Is a Good Forecast?mentioning

confidence: 99%

“…By presenting a wide spectrum of error metrics, forecasters are able to choose freely among the metrics that can "best" highlight the strengths of their results. There are many studies that propose, contrast, and recommend error metrics to forecasters (e.g., Vallance et al, 2017;Zhang et al, 2015;Hoff et al, 2013;Beyer et al, 2009). However, despite the well-argued discussions, these works can rarely change another forecaster's sentiment towards some specific metrics, if they are perceived as having important advantages or disadvantages.…”

Section: What Is a Good Forecast?mentioning

confidence: 99%

Verification of deterministic solar forecasts

et al. 2020

View full text Add to dashboard Cite

The field of energy forecasting has attracted many researchers from different fields (e.g., meteorology, data sciences, mechanical or electrical engineering) over the last decade. Solar forecasting is a fast-growing subdomain of energy forecasting. Despite several previous attempts, the methods and measures used for verification of deterministic (also known as single-valued or point) solar forecasts are still far from being standardized, making forecast analysis and comparison difficult. To analyze and compare solar forecasts, the well-established Murphy-Winkler framework for distribution-oriented forecast verification is recommended as a standard practice. This framework examines aspects of forecast quality, such as reliability, resolution, association, or discrimination, and analyzes the joint distribution of forecasts and observations, which contains all time-independent information relevant to verification. To verify forecasts, one can use any graphical display or mathematical/statistical measure to provide insights and summarize the aspects of forecast quality. The majority of graphical methods and accuracy measures known to solar forecasters are specific methods under this general framework. Additionally, measuring the overall skillfulness of forecasters is also of general interest. The use of the root mean square error (RMSE) skill score based on the optimal convex combination of climatology and persistence methods is highly recommended. By standardizing the accuracy measure and reference forecasting method, the RMSE skill score allows-with appropriate caveats-comparison of forecasts made using different models, across different locations and time periods.

show abstract

Towards a standardized procedure to assess solar forecast accuracy: A new ramp and time alignment metric

Cited by 65 publications

References 25 publications

Solar Energy Estimations in India Using Remote Sensing Technologies and Validation with Sun Photometers in Urban Areas

Solar Energy Estimations in India Using Remote Sensing Technologies and Validation with Sun Photometers in Urban Areas

MiRTaW: An Algorithm for Atmospheric Temperature and Water Vapor Profile Estimation from ATMS Measurements Using a Random Forests Technique

Verification of deterministic solar forecasts

Contact Info

Product

Resources

About