To address challenges of assessing space weather modeling capabilities, the Community Coordinated Modeling Center is leading a newly established International Forum for Space Weather Modeling Capabilities Assessment. This paper presents preliminary results of validation of modeled foF2 (F 2 layer critical frequency) and TEC (total electron content) during the first selected 2013 March storm event (17 March 2013). In this study, we used eight ionospheric models ranging from empirical to physics-based, coupled ionosphere-thermosphere and data assimilation models. The quantities we considered are TEC and foF2 changes and percentage changes compared to quiet time background, and the maximum and minimum percentage changes. In addition, we considered normalized percentage changes of TEC. We compared the modeled quantities with ground-based observations of vertical Global Navigation Satellite System TEC (provided by Massachusetts Institute of Technology Haystack Observatory) and foF2 data (provided by Global Ionospheric Radio Observatory) at the 12 locations selected in middle latitudes of the American and European-African longitude sectors. To quantitatively evaluate the models' performance, we calculated skill scores including correlation coefficient, root-mean square error (RMSE), ratio of the modeled to observed maximum percentage changes (yield), and timing error. Our study indicates that average RMSEs of foF2 range from about 1 MHz to 1.5 MHz. The average RMSEs of TEC are between~5 and~10 TECU (1 TEC Unit = 10 16 el/m 2 ). dfoF2[%] RMSEs are between 15% and 25%, which is smaller than RMSE of dTEC[%] ranging from 30% to 60%. The performance of the models varies with the location and metrics considered.To address the needs and challenge of assessment of our current knowledge about space weather effects on IT system and current state of IT modeling capabilities, the Community Coordinated Modeling Center (CCMC) has been supporting community-wide model validation projects, including Coupling, Energetics and Dynamics of Atmospheric Regions (CEDAR) and Geospace Environment Modeling (GEM)-CEDAR modeling challenges. The CCMC initiated the CEDAR electrodynamics thermosphere ionosphere challenge in 2009 focusing on the evaluation of basic IT system parameters modeled, such as electron and neutral densities, the ionospheric F 2 layer peak electron density (NmF2) and peak height (hmF2), and vertical drift (Shim Key Points: • foF2/TEC and foF2/TEC changes during a storm predicted by eight ionosphere models were compared with GIRO foF2 and GPS TEC measurements • Skill scores (e.g., correlation coefficient, RMSE, yield, and timing error) were calculated • Model performance strongly depends on the quantities considered, the type of metrics used, and the location considered Supporting Information: • Supporting Information S1 Correspondence to: J. S. Shim, jasoon.shim@nasa.gov Citation: Shim, J. S., Tsagouri, I., Goncharenko, L., Rastaetter, L., Kuznetsova, M., Bilitza, D., et al. (2018). Validation of ionospheric specifications du...

Abstract. High-precision (4).

No abstract

In order to assess current modeling capability of reproducing storm impacts on total electron content (TEC), we considered quantities such as TEC, TEC changes compared to quiet time values, and the maximum value of the TEC and TEC changes during a storm. We compared the quantities obtained from ionospheric models against ground‐based GPS TEC measurements during the 2006 AGU storm event (14–15 December 2006) in the selected eight longitude sectors. We used 15 simulations obtained from eight ionospheric models, including empirical, physics‐based, coupled ionosphere‐thermosphere, and data assimilation models. To quantitatively evaluate performance of the models in TEC prediction during the storm, we calculated skill scores such as RMS error, Normalized RMS error (NRMSE), ratio of the modeled to observed maximum increase (Yield), and the difference between the modeled peak time and observed peak time. Furthermore, to investigate latitudinal dependence of the performance of the models, the skill scores were calculated for five latitude regions. Our study shows that RMSE of TEC and TEC changes of the model simulations range from about 3 TECU (total electron content unit, 1 TECU = 1016 el m−2) (in high latitudes) to about 13 TECU (in low latitudes), which is larger than latitudinal average GPS TEC error of about 2 TECU. Most model simulations predict TEC better than TEC changes in terms of NRMSE and the difference in peak time, while the opposite holds true in terms of Yield. Model performance strongly depends on the quantities considered, the type of metrics used, and the latitude considered.

No abstract