MM-RNN: A Multimodal RNN for Precipitation Nowcasting

Ma, Zhifeng; Zhang, Hao; Liu, Jie

doi:10.1109/tgrs.2023.3264545

Cited by 19 publications

(6 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, through visual analysis, it is evident that compared to ConvLSTM, the MultiPred model produces clearer boundaries in its predictive results and captures light and moderate precipitation more sensitively. This aligns with the viewpoint in Ma et al's research [28] regarding the enhancement of model accuracy through multimodal fusion structures. Finally, short-term precipitation forecasting models commonly face the challenge of insufficient accuracy in precipitation datasets in practical applications, especially when predicting heavy precipitation using satellite precipitation data.…”

Section: Discussionsupporting

confidence: 88%

See 1 more Smart Citation

Improving Precipitation Forecasting through Early Fusion and Spatiotemporal Prediction: A Case Study Using the MultiPred Model

Wang,

Tian,

Fang

et al. 2024

Atmosphere

View full text Add to dashboard Cite

Due to the complexity and uncertainty of meteorological systems, traditional precipitation forecasting methods have certain limitations. Therefore, based on the common characteristics of meteorological data, a precipitation forecasting model named MultiPred is proposed, with the goal of continuously predicting precipitation for 4 h in a specific region. This model combines the multimodal fusion method with recursive spatiotemporal prediction models. The training and testing process of the model roughly involves using spatial feature extraction networks and temporal feature extraction networks to generate preliminary predictions for multimodal data. Subsequently, a modal fusion layer is employed to further extract and fuse the spatial features of the preliminary predictions from the previous step, outputting the predicted precipitation values for the target area. Experimental tests and training were conducted using ERA5 multi-meteorological modal data and GPM satellite precipitation data from 2017 to 2020, covering longitudes from 110° to 122° and latitudes from 20° to 32°. The training set used data from the first three years, while the validation set and test set each comprised 50% of the data from the fourth year. The initial learning rate for the experiment was set to 1 × 10−4, and training was performed for 1000 epochs. Additionally, the training process utilized a loss function composed of Mean Absolute Error (MAE), Mean Squared Error (MSE), and Structural Similarity Index (SSIM). The model was evaluated using the Critical Success Index (CSI), Probability of Detection (POD), and the Heidke Skill Score (HSS). Experimental results demonstrate that MultiPred excels in precipitation forecasting, particularly for light precipitation events with amounts greater than or equal to 0.1 mm and less than 2 mm. It achieves optimal performance in both light and heavy precipitation forecasting tasks.

show abstract

Section: Discussionsupporting

confidence: 88%

“…In late fusion models, Ma et al [28] proposed a late fusion recurrent neural network, which not only provides accurate short-term precipitation forecasts but also predicts other meteorological elements. It demonstrates high flexibility and compatibility with various recurrent neural network models.…”

Section: Multimodal Fusionmentioning

confidence: 99%

Improving Precipitation Forecasting through Early Fusion and Spatiotemporal Prediction: A Case Study Using the MultiPred Model

Wang,

Tian,

Fang

et al. 2024

Atmosphere

View full text Add to dashboard Cite

show abstract

“…This algorithm seeks to maximize the frame-level crossentropy between the actual and projected probability distributions concerning class labels. The advent of RNNs (Recurrent Neural Networks), as referenced in [33], has brought about a transformative impact on the landscape of data processing. These networks introduce a potent instrument for managing sequential and time-dependent data.…”

Section: Deep Belief Networkmentioning

confidence: 99%

“…Our approach integrated several traditional algorithms, namely Tuna Swarm Optimization (TSO)-DB-RNN [27], Beluga Whale Optimization (BWO)-DB-RNN [28], Cuttle Fish Optimization (CO)-DB-RNN [29], and Billiards-Inspired Optimization (BIO)-DB-RNN [26], into the proposed model. Additionally, we conducted a comprehensive comparison of classifiers, including RBF [30], CNN-DNN [31], CNN [5], DBN [32], RNN [33], and ICBPOA-ECDNN. For our experiments, set the population size at 10, established a maximum iteration limit of 50, and defined a chromosome length of 1.…”

Section: Simulation Setupmentioning

confidence: 99%

A novel time series classification for multivariate data using improved deep belief-recurrent neural network with optimal dynamic time warping

Mohan Babu,

Sandhya

2024

MATEC Web Conf.

View full text Add to dashboard Cite

In the past ten years, data from time series extraction has attracted a lot of attention. Several methods have concentrated on classification problems, where the objective is to identify the labelling of a test period, given labelled training data. Feature-based and Instance-based methods are the two fundamental groups into which time series categorization methodologies may be divided. To categorize time series data, instance-based techniques use similarity data in a nearest-neighbor context. While methods in this category deliver reliable findings, their efficacy suffers when dealing with lengthy and noisy time series. Feature-based approaches, on the other together, extract characteristics to address the shortcomings of instance-based methods; nevertheless, these approaches use predetermined features and might not be effective in all classification issues. This paper seeks to introduce a novel deep learning-based Optimal Dynamic Time Warping (ODTW) paradigm for multimodal time’s series data categorization. This model covers several phases. At initial stage, the standard data is gathered from standard public source. Secondly, ODTW is proposed, where the parameters are optimized by Random Opposition Billiards-Inspired Optimization (RO-BIO) for extracting the most essential information. Finally, the classification is carried out through “Deep Belief Network (DBN) and Recurrent Neural Networks (RNN) termed as Deep Belief-RNN (DB-RNN)”. Finally, the extracted deep features are given to the optimized RNN for attaining the final classified results. The simulation results have resulted in superior classification performance in terms of standard performance measures.

show abstract

“…Wu Haixu proposed MotionRNN [42] and designed MotionGRU to model overall motion trends and instantaneous changes cohesively. Additionally, there are other prediction models like CMS-LSTM [43], MS-LSTM [44], PrecipLSTM [45], and MM-RNN [46], all belonging to iterative prediction models based on recurrent neural networks. Currently, RNN-based precipitation prediction models lack consideration for meteorological features and do not make sufficient use of meteorological characteristics within precipitation.…”

Section: Related Workmentioning

confidence: 99%

METEO-DLNet: Quantitative Precipitation Nowcasting Net Based on Meteorological Features and Deep Learning

Hu,

Yin,

Guo

2024

Remote Sensing

View full text Add to dashboard Cite

Precipitation prediction plays a crucial role in people’s daily lives, work, and social development. Especially in the context of global climate variability, where extreme precipitation causes significant losses to the property of people worldwide, it is urgently necessary to use deep learning algorithms based on radar echo extrapolation for short-term precipitation forecasting. However, there are inadequately addressed issues with radar echo extrapolation methods based on deep learning, particularly when considering the inherent meteorological characteristics of precipitation on spatial and temporal scales. Additionally, traditional forecasting methods face challenges in handling local images that deviate from the overall trend. To address these problems, we propose the METEO-DLNet short-term precipitation prediction network based on meteorological features and deep learning. Experimental results demonstrate that the Meteo-LSTM of METEO-DLNet, utilizing spatial attention and differential attention, adequately learns the influence of meteorological features on spatial and temporal scales. The fusion mechanism, combining self-attention and gating mechanisms, resolves the divergence between local images and the overall trend. Quantitative and qualitative experiments show that METEO-DLNet outperforms current mainstream deep learning precipitation prediction models in natural spatiotemporal sequence problems.

show abstract

MM-RNN: A Multimodal RNN for Precipitation Nowcasting

Cited by 19 publications

References 38 publications

Improving Precipitation Forecasting through Early Fusion and Spatiotemporal Prediction: A Case Study Using the MultiPred Model

Improving Precipitation Forecasting through Early Fusion and Spatiotemporal Prediction: A Case Study Using the MultiPred Model

A novel time series classification for multivariate data using improved deep belief-recurrent neural network with optimal dynamic time warping

METEO-DLNet: Quantitative Precipitation Nowcasting Net Based on Meteorological Features and Deep Learning

Contact Info

Product

Resources

About