Transformer-based deep learning models for predicting permeability of porous media

A Physics-Enhanced Neural Network for Estimating Longitudinal Dispersion Coefficient and Average Solute Transport Velocity in Porous Media

“…On the other hand, Transformer networks, with their self-attention mechanism, proficient in capturing long-range dependencies, enhancing feature extraction and generalization when combined with CNNs (Bai & Tahmasebi, 2022;Vaswani et al, 2017). This integrated CNN-Transformer approach achieves high accuracy with fewer parameters, proving advantageous in computational resource-limited settings and facilitating research on large-sized porous media (Meng et al, 2023). Moreover, the infusion of physical information into the network-permitting the direct assimilation of porous media's physical parameters during training-markedly elevates the model's predictive precision and its generalization capacity (Kamrava, Im, et al, 2021;Meng et al, 2023;Tang et al, 2022).…”

“…This integrated CNN-Transformer approach achieves high accuracy with fewer parameters, proving advantageous in computational resource-limited settings and facilitating research on large-sized porous media (Meng et al, 2023). Moreover, the infusion of physical information into the network-permitting the direct assimilation of porous media's physical parameters during training-markedly elevates the model's predictive precision and its generalization capacity (Kamrava, Im, et al, 2021;Meng et al, 2023;Tang et al, 2022). In this letter, we propose the Physics-enhanced Convolutional Neural Network-Transformer (PhysenCT-Net), designed for the prediction of parameters D L and u in large-sized (200 × 200 × 1,000 cubic voxels) porous media flows.…”