A Transformer-Convolutional Neural Network Based Framework for Predicting Ionic Liquid Properties

Abstract

Of central importance to evaluate the suitability of ionic liquids (ILs) for a process is the accurate estimation of IL properties related to target performances. In this work, a versatile deep learning method for predicting IL properties is developed. Molecular fingerprints are derived from the encoder state of a Transformer model pre-trained on the PubChem database, which allows transfer learning from large-scale unlabeled data and significantly improves generalization performance for developing models with small datasets. Employing the pre-trained molecular fingerprints, convolutional neural network (CNN) models for IL properties prediction are trained and tested on 11 databases. The obtained Transformer-CNN models present superior performance to state-of-the-art models in all cases and enable property prediction of millions of ILs shortly. The application of the proposed models is exemplified by searching CO2 absorbent from a huge database of 8,333,096 synthetically feasible ILs, which is by far the most high-throughput IL screening in literature.