Bioplastic Design using Multitask Deep Neural Networks

Abstract

Non-degradable plastic waste stays for decades on land and in water, jeopardizing our environment; yet our modern lifestyle and current technologies are impossible to sustain without plastics. Bio-synthesized and biodegradable alternatives such as the polymer family of polyhydroxyalkanoates (PHAs) have the potential to replace large portions of the world's plastic supply with cradle-to-cradle materials, but their chemical complexity and diversity limit traditional resource-intensive experimentation. In this work, we develop multitask deep neural network property predictors using available experimental data for a diverse set of nearly 23 000 homo-and copolymer chemistries.

polyBERT: A chemical language model to enable fully machine-driven ultrafast polymer informatics

“…Each of the 7 456 copolymer data points involves two distinct comonomers at various compositions. All data points in the data set have been used in past studies 6,7,11,[34][35][36][37][38][39][40][41] and were produced using computational methods or obtained from literature and other public sources. Supplementary Figures S3-S8 show Third, polyBERT masks 15 % (default parameter for masked language models) of the tokens to create a self-supervised training task.…”

“…As discussed recently, 6,11 we sum the composition-weighted polymer fingerprints to compute copolymer fingerprints F = N i F i c i , where N is the number of comonomers in the copolymer, F i the i th comonomer fingerprint, and c i the fraction of the i th comonomer. This approach renders copolymer fingerprints invariant to the order in which one may sort the comonomers and satisfies the two main demands of uniqueness and invariance to different (but equivalent) periodic unit specifications.…”

“…The training protocol of the concatenation-conditioned multitask predictors follows state-of-theart techniques involving five-fold cross-validation and a meta learner that forecasts the final property values based upon the ensemble of cross-validation predictors. 6,7,11 Supplementary Figure S2 details this process. After shuffling, we split the data set into two parts and use 80 % for the five cross-validation models and for validating the meta learners.…”

“…The nascent field of polymer informatics [2][3][4][5] allows access to the depth of the polymer universe and demonstrates the potency of machine learning (ML) models to overcome this challenge. ML frameworks have enabled substantial progress in the development of polymer property predictors [6][7][8][9][10] and solving inverse problems in which polymers that meet specific property requirements are either identified from candidate sets, 11,12 or are freshly designed using genetic 13,14 or generative [15][16][17] algorithms.…”

“…c Minimum, mean, and maximum of polyBERT-based property predictions for 100 million hypothetical polymers. Symbols are defined in Table1.for polymer property predictions,6,7,11 while being fast, scalable, and readily amenable if more data points become available. Unlike single-task models, multitask models simultaneously predict numerous properties (tasks) and harness inherent but hidden correlations in data to improve their performance.…”