Motivation: Biomedical text mining is becoming increasingly important as the number of biomedical documents rapidly grows. With the progress in natural language processing (NLP), extracting valuable information from biomedical literature has gained popularity among researchers, and deep learning has boosted the development of effective biomedical text mining models. However, directly applying the advancements in NLP to biomedical text mining often yields unsatisfactory results due to a word distribution shift from general domain corpora to biomedical corpora. In this article, we investigate how the recently introduced pre-trained language model BERT can be adapted for biomedical corpora. Results: We introduce BioBERT (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining), which is a domain-specific language representation model pre-trained on large-scale biomedical corpora. With almost the same architecture across tasks, BioBERT largely outperforms BERT and previous state-of-the-art models in a variety of biomedical text mining tasks when pre-trained on biomedical corpora. While BERT obtains performance comparable to that of previous state-of-the-art models, BioBERT significantly outperforms them on the following three representative biomedical text mining tasks: biomedical named entity recognition (0.62% F1 score improvement), biomedical relation extraction (2.80% F1 score improvement) and biomedical question answering (12.24% MRR improvement). Our analysis results show that pre-training BERT on biomedical corpora helps it to understand complex biomedical texts. Availability and implementation: We make the pre-trained weights of BioBERT freely available at https://github. com/naver/biobert-pretrained, and the source code for fine-tuning

Lithium-ion batteries are now reaching the energy density limits set by their electrode materials, requiring new paradigms for Li(+) and electron hosting in solid-state electrodes. Reversible oxygen redox in the solid state in particular has the potential to enable high energy density as it can deliver excess capacity beyond the theoretical transition-metal redox-capacity at a high voltage. Nevertheless, the structural and chemical origin of the process is not understood, preventing the rational design of better cathode materials. Here, we demonstrate how very specific local Li-excess environments around oxygen atoms necessarily lead to labile oxygen electrons that can be more easily extracted and participate in the practical capacity of cathodes. The identification of the local structural components that create oxygen redox sets a new direction for the design of high-energy-density cathode materials.

Nearly all high-energy density cathodes for rechargeable lithium batteries are well-ordered materials in which lithium and other cations occupy distinct sites. Cation-disordered materials are generally disregarded as cathodes because lithium diffusion tends to be limited by their structures. The performance of Li1.211Mo0.467Cr0.3O2 shows that lithium diffusion can be facile in disordered materials. Using ab initio computations, we demonstrate that this unexpected behavior is due to percolation of a certain type of active diffusion channels in disordered Li-excess materials. A unified understanding of high performance in both layered and Li-excess materials may enable the design of disordered-electrode materials with high capacity and high energy density.

There is an urgent need for low-cost, resource-friendly, high-energy-density cathode materials for lithium-ion batteries to satisfy the rapidly increasing need for electrical energy storage. To replace the nickel and cobalt, which are limited resources and are associated with safety problems, in current lithium-ion batteries, high-capacity cathodes based on manganese would be particularly desirable owing to the low cost and high abundance of the metal, and the intrinsic stability of the Mn oxidation state. Here we present a strategy of combining high-valent cations and the partial substitution of fluorine for oxygen in a disordered-rocksalt structure to incorporate the reversible Mn/Mn double redox couple into lithium-excess cathode materials. The lithium-rich cathodes thus produced have high capacity and energy density. The use of the Mn/Mn redox reduces oxygen redox activity, thereby stabilizing the materials, and opens up new opportunities for the design of high-performance manganese-rich cathodes for advanced lithium-ion batteries.

A unifying theory is presented to explain the lithium exchange capacity of rocksalt‐like structures with any degree of cation ordering, and how lithium percolation properties can be used as a guideline for the development of novel high‐capacity electrode materials is demonstrated. The lithium percolation properties of the three most common lithium metal oxide phases, the layered α‐NaFeO2 structure, the spinel‐like LT‐LiCoO2 structure, and the γ‐LiFeO2 structure, are demonstrated and a strong dependence of the percolation thresholds on the cation ordering and the lithium content is observed. The poor performance of γ‐LiFeO2‐type structures is explained by their lack of percolation of good Li migration channels. The spinel‐like structure exhibits excellent percolation properties that are robust with respect to off‐stoichiometry and some amount of cation disorder. The layered structure is unique, as it possesses two different types of lithium diffusion channels, one of which is, however, strongly dependent on the lattice parameters, and therefore very sensitive to disorder. In general it is found that a critical Li‐excess concentration exists at which Li percolation occurs, although the amount of Li excess needed depends on the partial cation ordering. In fully cation‐disordered materials, macroscopic lithium diffusion is enabled by ≈10% excess lithium.

Percolation theory enables the design of high capacity cation-disordered oxides for rechargeable lithium battery cathodes.

Recent progress in the understanding of percolation theory points to cation-disordered lithium-excess transition metal oxides as high-capacity lithium-ion cathode materials. Nevertheless, the oxygen redox processes required for these materials to deliver high capacity can trigger oxygen loss, which leads to the formation of resistive surface layers on the cathode particles. We demonstrate here that, somewhat surprisingly, fluorine can be incorporated into the bulk of disordered lithium nickel titanium molybdenum oxides using a standard solid-state method to increase the nickel content, and that this compositional modification is very effective in reducing oxygen loss, improving energy density, average voltage, and rate performance. We argue that the valence reduction on the anion site, offered by fluorine incorporation, opens up significant opportunities for the design of high-capacity cation-disordered cathode materials.

No abstract