Mixture-of-Experts (MoE) models can achieve promising results with outrageous large amount of parameters but constant computation cost, and thus it has become a trend in model scaling. Still it is a mystery how MoE layers bring quality gains by leveraging the parameters with sparse activation. In this work, we investigate several key factors in sparse expert models. We observe that load imbalance may not be a significant problem affecting model quality, contrary to the perspectives of recent studies [9; 5; 10], while the number of sparsely activated experts k and expert capacity C in top-k routing can significantly make a difference in this context. Furthermore, we take a step forward to propose a simple method called expert prototyping that splits experts into different prototypes and applies k top-1 routing. This strategy improves the model quality but maintains constant computational costs, and our further exploration on extremely large-scale models reflects that it is more effective in training larger models. We push the model scale to over 1 trillion parameters and implement it on solely 480 NVIDIA V100-32GB GPUs, in comparison with the recent SOTAs [9; 5] on 2048 TPU cores. The proposed giant model achieves substantial speedup in convergence over the same-size baseline.
Objectives:
Disaster literacy, which is a concept involved in moral quality, knowledge, and ability, impacts the disaster response. This study aimed to evaluate disaster education needs and to determine the affecting factors of disaster literacy among college students.
Methods:
A cross-sectional study was conducted through questionnaires. A total of 7200 college students from 10 colleges and universities in 5 provinces and cities in China were investigated.
Results:
Disaster education needs were urgently. For the participants, 79.43% stated that their families were not prepared for disaster prevention items, 96.36% suggested a campus emergency rescue team, and 88.64% obtained disaster knowledge through TV or Internet, and only 12.89% had offered relevant courses in their disciplines. The total scores of college students’ disaster literacy were 87.85 ± 19.86, which was at a low level. Multiple linear regression analysis showed that age, major, grade, place of residence, parental education, mother’s occupation, disaster experience, disaster training experience, and family disaster preparedness significantly affected college students’ disaster literacy, especially grade, family disaster preparedness, and place of residence.
Conclusions:
College students’ disaster literacy education is urgently needed. Families, communities, colleges and the government should attach importance to disaster literacy education for college students.
Climate change becomes increasingly more relevant in the context of water systems planning. Tools are necessary to provide the most economic investment option considering the reliability of the infrastructure from technical and environmental perspectives. Accordingly, in this work, an optimisation approach, formulated as a spatially-explicit multi-period Mixed Integer Linear Programming (MILP) model, is proposed for the design of water supply chains at regional and national scales. The optimisation framework encompasses decisions such as installation of new purification plants, capacity expansion, and raw water trading schemes. The objective is to minimise the total cost incurring from capital and operating expenditures. Assessment of available resources for withdrawal is performed based on hydrological balances, governmental rules and sustainable limits. In the light of the increasing importance of reliability of water supply, a second objective, seeking to maximise the reliability of the supply chains, is introduced. The epsilon-constraint method is used as a solution procedure for the multi-objective formulation. Nash bargaining approach is applied to investigate the fair trade-offs between the two objectives and find the Pareto optimality. The models' capability is addressed through a case study based on Australia. The impact of variability in key input parameters is tackled through the implementation of a rigorous global sensitivity analysis (GSA). The findings suggest that variations in water demand can be more disruptive for the water supply chain than scenarios in which rainfalls are reduced. The frameworks can facilitate governmental multi-aspect decision making processes for the adequate and strategic investments of regional water supply infrastructure.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.