LayerOut: Freezing Layers in Deep Neural Networks

Goutam, Kelam; Balasubramanian, Siva; Gera, Darshan; Sarma, R. Raghunatha

doi:10.1007/s42979-020-00312-x

Cited by 17 publications

(10 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, in this study, only the dense layer or convolutional blocks were set to be trainable, followed by the output of three soybean tolerance classes. Other layers of each pre-trained model were frozen and their weights were not updated by the optimizer during training processing in order to reduce the risk of overfitting [ 38 ]. Different models have the same options for tuning parameters in order to compare them at the same level.…”

Section: Methodsmentioning

confidence: 99%

Estimation of Off-Target Dicamba Damage on Soybean Using UAV Imagery and Deep Learning

Tian

Vieira²,

Zhou

et al. 2023

Sensors

View full text Add to dashboard Cite

Weeds can cause significant yield losses and will continue to be a problem for agricultural production due to climate change. Dicamba is widely used to control weeds in monocot crops, especially genetically engineered dicamba-tolerant (DT) dicot crops, such as soybean and cotton, which has resulted in severe off-target dicamba exposure and substantial yield losses to non-tolerant crops. There is a strong demand for non-genetically engineered DT soybeans through conventional breeding selection. Public breeding programs have identified genetic resources that confer greater tolerance to off-target dicamba damage in soybeans. Efficient and high throughput phenotyping tools can facilitate the collection of a large number of accurate crop traits to improve the breeding efficiency. This study aimed to evaluate unmanned aerial vehicle (UAV) imagery and deep-learning-based data analytic methods to quantify off-target dicamba damage in genetically diverse soybean genotypes. In this research, a total of 463 soybean genotypes were planted in five different fields (different soil types) with prolonged exposure to off-target dicamba in 2020 and 2021. Crop damage due to off-target dicamba was assessed by breeders using a 1–5 scale with a 0.5 increment, which was further classified into three classes, i.e., susceptible (≥3.5), moderate (2.0 to 3.0), and tolerant (≤1.5). A UAV platform equipped with a red-green-blue (RGB) camera was used to collect images on the same days. Collected images were stitched to generate orthomosaic images for each field, and soybean plots were manually segmented from the orthomosaic images. Deep learning models, including dense convolutional neural network-121 (DenseNet121), residual neural network-50 (ResNet50), visual geometry group-16 (VGG16), and Depthwise Separable Convolutions (Xception), were developed to quantify crop damage levels. Results show that the DenseNet121 had the best performance in classifying damage with an accuracy of 82%. The 95% binomial proportion confidence interval showed a range of accuracy from 79% to 84% (p-value ≤ 0.01). In addition, no extreme misclassifications (i.e., misclassification between tolerant and susceptible soybeans) were observed. The results are promising since soybean breeding programs typically aim to identify those genotypes with ‘extreme’ phenotypes (e.g., the top 10% of highly tolerant genotypes). This study demonstrates that UAV imagery and deep learning have great potential to high-throughput quantify soybean damage due to off-target dicamba and improve the efficiency of crop breeding programs in selecting soybean genotypes with desired traits.

show abstract

Section: Methodsmentioning

confidence: 99%

Estimation of Off-Target Dicamba Damage on Soybean Using UAV Imagery and Deep Learning

Tian

Vieira²,

Zhou

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Thus, the appropriate learning rate depends on the model architecture and the training dataset. During the fine-tuning process, reducing the number of trainable layers with freezing some layers can shorten the training time since the number of parameters to be updated is reduced [23]. Xiao et al demonstrated that freezing some layers during training process might improve the model accuracy if the less updated layers are frozen [24].…”

Section: Fine-tuning Of Prediction Modelsmentioning

confidence: 99%

Toward Predictive Modeling of Solar Power Generation for Multiple Power Plants

Thonglek

Ichikawa

Takahashi

et al. 2023

IEICE Trans. Commun.

View full text Add to dashboard Cite

Solar power is the most widely used renewable energy source, which reduces pollution consequences from using conventional fossil fuels. However, supplying stable power from solar power generation remains challenging because it is difficult to forecast power generation. Accurate prediction of solar power generation would allow effective control of the amount of electricity stored in batteries, leading in a stable supply of electricity. Although the number of power plants is increasing, building a solar power prediction model for a newly constructed power plant usually requires collecting a new training dataset for the new power plant, which takes time to collect a sufficient amount of data. This paper aims to develop a highly accurate solar power prediction model for multiple power plants available for both new and existing power plants. The proposed method trains the model on existing multiple power plants to generate a general prediction model, and then uses it for a new power plant while waiting for the data to be collected. In addition, the proposed method tunes the general prediction model on the newly collected dataset and improves the accuracy for the new power plant. We evaluated the proposed method on 55 power plants in Japan with the dataset collected for two and a half years. As a result, the pre-trained models of our proposed method significantly reduces the average RMSE of the baseline method by 73.19%. This indicates that the model can generalize over multiple power plants, and training using datasets from other power plants is effective in reducing the RMSE. Fine-tuning the pre-trained model further reduces the RMSE by 8.12%.

show abstract

“…In summary, achieving a fast convergence and high accuracy requires keeping the trained layers in full precision (activation and parameters in the forward and backward pass). The work in [6] stochastically freezes layers of an NN to speed up training, keeping the frozen layers at full precision which limits the achievable speedup. Also, due to its stochastic nature, it is not applicable to a hard computation constraint.…”

Section: Quantization and Freezing In Centralizedmentioning

confidence: 99%

“…However, quantized gradient computation still suffers from reduced accuracy [7]. Another branch of work studies partial freezing of parameters during training to reduce the number of gradients to be computed [6]. The performance gains, however, are limited, especially if layers towards the beginning of the NN are trained, which requires expensive backpropagation through most layers.…”

Section: Introductionmentioning

confidence: 99%

CoCo-FL: Communication- and Computation-Aware Federated Learning via Partial NN Freezing and Quantization

Pfeiffer¹,

Rapp²,

Khalili³

et al. 2022

Preprint

View full text Add to dashboard Cite

Devices participating in federated learning (FL) typically have heterogeneous communication and computation resources. However, all devices need to finish training by the same deadline dictated by the server when applying synchronous FL, as we consider in this paper. Reducing the complexity of the trained neural network (NN) at constrained devices, i.e., by dropping neurons/filters, is insufficient as it tightly couples reductions in communication and computation requirements, wasting resources. Quantization has proven effective to accelerate inference, but quantized training suffers from accuracy losses. We present a novel mechanism that quantizes during training parts of the NN to reduce the computation requirements, freezes them to reduce the communication and computation requirements, and trains the remaining parts in full precision to maintain a high convergence speed and final accuracy. Using this mechanism, we present the first FL technique that independently optimizes for specific communication and computation constraints in FL: CoCo-FL. We show that CoCo-FL reaches a much higher convergence speed than the state of the art and a significantly higher final accuracy.

show abstract

LayerOut: Freezing Layers in Deep Neural Networks

Cited by 17 publications

References 17 publications

Estimation of Off-Target Dicamba Damage on Soybean Using UAV Imagery and Deep Learning

Estimation of Off-Target Dicamba Damage on Soybean Using UAV Imagery and Deep Learning

Toward Predictive Modeling of Solar Power Generation for Multiple Power Plants

CoCo-FL: Communication- and Computation-Aware Federated Learning via Partial NN Freezing and Quantization

Contact Info

Product

Resources

About