Machine learning techniques for the automation of literature reviews and systematic reviews in EFSA

Abstract

This Report presents the results from EFSA project RC/EFSA/AMU/2016/01 related to the implementation of machine learning techniques for literature reviews and systematic reviews in EFSA. An overview of the different steps of a systematic review is provided, along with possible ways for automation. Although it was found that most steps could benefit from automation, it was also observed that some steps require more sophisticated methods than those encompassed within the machine learning framework. Availability of data and methodology allowed for the development of an automatic screening tool based on several machine learning techniques. The developed shiny R application can be used for the screening of abstracts and full texts. Properties of machine learning techniques are discussed in this Report together with their most important advantages and disadvantages. The latter discussion includes both general properties, as well as context-specific properties based on their performance in three case studies. Although creating a universal automatic data extraction tool was considered to be infeasible in this stage, this step of the systematic review was addressed to allow the reviewer to scan the uploaded pdf files for certain words or string of words. Based on observations from the performed case studies, recommendations were made regarding which methods are preferred in specific situations. More explicitly, a discussion is made about the performance of the classifiers with respect to the magnitude of the pool of papers to be screened as well as to the amount of imbalance, referring to the proportion of relevant and irrelevant papers. Finally, it was concluded that the results presented in this report provide proof that the developed shiny application could be efficiently used in combination with other software such as DistillerSR. © European Food Safety Authority, 2018Key words: Systematic Reviews, Machine Learning, screening, data extraction, Sensitivity, Specificity Disclaimer: The present document has been produced and adopted by the bodies identified above as author(s). This task has been carried out exclusively by the author(s) in the context of a contract between the European Food Safety Authority and the author(s), awarded following a tender procedure. The present document is published complying with the transparency principle to which the Authority is subject. It may not be considered as an output adopted by the Authority. The European Food Safety Authority reserves its rights, view and position as regards the issues addressed and the conclusions reached in the present document, without prejudice to the rights of the authors. Reproduction is authorised provided the source is acknowledged. Machine Learning Techniques for Literature and Systematic Reviewswww.efsa.europa.eu/publications 3 EFSA Supporting publication 2018:EN-1427The present document has been produced and adopted by the bodies identified above as author. This task has been carried out exclusively by the author in the context of a contract ...

A Survey on Machine learning and IoT

“…Machine learning memampukan sebuah perangkat untuk bisa membantu pekerjaan pengguna atau menghasilkan informasi seperti yang diinginkan pengguna dengan cara perangkat tersebut dilatih dengan menggunakan data-data yang ada [11]. Sejalan dengan penelitian yang dilakukan oleh Jaspers, machine learning adalah sebuah proses yang secara otomatis berkembang atau belajar dari pengalaman atau riset, dan kemudian bertindak tanpa di program terlebih dahulu (karena sudah dilatih menggunakan data riset atau data terdahulu) [19].…”

“…Machine learning mampu mengolah data bahkan yang kompleks sekalipun dengan lebih cepat dan akurat. Kelebihan dari machine learning adalah terletak pada kemampuannya untuk menyediakan solusi umum melalui arsitekturnya yang dapat mengalami proses pembelajaran untuk meningkatkan kinerja [19], [20]. Machine learning menggunakan data terdahulu untuk mengolah data saat ini.…”

“…Sampai saat ini machine learning mampu digunakan untuk menyelesaikan berbagai jenis permasalahan. Karena kemampuannya untuk menyelesaikan berbagai macam masalah, maka machine learning dapat diterapkan pada berbagai bidang aplikasi kehidupan untuk membantu dalam hal prediksi dan membantu proses pengambilan keputusan [19].…”

“…Algoritma ini merupakan algoritma prediksi yang umum digunakan karena sederhana dan menghasilkan hasil prediksi yang akurat. Rumus dari algoritam regresi adalah sebagai berikut seperti terlihat di persamaan 1 [19], [20].…”

Decision support tools for SLR search string construction

“…There have been different proposals regarding the automatization of different stages of the SLR process, most of them being focused on the technical and repetitive tasks in the retrieval, appraisal and synthesis stages ( Table 1). Rather than performing a complete state of the art on the field which would be out of the scope of the present paper, we will refer to a very recent scientific report from the European Food Safety Authority which deeply describes the existing automation methods for the different tasks of a systematic literature reviews [7].…”