Expert Evaluation of ChatGPT Performance for Risk Management Process based on ISO 31000 Standard

Al-Mhdawi, M.K.S.; Qazi, Abroon; Alzarrad, Ammar; Dacre, Nicholas; Rahimian, Farzad Pour; Buniya, Mohanad K.; Zhang, Hanqin

doi:10.3850/978-981-18-8071-1_p733-cd

Cited by 2 publications

(1 citation statement)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Consistency and Standardization: ChatGPT can help ensure a more consistent and standardized review process. It can follow predefined criteria and guidelines consistently, reducing subjective biases and variations that can arise among human reviewers [ 10 ].…”

Section: Potential Benefits and Challenges Associated With Integratin...mentioning

confidence: 99%

ChatGPT and the Future of Journal Reviews: A Feasibility Study

Biswas,

Dobaria,

Cohen

2023

Yale J Biol Med

View full text Add to dashboard Cite

The increasing volume of research submissions to academic journals poses a significant challenge for traditional peer-review processes. To address this issue, this study explores the potential of employing ChatGPT, an advanced large language model (LLM), developed by OpenAI, as an artificial intelligence (AI) reviewer for academic journals. By leveraging the vast knowledge and natural language processing capabilities of ChatGPT, we hypothesize it may be possible to enhance the efficiency, consistency, and quality of the peer-review process. This research investigated key aspects of integrating ChatGPT into the journal review workflow. We compared the critical analysis of ChatGPT, acting as an AI reviewer, to human reviews for a single published article. Our methodological framework involved subjecting ChatGPT to an intricate examination, wherein its evaluative acumen was juxtaposed against human-authored reviews of a singular published article. As this is a feasibility study, one article was reviewed, which was a case report on scurvy. The entire article was used as an input into ChatGPT and commanded it to "Please perform a review of the following article and give points for revision." Since this was a case report with a limited word count the entire article could fit in one chat box. The output by ChatGPT was then compared with the comments by human reviewers. Key performance metrics, including precision and overall agreement, were judiciously and subjectively measured to portray the efficacy of ChatGPT as an AI reviewer in comparison to its human counterparts. The outcomes of this rigorous analysis unveiled compelling evidence regarding ChatGPT's performance as an AI reviewer. We demonstrated that ChatGPT's critical analyses aligned with those of human reviewers, as evidenced by the inter-rater agreement. Notably, ChatGPT exhibited commendable capability in identifying methodological flaws, articulating insightful feedback on theoretical frameworks, and gauging the overall contribution of the articles to their respective fields. While the integration of ChatGPT showcased immense promise, certain challenges and caveats surfaced. For example, ambiguities might present with complex research articles, leading to nuanced discrepancies between AI and human reviews. Also figures and images cannot be reviewed by ChatGPT. Lengthy articles need to be reviewed in parts by ChatGPT as the entire article will not fit in one chat/response. The benefits consist of reduction in time needed by journals to review the articles submitted to them, as well as an AI assistant to give a different perspective about the research papers other than the human reviewers. In conclusion, this research contributes a groundbreaking foundation for incorporating ChatGPT into the pantheon of journal reviewers. The delineated guidelines distill key insights into operationalizing ChatGPT as a proficient reviewer within academic journal frameworks, paving the way for a more efficient and insightful review process.

show abstract

Section: Potential Benefits and Challenges Associated With Integratin...mentioning

confidence: 99%

ChatGPT and the Future of Journal Reviews: A Feasibility Study

Biswas,

Dobaria,

Cohen

2023

Yale J Biol Med

View full text Add to dashboard Cite

show abstract

Integrating ChatGPT, Bard, and Leading-Edge Generative Artificial Intelligence in Architectural Design and Engineering: Applications, Framework, and Challenges

Rane,

Choudhary,

Rane

2023

IJARP

View full text Add to dashboard Cite

This research paper investigates the integration of advanced generative artificial intelligence (AI) models, such as ChatGPT, Bard, and similar architectures, in architectural design and engineering. The comprehensive study explores various aspects, including applications, frameworks, challenges, and prospective developments in the context of architectural design and architectural engineering. In architectural design, the paper investigates the transformative impact on Architectural Theory, highlighting how generative AI fosters creativity and innovation in design thinking. The Design Process is scrutinized, showcasing how AI models streamline ideation, iteration, and collaboration among design teams. Furthermore, the research examines the influence of generative AI in Interior Design, Urban Design and Planning, and considers nuanced aspects of Cultural and Social factors, elucidating how these technologies contribute to inclusive and context-sensitive design practices. In architectural engineering, the study assesses the integration of generative AI in Structural Engineering, demonstrating its potential to optimize and innovate structural analysis and designs for enhanced safety and efficiency. It explores applications in Building Systems and Construction Management, illustrating how AI can streamline project workflows and resource allocation. The impact of generative AI on compliance with Building Codes and Regulations is analyzed, emphasizing its potential for error reduction and adherence to standards. Additionally, the research probes into the influence of AI in Materials and Construction Technology, highlighting advancements in material selection and construction methodologies. The paper also investigates the role of generative AI in promoting Sustainability and Environmental Design, showcasing its potential to optimize energy efficiency, reduce environmental impact, and enhance overall sustainability. Finally, the paper outlines the challenges and future directions for development to fully harness the potential of generative AI in shaping the future of architectural design and engineering.

show abstract

Expert Evaluation of ChatGPT Performance for Risk Management Process based on ISO 31000 Standard

Cited by 2 publications

References 8 publications

ChatGPT and the Future of Journal Reviews: A Feasibility Study

ChatGPT and the Future of Journal Reviews: A Feasibility Study

Integrating ChatGPT, Bard, and Leading-Edge Generative Artificial Intelligence in Architectural Design and Engineering: Applications, Framework, and Challenges

Contact Info

Product

Resources

About