Jeon, K., and G. Lee. 2025. “Hybrid large language model approach for prompt and sensitive defect management: A comparative analysis of hybrid, non-hybrid, and GraphRAG approaches.” Advanced Engineering Informatics, 64: 103076. https://doi.org/10.1016/j.aei.2024.103076.

This study aims to propose a large language model (LLM)-enhanced defect question-answering (QA) method that can secure private and sensitive data while yielding high performance. Prompt responses to residents’ complaints are crucial for preventing recurring defects. However, traditional defect analysis and response methods rely on the expertise of a few skilled workers, making it difficult to ensure timely responses. The rapid advancement of LLMs offers a potential solution for improving defect QA tasks. However, many companies prohibit the use of closed-source LLM services, such as ChatGPT, due to concerns about potential data breaches. One possible solution is to use open-source LLMs like Llama and BERT, which can be locally installed and used. However, open-source LLMs typically perform worse than closed-source LLMs. Although the performance of open-source LLMs can be greatly improved through fine-tuning, the preparation of training datasets requires a significant amount of time and labor. To address these challenges, this study proposes a hybrid defect QA method that deploys an open-source LLM for defect management to secure sensitive information, and a closed-source LLM for generating a training dataset to reduce both the time and labor required. To validate the proposed method, we compare it to the state-of-the-art LLMs, GPT-4o and Llama 3, as well as graph retrieval-augmented generation (GraphRAG)-based QA systems, which have been extensively studied recently. Our results show that the hybrid LLM-based QA method achieved the highest ROUGE score of 81.6%. These findings demonstrate superior practical applicability, enabling cost-effective data generation and reliable domain adaptation within a secure data environment. This approach is beneficial for domain-specific tasks beyond defect management, where the accurate provision of specialized information and integration of historical knowledge are essential. Learn more…

2025-03-10