Zero-shot learning, applied with vision-language pretrained (VLP) models, are expected to be an alternative to existing deep learning models for defect detection, under insufficient dataset. However, VLP models, including contrastive language-image pre-training (CLIP), showed fluctuated performance on prompts (inputs), resulting in research on prompt engineering—optimization of prompts for improving performance. Therefore, this study aims to identify the features of a prompt that can yield the best performance in classifying and detecting building defects using the zero-shot and few-shot capabilities of CLIP. The results reveal the following: (1) domain-specific definitions are better than general definitions and images; (2) a complete sentence is better than a set of core terms; and (3) multimodal information is better than single-modal information. The resulting detection performance using the proposed prompting method outperformed that of existing supervised models. Learn more…
Yong, G., Jeon, K., Gil, D., and Lee. G., (2022), "Prompt Engineering for Zero-Shot and Few-Shot Defect Detection and Classification Using a Visual-Language Pretrained Model". "Computer-Aided Civil and Infrastructure Engineering (CACIE)
공공 빅데이터를 활용한 적정 흙막이 공법 자동 추천 방법, 이를 수행하는 장치 및 컴퓨터 프로그램
표시어를 자동 인식하기 위한 문자 인식 방법 및 장치
Jeon, K., Lee, G., and Yang, S. (2022). "Named entity recognition of building construction defect information from text with linguistic noise." Automation in Construction 143: 104543.