Yan Li, Qiyuan Wang^*, Kaidi Jia

CMC-Computers, Materials & Continua, Vol.78, No.2, pp. 2469-2489, 2024, DOI:10.32604/cmc.2024.047822

Abstract Image description task is the intersection of computer vision and natural language processing, and it has important prospects, including helping computers understand images and obtaining information for the visually impaired. This study presents an innovative approach employing deep reinforcement learning to enhance the accuracy of natural language descriptions of images. Our method focuses on refining the reward function in deep reinforcement learning, facilitating the generation of precise descriptions by aligning visual and textual features more closely. Our approach comprises three key architectures. Firstly, it utilizes Residual Network 101 (ResNet-101) and Faster Region-based Convolutional Neural Network (Faster R-CNN) to extract average… More >

Zhaowei Qu^1,*, Bingyu Cao¹, Xiaoru Wang¹, Fu Li², Peirong Xu¹, Luhan Zhang¹

CMC-Computers, Materials & Continua, Vol.59, No.2, pp. 575-589, 2019, DOI:10.32604/cmc.2019.05569

Abstract Images are complex multimedia data which contain rich semantic information. Most of current image description generator algorithms only generate plain description, with the lack of distinction between primary and secondary object, leading to insufficient high-level semantic and accuracy under public evaluation criteria. The major issue is the lack of effective network on high-level semantic sentences generation, which contains detailed description for motion and state of the principal object. To address the issue, this paper proposes the Attention-based Feedback Long Short-Term Memory Network (AFLN). Based on existing codec framework, there are two independent sub tasks in our method: attention-based feedback LSTM… More >