Luyu Ma^1,*, Xiu Cheng^1,*, Zongyan Xing¹, Yue Wu¹, Weiwei Jiang²

CMC-Computers, Materials & Continua, Vol.85, No.2, pp. 3921-3943, 2025, DOI:10.32604/cmc.2025.067786 - 23 September 2025

Abstract Green consumption (GC) are crucial for achieving the Sustainable Development Goals (SDGs). However, few studies have explored public attitudes toward GC using social media data, missing potential public concerns captured through big data. To address this gap, this study collects and analyzes public attention toward GC using web crawler technology. Based on the data from Sina Weibo, we applied RoBERTa, an advanced NLP model based on transformer architecture, to conduct fine-grained sentiment analysis of the public’s attention, attitudes and hot topics on GC, demonstrating the potential of deep learning methods in capturing dynamic and contextual… More >

S. Samsudeen Shaffi^1,*, I. Muthulakshmi²

Intelligent Automation & Soft Computing, Vol.36, No.1, pp. 183-192, 2023, DOI:10.32604/iasc.2023.031494 - 29 September 2022

Abstract As data grows in size, search engines face new challenges in extracting more relevant content for users’ searches. As a result, a number of retrieval and ranking algorithms have been employed to ensure that the results are relevant to the user’s requirements. Unfortunately, most existing indexes and ranking algorithms crawl documents and web pages based on a limited set of criteria designed to meet user expectations, making it impossible to deliver exceptionally accurate results. As a result, this study investigates and analyses how search engines work, as well as the elements that contribute to higher… More >

Ziwen Yu¹, Jianjun Zhang^1,*, Wenwu Tan¹, Ziyi Xiong¹, Peilun Li¹, Liangqing Meng², Haijun Lin¹, Guang Sun³, Peng Guo⁴

Journal on Big Data, Vol.4, No.2, pp. 135-143, 2022, DOI:10.32604/jbd.2022.031024 - 31 October 2022

Abstract Many countries are paying more and more attention to the protection of water resources at present, and how to protect water resources has received extensive attention from society. Water quality monitoring is the key work to water resources protection. How to efficiently collect and analyze water quality monitoring data is an important aspect of water resources protection. In this paper, python programming tools and regular expressions were used to design a web crawler for the acquisition of water quality monitoring data from Global Freshwater Quality Database (GEMStat) sites, and the multi-thread parallelism was added to More >

Tsung-Yin Ou¹, Guan-Yu Lin², Hsin-Pin Fu¹, Shih-Chia Wei¹, Wen-Lung Tsai^3,*

Computer Systems Science and Engineering, Vol.42, No.3, pp. 881-897, 2022, DOI:10.32604/csse.2022.022637 - 08 February 2022

Abstract Most real estate agents develop new objects by visiting unfamiliar clients, distributing leaflets, or browsing other real estate trading website platforms, whereas consumers often rely on websites to search and compare prices when purchasing real property. In addition to being time consuming, this search process renders it difficult for agents and consumers to understand the status changes of objects. In this study, Python is used to write web crawler and image recognition programs to capture object information from the web pages of real estate agents; perform data screening, arranging, and cleaning; compare the text of… More >

Sawroop Kaur¹, Aman Singh^1,*, G. Geetha², Mehedi Masud³, Mohammed A. Alzain⁴

CMC-Computers, Materials & Continua, Vol.69, No.3, pp. 2933-2948, 2021, DOI:10.32604/cmc.2021.019030 - 24 August 2021

Abstract Web crawlers have evolved from performing a meagre task of collecting statistics, security testing, web indexing and numerous other examples. The size and dynamism of the web are making crawling an interesting and challenging task. Researchers have tackled various issues and challenges related to web crawling. One such issue is efficiently discovering hidden web data. Web crawler’s inability to work with form-based data, lack of benchmarks and standards for both performance measures and datasets for evaluation of the web crawlers make it still an immature research domain. The applications like vertical portals and data integration… More >