Ziwen Yu¹, Jianjun Zhang^1,*, Wenwu Tan¹, Ziyi Xiong¹, Peilun Li¹, Liangqing Meng², Haijun Lin¹, Guang Sun³, Peng Guo⁴

Journal on Big Data, Vol.4, No.2, pp. 135-143, 2022, DOI:10.32604/jbd.2022.031024

Abstract Many countries are paying more and more attention to the protection of water resources at present, and how to protect water resources has received extensive attention from society. Water quality monitoring is the key work to water resources protection. How to efficiently collect and analyze water quality monitoring data is an important aspect of water resources protection. In this paper, python programming tools and regular expressions were used to design a web crawler for the acquisition of water quality monitoring data from Global Freshwater Quality Database (GEMStat) sites, and the multi-thread parallelism was added to improve the efficiency in the… More >

S. Samsudeen Shaffi^1,*, I. Muthulakshmi²

Intelligent Automation & Soft Computing, Vol.36, No.1, pp. 183-192, 2023, DOI:10.32604/iasc.2023.031494

Abstract As data grows in size, search engines face new challenges in extracting more relevant content for users’ searches. As a result, a number of retrieval and ranking algorithms have been employed to ensure that the results are relevant to the user’s requirements. Unfortunately, most existing indexes and ranking algorithms crawl documents and web pages based on a limited set of criteria designed to meet user expectations, making it impossible to deliver exceptionally accurate results. As a result, this study investigates and analyses how search engines work, as well as the elements that contribute to higher ranks. This paper addresses the… More >

Tsung-Yin Ou¹, Guan-Yu Lin², Hsin-Pin Fu¹, Shih-Chia Wei¹, Wen-Lung Tsai^3,*

Computer Systems Science and Engineering, Vol.42, No.3, pp. 881-897, 2022, DOI:10.32604/csse.2022.022637

Abstract Most real estate agents develop new objects by visiting unfamiliar clients, distributing leaflets, or browsing other real estate trading website platforms, whereas consumers often rely on websites to search and compare prices when purchasing real property. In addition to being time consuming, this search process renders it difficult for agents and consumers to understand the status changes of objects. In this study, Python is used to write web crawler and image recognition programs to capture object information from the web pages of real estate agents; perform data screening, arranging, and cleaning; compare the text of real estate object information; as… More >

Sawroop Kaur¹, Aman Singh^1,*, G. Geetha², Mehedi Masud³, Mohammed A. Alzain⁴

CMC-Computers, Materials & Continua, Vol.69, No.3, pp. 2933-2948, 2021, DOI:10.32604/cmc.2021.019030

Abstract Web crawlers have evolved from performing a meagre task of collecting statistics, security testing, web indexing and numerous other examples. The size and dynamism of the web are making crawling an interesting and challenging task. Researchers have tackled various issues and challenges related to web crawling. One such issue is efficiently discovering hidden web data. Web crawler’s inability to work with form-based data, lack of benchmarks and standards for both performance measures and datasets for evaluation of the web crawlers make it still an immature research domain. The applications like vertical portals and data integration require hidden web crawling. Most… More >