EMACrawler: web search engine database freshness optimization

In today's information and technology age, search engines have become an important part of our lives. However, search engines are the first to be used to access information, old and unnecessary information is included in the content offered to users. Regarding providing up-to-date data, today's search engines often cannot offer the desired success. In order to keep the data presented by web browsers up-to-date, the time of return visits must be accurately estimated. In this study, EMACrawler based on exponential moving averages is proposed to determine the revisit times, which is the most important feature that affects the performance of search engines. The proposed method is tested using precision, total coverage, and efficiency metrics. It has been seen that EMACrawler obtains the current data on the web pages accurately and quickly. As a result of the experimental studies, it has been seen that EMACrawler is more successful than other methods in obtaining up-to-date data and maintaining the freshness of the browser database.