Profiling web archive coverage for top-level domain and content language

The Memento Aggregator currently polls every known public web archive when serving a request for an archived web page, even though some web archives focus on only specific domains and ignore the others. Similar to query routing in distributed search, we investigate the impact on aggregated Memento TimeMaps (lists of when and where a web page was archived) by only sending queries to archives likely to hold the archived page. We profile fifteen public web archives using data from a variety of sources (the web, archives' access logs, and fulltext queries to archives) and use these profiles as resource descriptor. These profiles are used in matching the URI-lookuprequests to the most probable web archives. We define Recall(TM)(n) as the percentage of a Time Map that was returned using n web archives. We discover that only sending queries to the top three web archives (i.e., 80 % reduction in the number of queries) for any request reaches on average Recall(TM) = 0.96. If we exclude the Internet Archive from the list, we can reach Recall(TM) = 0.647 on average using only the remaining top three web archives.