Adaptive parallel downloading method for proxy systems

Parallel downloading retrieves different pieces of a file from different servers simultaneously and so is expected to greatly shorten file fetch times. A key requirement is that the different servers must hold the same file. We have already proposed a proxy system that can ensure file freshness and concordance. In this paper, we combine parallel downloading with the proxy server technology in order to download a file quickly and ensure that it is the latest version. Our previous paper on parallel downloading took neither the downloading order of file fragments nor the buffer space requirements into account; this paper corrects those omissions. In order to provide the user with the required file in correct order as a byte stream, the proxy server must reorder the pieces fetched from multiple servers and shuffle in the delayed blocks as soon as possible. Thus, "substitution download" is newly introduced, which requests delayed blocks from other servers to complete downloading earlier. Experiments on substitution download across the Internet clarify the tradeoff between the buffering time and the redundant traffic generated by duplicate requests to multiple servers. As a result, the pseudo-optimum balance is discovered and our method is shown both not to increase downloading time and to limit the buffer space. This network software can be applied to download files smoothly absorbing the difference in performance characteristics among heterogeneous networks.