Engineering a fast online persistent suffix tree construction

Online persistent suffix tree construction has been considered impractical due to its excessive 110 costs. However these prior studies have not taken into account the effects of the buffer management policy and the internal node structure of the suffix tree on I/O behavior o construction and subsequent retrievals over the tree. In this paper we study these two issues in detail in the context of large genomic DNA and Protein sequences. In particular we make the following contributions: (i) a novel, low-overhead buffering policy called TOP-Q which improves the on-disk behavior of suffix tree construction and subsequent retrievals, and (ii) empirical evidence that the space efficient linked-list representation of suffix tree nodes provides significantly inferior performance when compared to the array representation. These results demonstrate that a careful choice of implementation strategies can make online persistent suffix tree construction considerably more scalable - in terms of length of sequences indexed with a fixed memory budget, than currently perceived.