Cluster ensemble techniques are effective in improving both the robustness and the stability of the single clustering algorithm. How to combine multiple clusters to yield a final superior clustering result is critical in cluster ensemble. Spectral clustering algorithm is introduced to solve document cluster ensemble problem. Normalized Laplacian matrix-based spectral algorithm (NLMSA) is proposed. According to algebraic transformation, it computes eigenvalues and eigenvectors of a small matrix to obtain the eigenvectors of normalized Laplacian matrix. The key idea of spectral clustering algorithm is further investigated, and hyperedge transition matrix-based spectral algorithm (HTMSA) is proposed. It attains the low dimensional embeddings of documents by those of hyperedges and then the K-means algorithm is used to cluster according to those embedding results of documents. Experimental results on TREC and Reuters document sets demonstrate the effectiveness of the proposed algorithms. Both NLMSA and HTMSA outperform other cluster ensemble techniques based on graph partitioning. NLMSA obtains better results than HTMSA while the computational cost of HTMSA is much lower than that of NLMSA.