FedLSF: Federated Local Graph Learning via Specformers

The abundance of graphical data and associated privacy concerns in real-world scenarios highlight the need for a secure and distributed methodology utilizing Federated Learning for Graph Neural Networks(GNNs). While spatial GNNs have been explored in FL, spectral GNNs, which capture rich spectral information, remain relatively unexplored. Despite enhancing GNNs' expressiveness through attention-based mechanisms, challenges persist in the spatial approach for FL due to cross-client edges. This work introduces two information capture methods for spectral GNNs in FL settings, Global Information Capture and Local Information Capture, which address cross-client edges. Federated Local Specformer (FedLSF) is proposed as a novel methodology that combines local information capture with state-of-the-art(SOTA) Specformer, enabling local graph learning on clients. FedLSF leverages Specformers involving spectral and attention approaches by integrating Eigen Encoding, Transformer architecture, and graph convolution. This enables capturing rich information from eigen spectra and addresses concerns related to cross-client edges through fully connected eigen-spaces. Experimental results demonstrate FedLSF's efficacy in both homophily and heterophily datasets, showing significant accuracy improvements (2-50)% in highly non-independent and identically distributed (Non-IID) scenarios compared to the present SOTA. This research advances attention-based spectral mechanisms in FL for GNNs, providing a promising solution for preserving privacy in non-IID graph data environments. Implementation can be found at https://github.com/achiverram28/FedLSF-DCOSS