Many efforts on domain adaptation focus on stationary environments and assume that the target domain samples are available before the learning process. However, real-world applications frequently involve the availability of non-stationary data sequentially. This study develops an unsupervised heterogeneous domain adaptation approach to address non-stationary scenarios where data streams continually feed the learning model. This process employs a fuzzy-based model that has been trained on a different but related domain. Subsequently, a neighborhood-based weight assignment fine-tunes the attraction and repulsion between neighbors based on prior knowledge about their domains and the similarity between class labels. To avoid unnecessary adaptation for each target domain chunk, domain adaptation is triggered only when concept drift is detected. This way, the model gradually adjusts to the evolving data, incorporating the unique characteristics of the new domain. When no drift is detected, existing parameters are reused for feature adaptation. At the end, the source domain is updated by incorporating the drifting data and their predicted labels. The proposed method offers several advantages, including avoidance of excessive alignment, reduction in domain adaptation cost, and a gradual reduction in dependency on the source domain for domain adaptation. To evaluate the method's performance, experiments were conducted on several tasks extracted from two benchmark datasets, considering different types of concept drift. The experimental results demonstrate that the proposed model significantly improves classification accuracy while reducing computational time.