The convenience offered by the Internet accelerates the evolution of fraudulent behavior during facilitating the rapid development of online payment services. Fraudsters can change their behavior patterns frequently and at a low cost in the online space, allowing them to evade regulatory oversight. This poses a significant challenge for meticulously trained learning-based security applications for fraud detection and can lead to serious social security risks. Most of them depend on the static learning paradigm, which trains a model over a static training dataset and deploys the trained model for inference with the frozen model parameters under the i.i.d. assumption. To stay ahead of the rapidly evolving fraud, researchers have been exploring models with low latency and fast response capabilities to effectively combat fraudulent behavior. Unfortunately, the evolving fraud is not only reflected in the drift of their superimposed risk features but also in the openness of their category. The interweaving of open-category and concept-drift accelerates the process of existing security methods becoming powerless. In this paper, we propose EvoFD, an online evolving fraud detection framework to enable continual learning to cope with undercurrent surges of evolving fraud. The core idea of EvoFD is to weaken the bias caused by the anchoring effect on the learned information. It learns in an online streaming fashion by using instructive representations as anchors. Specially, we maintain the progressively updatable class anchors and optimize the representation network to embed features and class anchors into a unified normalized space, where the training and predicting can be conducted simultaneously or independently. In the framework, we preserve the balanced replay memory for each class to accumulate knowledge and avoid forgetting. The advantages of our method are validated by extensive experiments over the real-world dataset from a prestigious bank.