Explainable artificial intelligence for reliable water demand forecasting to increase trust in predictions

The "EU Artificial Intelligence Act" sets a framework for the implementation of artificial intelligence (AI) in Europe. As a legal assessment reveals, AI applications in water supply systems are categorised as high-risk AI if a failure in the AI application results in a significant impact on physical infrastructure or supply reliability. The use case of water demand forecasts with AI for automatic tank operation is for example categorised as high-risk AI and must fulfil specific requirements regarding model transparency (traceability, explainability) and technical robustness (accuracy, reliability). To this end, six widely established machine learning models, including both transparent and opaque models, are applied to different datasets for daily water demand forecasting and the requirements regarding model accuracy, transparency and technical robustness are systematically evaluated for this use case. Opaque models generally achieve higher prediction accuracy compared to transparent models due to their ability to capture the complex relationship between parameters like for example weather data and water demand. However, this also makes them vulnerable to deviations and irregularities in weather forecasts and historical water demand. In contrast, transparent models rely mainly on historical water demand data for the utilised dataset and are less influenced by weather data, making them more robust against various data irregularities. In summary, both transparent and opaque models can fulfil the requirements regarding explainability but differ in their level of transparency and robustness to input errors. The choice of model depends also on the operator's preferences and the context of the application.