Shallow Deep Learning: Embedding Verbatim K-Means in Deep Neural Networks

In this paper we show how to implement a deep neural network that is strictly equivalent (sans floating-point errors) to the verbatim (batch) k-means algorithm or Lloyd's algorithm, when trained with gradient descent. Most interestingly, doing so shows that the k-means algorithm, a staple of "conventional" or "shallow" machine learning, can actually be seen as a special case of deep learning, contrary to the general perception that deep learning is a subset of machine learning. Doing so also automatically introduces yet another unsupervised learning technique into the arsenal of deep learning, which happens to be an example of interpretable deep neural networks as well. Finally, we also show how to utilize the powerful deep learning infrastructures with very little extra effort for adaptation.