Head direction (HD) cells are neurons found in an extended cortical and subcortical network that signal the orientation of an animal's head relative to its environment [1-3]. They are a fundamental component of the wider circuit of spatially responsive hippocampal formation neurons that make up the neural cognitive map of space [4]. During post-natal development, HD cells are the first among spatially modulated neurons in the hippocampal circuit to exhibit mature firing properties [5, 6], but before eye opening, HD cell responses in rat pups have low directional information and are directionally unstable [7, 8]. Using Bayesian decoding of HD cell ensemble activity recorded in the anterodorsal thalamic nucleus (ADN), we characterize this instability and identify its source: under-signaling of angular head velocity, which incompletely shifts the directional signal in proportion to head turns. We find evidence that geometric cues (the corners of a square environment) can be used to mitigate this under-signaling and, thereby, stabilize the directional signal even before eye opening. Crucially, even when directional firing cannot be stabilized, ensembles of unstable HD cells show short-timescale (1-10 s) temporal and spatial couplings consistent with an adult-like HD network. The HD network is widely modeled as a continuous attractor whose output is one coherent activity peak, updated during movement by angular head velocity signals and anchored by landmark cues [9-11]. Our findings present strong evidence for this model, and they demonstrate that the required network circuitry is in place and functional early during development, independent of reference to landmark information.