We demonstrate a cryogenic optomechanical system comprising a flexible Si₃N₄ membrane placed at the center of a free-space optical cavity in a 400 mK cryogenic environment. We observe a mechanical quality factor Q > 4 x 10⁶ for the 261 kHz fundamental drum-head mode of the membrane, and a cavity resonance halfwidth of 60 kHz. The optomechanical system therefore operates in the resolved sideband limit. We monitor the membrane’s thermal motion using a heterodyne optical circuit capable of simultaneously measuring both of the mechanical sidebands, and find that the observed optical spring and damping quantitatively agree with theory. The mechanical sidebands exhibit a Fano lineshape, and to explain this we develop a theory describing heterodyne measurements in the presence of correlated classical laser noise. Finally, we discuss the use of a passive filter cavity to remove classical laser noise, and consider the future requirements for laser cooling this relatively large and low-frequency mechanical element to very near its quantum mechanical ground state.