Optomechanical devices in which a flexible SiN membrane is placed inside an optical cavity allow for very high finesse and mechanical quality factor in a single device. They also provide fundamentally new functionality: the cavity detuning can be a quadratic function of membrane position. This enables a measurement of “position squared” (x²) and in principle a QND phonon number readout of the membrane. However, the readout achieved using a single transverse cavity mode is not sensitive enough to observe quantum jumps between phonon Fock states.

Here we demonstrate an x²-sensitivity that is orders of magnitude stronger using two transverse cavity modes that are nearly degenerate. We derive a first-order perturbation theory to describe the interactions between nearly-degenerate cavity modes and achieve good agreement with our measurements using realistic parameters. We also demonstrate theoretically that the x²-coupling should be easily tunable over a wide range.