Measurements of weak magnetic fields demand a small distance between the sensor and the to-be-measured object. Optically pumped magnetometers (OPMs) utilize laser light and the Zeeman effect in alkali vapor cells to measure those fields. OPMs can be used in transmission or reflection geometry. A minimization of the distance between active volume and magnetized source calls for reflection geometry with integrated mirrors. Unfortunately, cesium reacts chemically with most materials, especially high-performing materials, such as gold. Herein, we show the first functional OPM cell using a gold mirror inside the cell. We fabricated the gold mirrors with and without a passivation layer in order to evaluate the feasibility of expanding on the limited list of possible mirror materials. A comparison of this implementation revealed that mirrors without a passivation layer only reach a reflectivity of about 6% while mirrors with a passivation layer retain reflectivity values of about 90% in the visible light to near-infrared spectrum. This result and the proof of elemental cesium in the alkali vapor cell demonstrates the feasibility of passivated gold mirrors for applications in alkali vapor cells for OPMs.