Breast tumor margin assessment requires rapid identification of regions of interest (ROIs) and detailed visualization of their microstructural alterations. Here, we report position-tracked handheld optical coherence tomography (PH-OCT) guided by projected wide-field near-infrared autofluorescence(PW-NIRAF) imaging to meet these requirements. The PW-NIRAF system comprises a near-infrared (NIR) imaging camera and a projector with calibrated fields of view, enabling ROI identification and projection of the corresponding ROI map over a 10 × 10 cm2 field of view (FOV). The PH-OCT system consists of a handheld gradient-index-rod-lens-based OCT probe for volumetric visualization of ROI microstructure, with each OCT volume covering 4 × 4 × 1 mm3 at 9.66 μm axial and 17.54 μm lateral resolution. Blue and green tapes used as color markers on the OCT-probe housing, together with a color camera and a deep-learning-based YOLOv11-pose model, enable real-time probe-tip localization for OCT volume acquisition across PW-NIRAF-projected ROIs, providing depth resolved microstructural information beyond wide-field NIRAF contrast. Color-camera/projector calibration further enables back-projection of OCT-derived tumor-region maps onto the tissue surface. Validation on ex vivo breast tissue plays the foundation for future intraoperative breast tumor margin assessment using PW-NIRAF-guided PH-OCT.