Quantum Transfer Learning with Distilled Quantum Teachers for Low-Resource Classification Tasks

Low-resource classification remains a persistent bottleneck for quantum machine learning due to label scarcity, class imbalance, and finite-shot measurement variance under NISQ constraints. This study proposes a hybrid framework that couples Quantum Transfer Learning with Distilled Quantum Teachers, where a high-capacity quantum teacher provides temperature-smoothed soft targets to train a compact student quantum adapter on top of a pretrained classical encoder. Experiments on three low-resource tasks (LR-Text-4, LR-Image-3, LR-Sensor-5) demonstrate consistent gains over non-distilled training at fixed student inference cost (2048 shots). The distilled student improves accuracy from 0.794 to 0.821 on LR-Text-4, from 0.781 to 0.812 on LR-Image-3, and from 0.768 to 0.809 on LR-Sensor-5, while macro F1 increases from 0.781 to 0.811, from 0.772 to 0.804, and from 0.742 to 0.789, respectively. Calibration improves substantially, with expected calibration error decreasing from 0.071 to 0.049 (LR-Text-4), from 0.074 to 0.052 (LR-Image-3), and from 0.083 to 0.056 (LR-Sensor-5), indicating more reliable confidence under scarcity. A temperature sweep reveals an optimal stability region around T ≈ 3.0 to 3.5, where aggregate accuracy peaks near 0.831 and mean calibration reaches its minimum near 0.043. Transfer ablations show that moderate encoder unfreezing (ratio 0.25) yields the best trade-off, achieving mean accuracy 0.842 and mean ECE 0.039, while full fine-tuning reduces accuracy to 0.818 and degrades calibration to 0.058. Resource analysis using a cost proxy (shots × depth) confirms that distilled students dominate the efficient regime, achieving 0.828 accuracy at cost 20,480 versus the teacher’s 0.867 accuracy at cost 180,224. Compared with classical baselines, the distilled student remains most advantageous in the high-overlap sensor setting, outperforming shallow random forests (0.809 vs 0.778 accuracy) while maintaining better calibrated uncertainty. These results position distilled quantum teachers as an effective offline supervision mechanism for compressing high-capacity quantum decision structure into deployable low-cost students for low-resource classification.