Noise-Resilient Variational Quantum Transformers for Sequence Modeling in the NISQ Era

Near-term quantum hardware offers an opportunity to augment sequence modeling, yet NISQ noise and finite-shot measurement variance often erase gains when quantum components are invoked repeatedly across long contexts. This paper proposes a Noise-Resilient Variational Quantum Transformer (VQT) that inserts shallow, connectivity-aware variational circuits selectively into transformer attention scoring, while retaining classical residual pathways for stability and scalability. The training objective is noise-aware, optimizing expected loss under sampled noise realizations, and inference applies tiered mitigation (always-on readout correction and selective zero-noise extrapolation for critical blocks). Across controlled sequence modeling and sequence-level classification evaluations, VQT improves endpoint quality and robustness under increasing noise. Relative to a parameter-matched classical transformer, the full VQT reduces perplexity from 26.8 to 24.2 (−2.6, −9.7%) and bits-per-character from 1.42 to 1.34 (−0.08), while increasing macro-F1 from 0.812 to 0.844 (+0.032, +3.9%). Calibration also improves, with ECE decreasing from 0.064 to 0.048 (−25%). Under a high-noise condition (noise strength 0.10), VQT with mitigation achieves perplexity 32.6 versus 36.2 for the classical baseline (−3.6, −9.9%) and shows a flatter degradation slope (84.0 vs 94.0) and lower robustness AUC (2.86 vs 3.04). Ablations confirm that removing noise-aware training increases the robustness penalty (Δperplexity under noise) from 8.4 to 12.1, while removing mitigation increases it to 15.3, indicating complementary contributions from optimization and mitigation. The approach remains feasible under bounded quantum cost, exemplified by 160 circuit executions per batch at 512 shots per execution.