SpecSteer: Synergizing Local Context and Global Reasoning for Efficient Personalized Generation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866918393403146240 |
|---|---|
| author | Lv, Hang Liang, Sheng Wang, Hao Zhang, Yongyue Gu, Hongchao Guo, Wei Lian, Defu Liu, Yong Chen, Enhong |
| author_facet | Lv, Hang Liang, Sheng Wang, Hao Zhang, Yongyue Gu, Hongchao Guo, Wei Lian, Defu Liu, Yong Chen, Enhong |
| contents | Realizing personalized intelligence faces a core dilemma: sending user history to centralized large language models raises privacy concerns, while on-device small language models lack the reasoning capacity required for high-quality generation. Our pilot study shows that purely local enhancements remain insufficient to reliably bridge this gap. We therefore propose SpecSteer, an asymmetric collaborative inference framework that synergizes private on-device context with cloud-scale reasoning. SpecSteer casts collaboration as Bayesian knowledge fusion and repurposes speculative decoding as a distributed alignment protocol, yielding a Draft--Verify--Recover pipeline: the on-device model drafts personalized sequences; the cloud validates via a ratio-based mechanism that decouples reasoning verification from private context, filtering logical flaws without accessing raw user context; upon rejection, a steering recovery injects local intent during correction. Experiments demonstrate that SpecSteer successfully closes the reasoning gap and achieves superior personalized generation performance, while delivering a 2.36x speedup over standard baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_16219 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SpecSteer: Synergizing Local Context and Global Reasoning for Efficient Personalized Generation Lv, Hang Liang, Sheng Wang, Hao Zhang, Yongyue Gu, Hongchao Guo, Wei Lian, Defu Liu, Yong Chen, Enhong Computation and Language Realizing personalized intelligence faces a core dilemma: sending user history to centralized large language models raises privacy concerns, while on-device small language models lack the reasoning capacity required for high-quality generation. Our pilot study shows that purely local enhancements remain insufficient to reliably bridge this gap. We therefore propose SpecSteer, an asymmetric collaborative inference framework that synergizes private on-device context with cloud-scale reasoning. SpecSteer casts collaboration as Bayesian knowledge fusion and repurposes speculative decoding as a distributed alignment protocol, yielding a Draft--Verify--Recover pipeline: the on-device model drafts personalized sequences; the cloud validates via a ratio-based mechanism that decouples reasoning verification from private context, filtering logical flaws without accessing raw user context; upon rejection, a steering recovery injects local intent during correction. Experiments demonstrate that SpecSteer successfully closes the reasoning gap and achieves superior personalized generation performance, while delivering a 2.36x speedup over standard baselines. |
| title | SpecSteer: Synergizing Local Context and Global Reasoning for Efficient Personalized Generation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2603.16219 |