Copertina

Squid: Long Context as a New Modality for Energy-Efficient On-Device Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Chen, Wei, Li, Zhiyuan, Xin, Shuo, Wang, Yihao
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

LongRecipe: Recipe for Efficient Long Context Generalization in Large Language Models
di: Hu, Zhiyuan, et al.
Pubblicazione: (2024)

On-Device Language Models: A Comprehensive Review
di: Xu, Jiajun, et al.
Pubblicazione: (2024)

Making Long-Context Language Models Better Multi-Hop Reasoners
di: Li, Yanyang, et al.
Pubblicazione: (2024)

OmniVLM: A Token-Compressed, Sub-Billion-Parameter Vision-Language Model for Efficient On-Device Inference
di: Chen, Wei, et al.
Pubblicazione: (2024)

Latent-Condensed Transformer for Efficient Long Context Modeling
di: You, Zeng, et al.
Pubblicazione: (2026)

Training-free Context-adaptive Attention for Efficient Long Context Modeling
di: You, Zeng, et al.
Pubblicazione: (2025)

D2O: Dynamic Discriminative Operations for Efficient Long-Context Inference of Large Language Models
di: Wan, Zhongwei, et al.
Pubblicazione: (2024)

LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
di: Chen, Yukang, et al.
Pubblicazione: (2023)

SEGMENT+: Long Text Processing with Short-Context Language Models
di: Shi, Wei, et al.
Pubblicazione: (2024)

Long-Context Language Modeling with Parallel Context Encoding
di: Yen, Howard, et al.
Pubblicazione: (2024)

Core Context Aware Transformers for Long Context Language Modeling
di: Chen, Yaofo, et al.
Pubblicazione: (2024)

Evaluating from Benign to Dynamic Adversarial: A Squid Game for Large Language Models
di: Chen, Zijian, et al.
Pubblicazione: (2025)

LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimization
di: Chen, Guanzheng, et al.
Pubblicazione: (2025)

Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads on Consumer-Grade Devices
di: Huang, Yuxiang, et al.
Pubblicazione: (2024)

Cross-Modal Knowledge Distillation for Speech Large Language Models
di: Wang, Enzhi, et al.
Pubblicazione: (2025)

LooGLE: Can Long-Context Language Models Understand Long Contexts?
di: Li, Jiaqi, et al.
Pubblicazione: (2023)

UniMem: Towards a Unified View of Long-Context Large Language Models
di: Fang, Junjie, et al.
Pubblicazione: (2024)

CCF: A Context Compression Framework for Efficient Long-Sequence Language Modeling
di: Li, Wenhao, et al.
Pubblicazione: (2025)

Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts
di: Chen, Yingfa, et al.
Pubblicazione: (2026)

Scaling Context, Not Parameters: Training a Compact 7B Language Model for Efficient Long-Context Processing
di: Wu, Chen, et al.
Pubblicazione: (2025)

E2LLM: Encoder Elongated Large Language Models for Long-Context Understanding and Reasoning
di: Liao, Zihan, et al.
Pubblicazione: (2024)

SWAN-GPT: An Efficient and Scalable Approach for Long-Context Language Modeling
di: Puvvada, Krishna C., et al.
Pubblicazione: (2025)

Membership Inference Attack against Long-Context Large Language Models
di: Wang, Zixiong, et al.
Pubblicazione: (2024)

Exploring Fine-Tuning for In-Context Retrieval and Efficient KV-Caching in Long-Context Language Models
di: Molfese, Francesco Maria, et al.
Pubblicazione: (2026)

Long Context is Not Long at All: A Prospector of Long-Dependency Data for Large Language Models
di: Chen, Longze, et al.
Pubblicazione: (2024)

Monocle: Hybrid Local-Global In-Context Evaluation for Long-Text Generation with Uncertainty-Based Active Learning
di: Wang, Xiaorong, et al.
Pubblicazione: (2025)

$\infty$Bench: Extending Long Context Evaluation Beyond 100K Tokens
di: Zhang, Xinrong, et al.
Pubblicazione: (2024)

Thus Spake Long-Context Large Language Model
di: Liu, Xiaoran, et al.
Pubblicazione: (2025)

LongProc: Benchmarking Long-Context Language Models on Long Procedural Generation
di: Ye, Xi, et al.
Pubblicazione: (2025)

Efficient Length-Generalizable Attention via Causal Retrieval for Long-Context Language Modeling
di: Hu, Xiang, et al.
Pubblicazione: (2024)

Implicit Graph, Explicit Retrieval: Towards Efficient and Interpretable Long-horizon Memory for Large Language Models
di: Zhang, Xin, et al.
Pubblicazione: (2026)

LongReD: Mitigating Short-Text Degradation of Long-Context Large Language Models via Restoration Distillation
di: Dong, Zican, et al.
Pubblicazione: (2025)

FlashBack:Efficient Retrieval-Augmented Language Modeling for Long Context Inference
di: Liu, Runheng, et al.
Pubblicazione: (2024)

LLM$\times$MapReduce: Simplified Long-Sequence Processing using Large Language Models
di: Zhou, Zihan, et al.
Pubblicazione: (2024)

WildLong: Synthesizing Realistic Long-Context Instruction Data at Scale
di: Li, Jiaxi, et al.
Pubblicazione: (2025)

LoPT: Lossless Parallel Tokenization Acceleration for Long Context Inference of Large Language Model
di: Shao, Wei, et al.
Pubblicazione: (2025)

LongAlign: A Recipe for Long Context Alignment of Large Language Models
di: Bai, Yushi, et al.
Pubblicazione: (2024)

LongSkywork: A Training Recipe for Efficiently Extending Context Length in Large Language Models
di: Zhao, Liang, et al.
Pubblicazione: (2024)

Efficient Context Scaling with LongCat ZigZag Attention
di: Zhang, Chen, et al.
Pubblicazione: (2025)

MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter
di: Liu, Zhiyuan, et al.
Pubblicazione: (2023)