Cover Image

Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokens

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Jiacheng, Min, Sewon, Zettlemoyer, Luke, Choi, Yejin, Hajishirzi, Hannaneh
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Information Retrieval
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Infini-gram mini: Exact n-gram Search at the Internet Scale with FM-Index
by: Xu, Hao, et al.
Published: (2025)

Scaling Retrieval-Based Language Models with a Trillion-Token Datastore
by: Shao, Rulin, et al.
Published: (2024)

BTR: Binary Token Representations for Efficient Retrieval Augmented Language Models
by: Cao, Qingqing, et al.
Published: (2023)

RAG over Thinking Traces Can Improve Reasoning Tasks
by: Arabzadeh, Negar, et al.
Published: (2026)

A Systematic Examination of Preference Learning through the Lens of Instruction-Following
by: Kim, Joongwon, et al.
Published: (2024)

SILO Language Models: Isolating Legal Risk In a Nonparametric Datastore
by: Min, Sewon, et al.
Published: (2023)

DS SERVE: A Framework for Efficient and Scalable Neural Retrieval
by: Liu, Jinjian, et al.
Published: (2025)

ReasonIR: Training Retrievers for Reasoning Tasks
by: Shao, Rulin, et al.
Published: (2025)

Empowering ChatGPT-Like Large-Scale Language Models with Local Knowledge Base for Industrial Prognostics and Health Management
by: Wang, Huan, et al.
Published: (2023)

MSA: Memory Sparse Attention for Efficient End-to-End Memory Model Scaling to 100M Tokens
by: Chen, Yu, et al.
Published: (2026)

TnT-LLM: Text Mining at Scale with Large Language Models
by: Wan, Mengting, et al.
Published: (2024)

Towards Lossless Token Pruning in Late-Interaction Retrieval Models
by: Zong, Yuxuan, et al.
Published: (2025)

TokenRec: Learning to Tokenize ID for LLM-based Generative Recommendation
by: Qu, Haohao, et al.
Published: (2024)

HITgram: A Platform for Experimenting with n-gram Language Models
by: Dasgupta, Shibaranjani, et al.
Published: (2024)

Arctic-Embed: Scalable, Efficient, and Accurate Text Embedding Models
by: Merrick, Luke, et al.
Published: (2024)

TableRAG: Million-Token Table Understanding with Language Models
by: Chen, Si-An, et al.
Published: (2024)

N-gram Prediction and Word Difference Representations for Language Modeling
by: Heo, DongNyeong, et al.
Published: (2024)

Pctx: Tokenizing Personalized Context for Generative Recommendation
by: Zhong, Qiyong, et al.
Published: (2025)

Stealthy Attack on Large Language Model based Recommendation
by: Zhang, Jinghao, et al.
Published: (2024)

WildVis: Open Source Visualizer for Million-Scale Chat Logs in the Wild
by: Deng, Yuntian, et al.
Published: (2024)

A Roadmap to Pluralistic Alignment
by: Sorensen, Taylor, et al.
Published: (2024)

Token-wise Influential Training Data Retrieval for Large Language Models
by: Lin, Huawei, et al.
Published: (2024)

ScIRGen: Synthesize Realistic and Large-Scale RAG Dataset for Scientific Research
by: Lin, Junyong, et al.
Published: (2025)

Evaluating the External and Parametric Knowledge Fusion of Large Language Models
by: Zhang, Hao, et al.
Published: (2024)

A Survey on Proactive Defense Strategies Against Misinformation in Large Language Models
by: Liu, Shuliang, et al.
Published: (2025)

Grounding Language Model with Chunking-Free In-Context Retrieval
by: Qian, Hongjin, et al.
Published: (2024)

A Theory for Token-Level Harmonization in Retrieval-Augmented Generation
by: Xu, Shicheng, et al.
Published: (2024)

Semi-Parametric Retrieval via Binary Bag-of-Tokens Index
by: Zhou, Jiawei, et al.
Published: (2024)

AutoSurvey: Large Language Models Can Automatically Write Surveys
by: Wang, Yidong, et al.
Published: (2024)

Model Editing at Scale leads to Gradual and Catastrophic Forgetting
by: Gupta, Akshat, et al.
Published: (2024)

Learning to Detect Language Model Training Data via Active Reconstruction
by: Yin, Junjie Oscar, et al.
Published: (2026)

Hierarchical Resolution Transformers: A Wavelet-Inspired Architecture for Multi-Scale Language Understanding
by: Sar, Ayan, et al.
Published: (2025)

In-context Learning with Retrieved Demonstrations for Language Models: A Survey
by: Luo, Man, et al.
Published: (2024)

ParaPO: Aligning Language Models to Reduce Verbatim Reproduction of Pre-training Data
by: Chen, Tong, et al.
Published: (2025)

AssistRAG: Boosting the Potential of Large Language Models with an Intelligent Information Assistant
by: Zhou, Yujia, et al.
Published: (2024)

Enhancing Temporal Sensitivity of Large Language Model for Recommendation with Counterfactual Tuning
by: Liu, Yutian, et al.
Published: (2025)

Large Language Models for Generative Recommendation: A Survey and Visionary Discussions
by: Li, Lei, et al.
Published: (2023)

Don't throw away your value model! Generating more preferable text with Value-Guided Monte-Carlo Tree Search decoding
by: Liu, Jiacheng, et al.
Published: (2023)

Improving Tool Retrieval by Leveraging Large Language Models for Query Generation
by: Kachuee, Mohammad, et al.
Published: (2024)

When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities
by: Chen, Jin, et al.
Published: (2023)