Copertina

Retrieval-Pretrained Transformer: Long-range Language Modeling with Self-retrieval

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Rubin, Ohad, Berant, Jonathan
Natura:	Preprint
Pubblicazione:	2023
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Making Retrieval-Augmented Language Models Robust to Irrelevant Context
di: Yoran, Ori, et al.
Pubblicazione: (2023)

Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors
di: Amos, Ido, et al.
Pubblicazione: (2023)

Large Language Models for Psycholinguistic Plausibility Pretesting
di: Amouyal, Samuel Joseph, et al.
Pubblicazione: (2024)

MLP Memory: A Retriever-Pretrained Memory for Large Language Models
di: Wei, Rubin, et al.
Pubblicazione: (2025)

From Loops to Oops: Fallback Behaviors of Language Models Under Uncertainty
di: Ivgi, Maor, et al.
Pubblicazione: (2024)

Comparing Human and Language Models Sentence Processing Difficulties on Complex Structures
di: Amouyal, Samuel Joseph, et al.
Pubblicazione: (2025)

In-Context Learning with Long-Context Models: An In-Depth Exploration
di: Bertsch, Amanda, et al.
Pubblicazione: (2024)

Transforming and Combining Rewards for Aligning Large Language Models
di: Wang, Zihao, et al.
Pubblicazione: (2024)

Neurocache: Efficient Vector Retrieval for Long-range Language Modeling
di: Safaya, Ali, et al.
Pubblicazione: (2024)

Weakly Supervised Text-to-SQL Parsing through Question Decomposition
di: Wolfson, Tomer, et al.
Pubblicazione: (2021)

A Family of Pretrained Transformer Language Models for Russian
di: Zmitrovich, Dmitry, et al.
Pubblicazione: (2023)

ALTA: Compiler-Based Analysis of Transformers
di: Shaw, Peter, et al.
Pubblicazione: (2024)

When the LM misunderstood the human chuckled: Analyzing garden path effects in humans and language models
di: Amouyal, Samuel Joseph, et al.
Pubblicazione: (2025)

DOLOMITES: Domain-Specific Long-Form Methodical Tasks
di: Malaviya, Chaitanya, et al.
Pubblicazione: (2024)

Memory Decoder: A Pretrained, Plug-and-Play Memory for Large Language Models
di: Cao, Jiaqi, et al.
Pubblicazione: (2025)

MT-PingEval: Evaluating Multi-Turn Collaboration with Private Information Games
di: Eisenstein, Jacob, et al.
Pubblicazione: (2026)

Dissecting Paraphrases: The Impact of Prompt Syntax and supplementary Information on Knowledge Retrieval from Pretrained Language Models
di: Linzbach, Stephan, et al.
Pubblicazione: (2024)

Generative Pretrained Structured Transformers: Unsupervised Syntactic Language Models at Scale
di: Hu, Xiang, et al.
Pubblicazione: (2024)

Adapting Pretrained Language Models for Citation Classification via Self-Supervised Contrastive Learning
di: Li, Tong, et al.
Pubblicazione: (2025)

Cite Pretrain: Retrieval-Free Knowledge Attribution for Large Language Models
di: Huang, Yukun, et al.
Pubblicazione: (2025)

Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Dataset
di: Su, Dan, et al.
Pubblicazione: (2024)

Literary Evidence Retrieval via Long-Context Language Models
di: Thai, Katherine, et al.
Pubblicazione: (2025)

Pretraining Language Models Using Translationese
di: Doshi, Meet, et al.
Pubblicazione: (2024)

Geographic Adaptation of Pretrained Language Models
di: Hofmann, Valentin, et al.
Pubblicazione: (2022)

Parametric Knowledge is Not All You Need: Toward Honest Large Language Models via Retrieval of Pretraining Data
di: Kusuma, Christopher Adrian, et al.
Pubblicazione: (2026)

AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?
di: Yoran, Ori, et al.
Pubblicazione: (2024)

End-to-end Contrastive Language-Speech Pretraining Model For Long-form Spoken Question Answering
di: Hu, Jiliang, et al.
Pubblicazione: (2025)

Transformer verbatim in-context retrieval across time and scale
di: Armeni, Kristijan, et al.
Pubblicazione: (2024)

Stronger Baselines for Retrieval-Augmented Generation with Long-Context Language Models
di: Laitenberger, Alex, et al.
Pubblicazione: (2025)

ARWKV: Pretrain is not what we need, an RNN-Attention-Based Language Model Born from Transformer
di: Yueyu, Lin, et al.
Pubblicazione: (2025)

signwriting-evaluation: Effective Sign Language Evaluation via SignWriting
di: Moryossef, Amit, et al.
Pubblicazione: (2024)

Progressive Residual Warmup for Language Model Pretraining
di: Chen, Tianhao, et al.
Pubblicazione: (2026)

Vision-and-Language Navigation Generative Pretrained Transformer
di: Hanlin, Wen
Pubblicazione: (2024)

Answering Questions by Meta-Reasoning over Multiple Chains of Thought
di: Yoran, Ori, et al.
Pubblicazione: (2023)

Vision-and-Language Pretraining
di: Nguyen, Thong, et al.
Pubblicazione: (2022)

From Interpretability to Performance: Optimizing Retrieval Heads for Long-Context Language Models
di: Ma, Youmi, et al.
Pubblicazione: (2026)

FlashBack:Efficient Retrieval-Augmented Language Modeling for Long Context Inference
di: Liu, Runheng, et al.
Pubblicazione: (2024)

Robust Preference Optimization through Reward Model Distillation
di: Fisch, Adam, et al.
Pubblicazione: (2024)

Making the Most of your Model: Methods for Finetuning and Applying Pretrained Transformers
di: Yoshida, Davis
Pubblicazione: (2024)

Language Models can Self-Lengthen to Generate Long Texts
di: Quan, Shanghaoran, et al.
Pubblicazione: (2024)