Cover Image

SWAN-GPT: An Efficient and Scalable Approach for Long-Context Language Modeling

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Puvvada, Krishna C., Ladhak, Faisal, Serrano, Santiago Akle, Hsieh, Cheng-Ping, Acharya, Shantanu, Majumdar, Somshubra, Jia, Fei, Kriman, Samuel, Sun, Simeng, Rekesh, Dima, Ginsburg, Boris
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

RULER: What's the Real Context Size of Your Long-Context Language Models?
by: Hsieh, Cheng-Ping, et al.
Published: (2024)

L0-Reasoning Bench: Evaluating Procedural Correctness in Language Models via Simple Program Execution
by: Sun, Simeng, et al.
Published: (2025)

nGPT: Normalized Transformer with Representation Learning on the Hypersphere
by: Loshchilov, Ilya, et al.
Published: (2024)

Star Attention: Efficient LLM Inference over Long Sequences
by: Acharya, Shantanu, et al.
Published: (2024)

From Output to Evaluation: Does Raw Instruction-Tuned Code LLMs Output Suffice for Fill-in-the-Middle Code Generation?
by: Ahmad, Wasi Uddin, et al.
Published: (2025)

Scoring Verifiers: Evaluating Synthetic Verification for Code and Reasoning
by: Ficek, Aleksander, et al.
Published: (2025)

Multi-blank Transducers for Speech Recognition
by: Xu, Hainan, et al.
Published: (2022)

Resource-Efficient Adaptation of Speech Foundation Models for Multi-Speaker ASR
by: Wang, Weiqing, et al.
Published: (2024)

From SWE-ZERO to SWE-HERO: Execution-free to Execution-based Fine-tuning for Software Engineering Agents
by: Ludwig, Nikolai, et al.
Published: (2026)

Stateful Conformer with Cache-based Inference for Streaming Automatic Speech Recognition
by: Noroozi, Vahid, et al.
Published: (2023)

Instruction Data Generation and Unsupervised Adaptation for Speech Language Models
by: Noroozi, Vahid, et al.
Published: (2024)

Multilingual Audio-Visual Speech Recognition with Hybrid CTC/RNN-T Fast Conformer
by: Burchi, Maxime, et al.
Published: (2024)

Less is More: Accurate Speech Recognition & Translation without Web-Scale Data
by: Puvvada, Krishna C., et al.
Published: (2024)

BESTOW: Efficient and Streamable Speech Language Model with the Best of Two Worlds in GPT and T5
by: Chen, Zhehuai, et al.
Published: (2024)

OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMs
by: Ahmad, Wasi Uddin, et al.
Published: (2025)

How much do contextualized representations encode long-range context?
by: Sun, Simeng, et al.
Published: (2024)

OpenCodeReasoning: Advancing Data Distillation for Competitive Coding
by: Ahmad, Wasi Uddin, et al.
Published: (2025)

Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models
by: Samadi, Mehrzad, et al.
Published: (2025)

LIDERAZGO FORMATIVO Y DESARROLLO SOSTENIBLE DE LA CALIDAD DE LA EDUCACIÓN
by: Álvaro Rodríguez Akle
Published: (2009)

Measuring text summarization factuality using atomic facts entailment metrics in the context of retrieval augmented generation
by: Kriman, N. E.
Published: (2024)

Genetic Instruct: Scaling up Synthetic Generation of Coding Instructions for Large Language Models
by: Majumdar, Somshubra, et al.
Published: (2024)

OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critique
by: Ahmad, Wasi Uddin, et al.
Published: (2025)

Incorporating Human Explanations for Robust Hate Speech Detection
by: Chen, Jennifer L., et al.
Published: (2024)

Sortformer: A Novel Approach for Permutation-Resolved Speaker Supervision in Speech-to-Text Systems
by: Park, Taejin, et al.
Published: (2024)

NEST: Self-supervised Fast Conformer as All-purpose Seasoning to Speech Processing Tasks
by: Huang, He, et al.
Published: (2024)

Word Level Timestamp Generation for Automatic Speech Recognition and Translation
by: Hu, Ke, et al.
Published: (2025)

Aligning Large Language Models via Fine-grained Supervision
by: Xu, Dehong, et al.
Published: (2024)

VoiceTextBlender: Augmenting Large Language Models with Speech Capabilities via Single-Stage Joint Speech-Text Supervised Fine-Tuning
by: Peng, Yifan, et al.
Published: (2024)

Training and Inference Efficiency of Encoder-Decoder Speech Models
by: Żelasko, Piotr, et al.
Published: (2025)

Transducers with Pronunciation-aware Embeddings for Automatic Speech Recognition
by: Xu, Hainan, et al.
Published: (2024)

STORYSUMM: Evaluating Faithfulness in Story Summarization
by: Subbiah, Melanie, et al.
Published: (2024)

Fast Context-Biasing for CTC and Transducer ASR models with CTC-based Word Spotter
by: Andrusenko, Andrei, et al.
Published: (2024)

Graph-Enhanced LLM for SWAN-ISAC
by: Gao, Qian, et al.
Published: (2026)

TurboBias: Universal ASR Context-Biasing powered by GPU-accelerated Phrase-Boosting Tree
by: Andrusenko, Andrei, et al.
Published: (2025)

A Chat About Boring Problems: Studying GPT-based text normalization
by: Zhang, Yang, et al.
Published: (2023)

SWAN: Semantic Watermarking with Abstract Meaning Representation
by: Ye, Ziping, et al.
Published: (2026)

Flexible Multichannel Speech Enhancement for Noise-Robust Frontend
by: Jukić, Ante, et al.
Published: (2024)

Scalable and Cost-Efficient de Novo Template-Based Molecular Generation
by: Gaiński, Piotr, et al.
Published: (2025)

Comparative Analysis of Deep Learning Models for Crop Disease Detection: A Transfer Learning Approach
by: Subramaniam, Saundarya, et al.
Published: (2025)

NGPU-LM: GPU-Accelerated N-Gram Language Model for Context-Biasing in Greedy ASR Decoding
by: Bataev, Vladimir, et al.
Published: (2025)