Image de couverture de livre

Revisiting Funnel Transformers for Modern LLM Architectures with Comprehensive Ablations in Training and Inference Configurations

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Choi, DongHyun, Spangher, Lucas, Hidey, Chris, Grabowski, Peter, Eskander, Ramy
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Cross-Architecture Transfer Learning for Linear-Cost Inference Transformers
par: Choi, Sehyun
Publié: (2024)

LLM Cache Bandit Revisited: Addressing Query Heterogeneity for Cost-Effective LLM Inference
par: Yang, Hantao, et autres
Publié: (2025)

Residual Stream Duality in Modern Transformer Architectures
par: Zhang, Yifan
Publié: (2026)

PatentEdits: Framing Patent Novelty as Textual Entailment
par: Lee, Ryan, et autres
Publié: (2024)

Understanding LLMs: A Comprehensive Overview from Training to Inference
par: Liu, Yiheng, et autres
Publié: (2024)

Peri-LN: Revisiting Normalization Layer in the Transformer Architecture
par: Kim, Jeonghoon, et autres
Publié: (2025)

Revisiting Padded Transformer Expressivity: Which Architectural Choices Matter and Which Don't
par: Svete, Anej, et autres
Publié: (2026)

Improving LLM-as-a-Judge Inference with the Judgment Distribution
par: Wang, Victor, et autres
Publié: (2025)

Transforming Slot Schema Induction with Generative Dialogue State Inference
par: Finch, James D., et autres
Publié: (2024)

Applications of the Transformer Architecture in AI-Assisted English Reading Comprehension
par: Li, Ping
Publié: (2026)

PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling
par: Cai, Zefan, et autres
Publié: (2024)

Learning Action Conditions from Instructional Manuals for Instruction Understanding
par: Wu, Te-Lin, et autres
Publié: (2022)

Extra Global Attention Designation Using Keyword Detection in Sparse Transformer Architectures
par: Lucas, Evan, et autres
Publié: (2024)

DiffAdapt: Difficulty-Adaptive Reasoning for Token-Efficient LLM Inference
par: Liu, Xiang, et autres
Publié: (2025)

Mirror Speculative Decoding: Breaking the Serial Barrier in LLM Inference
par: Bhendawade, Nikhil, et autres
Publié: (2025)

ModernBERT or DeBERTaV3? Examining Architecture and Data Influence on Transformer Encoder Models Performance
par: Antoun, Wissam, et autres
Publié: (2025)

Factored Agents: Decoupling In-Context Learning and Memorization for Robust Tool Use
par: Roth, Nicholas, et autres
Publié: (2025)

A Survey on LLM Inference-Time Self-Improvement
par: Dong, Xiangjue, et autres
Publié: (2024)

Fast KVzip: Efficient and Accurate LLM Inference with Gated KV Eviction
par: Kim, Jang-Hyun, et autres
Publié: (2026)

RoBIn: A Transformer-Based Model For Risk Of Bias Inference With Machine Reading Comprehension
par: Dias, Abel Corrêa, et autres
Publié: (2024)

DiscoSum: Discourse-aware News Summarization
par: Spangher, Alexander, et autres
Publié: (2025)

Transparent Screening for LLM Inference and Training Impacts
par: Pachot, Arnault, et autres
Publié: (2026)

ArcLight: A Lightweight LLM Inference Architecture for Many-Core CPUs
par: Xu, Yuzhuang, et autres
Publié: (2026)

FunnelRAG: A Coarse-to-Fine Progressive Retrieval Paradigm for RAG
par: Zhao, Xinping, et autres
Publié: (2024)

Inferflow: an Efficient and Highly Configurable Inference Engine for Large Language Models
par: Shi, Shuming, et autres
Publié: (2024)

Revisiting Word Embeddings in the LLM Era
par: Mahajan, Yash, et autres
Publié: (2025)

Folding Tensor and Sequence Parallelism for Memory-Efficient Transformer Training & Inference
par: Shyam, Vasu, et autres
Publié: (2026)

NewsEdits 2.0: Learning the Intentions Behind Updating News
par: Spangher, Alexander, et autres
Publié: (2024)

Latent Recurrent Transformer: Architecture Exploration, Training Strategies, and Scaling Behavior
par: Huang, Zeyi, et autres
Publié: (2026)

TransformLLM: Adapting Large Language Models via LLM-Transformed Reading Comprehension Text
par: Arbel, Iftach, et autres
Publié: (2024)

Generalized Probabilistic Attention Mechanism in Transformers
par: Heo, DongNyeong, et autres
Publié: (2024)

Horizon-LM: A RAM-Centric Architecture for LLM Training
par: Yuan, Zhengqing, et autres
Publié: (2026)

Diagnosing Training Inference Mismatch in LLM Reinforcement Learning
par: Zhong, Tianle, et autres
Publié: (2026)

End-to-End Training for Back-Translation with Categorical Reparameterization Trick
par: Heo, DongNyeong, et autres
Publié: (2022)

Matryoshka Re-Ranker: A Flexible Re-Ranking Architecture With Configurable Depth and Width
par: Liu, Zheng, et autres
Publié: (2025)

Unraveling the Dominance of Large Language Models Over Transformer Models for Bangla Natural Language Inference: A Comprehensive Study
par: Faria, Fatema Tuj Johora, et autres
Publié: (2024)

Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads on Consumer-Grade Devices
par: Huang, Yuxiang, et autres
Publié: (2024)

Revisiting Hierarchical Text Classification: Inference and Metrics
par: Plaud, Roman, et autres
Publié: (2024)

Training Nonlinear Transformers for Chain-of-Thought Inference: A Theoretical Generalization Analysis
par: Li, Hongkang, et autres
Publié: (2024)

NewsHomepages: Homepage Layouts Capture Information Prioritization Decisions
par: Welsh, Ben, et autres
Publié: (2024)