Imagen de Portada

Cross-Architecture Transfer Learning for Linear-Cost Inference Transformers

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autor principal:	Choi, Sehyun
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language Artificial Intelligence Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Decoupling Knowledge and Reasoning in Transformers: A Modular Architecture with Generalized Cross-Attention
por: Guo, Zhenyu, et al.
Publicado: (2025)

Your Transformer is Secretly Linear
por: Razzhigaev, Anton, et al.
Publicado: (2024)

Archon: An Architecture Search Framework for Inference-Time Techniques
por: Saad-Falcon, Jon, et al.
Publicado: (2024)

The Markovian Thinker: Architecture-Agnostic Linear Scaling of Reasoning
por: Aghajohari, Milad, et al.
Publicado: (2025)

Transformers Learn to Achieve Second-Order Convergence Rates for In-Context Linear Regression
por: Fu, Deqing, et al.
Publicado: (2023)

Does Transformer Interpretability Transfer to RNNs?
por: Paulo, Gonçalo, et al.
Publicado: (2024)

Green Prompting: Characterizing Prompt-driven Energy Costs of LLM Inference
por: Adamska, Marta, et al.
Publicado: (2025)

Supernova: Achieving More with Less in Transformer Architectures
por: Tanase, Andrei-Valentin, et al.
Publicado: (2025)

Residual Stream Duality in Modern Transformer Architectures
por: Zhang, Yifan
Publicado: (2026)

Pyramid MoA: A Probabilistic Framework for Cost-Optimized Anytime Inference
por: Khaled, Arindam
Publicado: (2026)

Peri-LN: Revisiting Normalization Layer in the Transformer Architecture
por: Kim, Jeonghoon, et al.
Publicado: (2025)

HARP: Hesitation-Aware Reframing in Transformer Inference Pass
por: Storaï, Romain, et al.
Publicado: (2024)

On the Robustness of Transformers against Context Hijacking for Linear Classification
por: Li, Tianle, et al.
Publicado: (2025)

Accelerating Transformer Inference for Translation via Parallel Decoding
por: Santilli, Andrea, et al.
Publicado: (2023)

Language-specific Neurons Do Not Facilitate Cross-Lingual Transfer
por: Mondal, Soumen Kumar, et al.
Publicado: (2025)

The Dual-Stream Transformer: Channelized Architecture for Interpretable Language Modeling
por: Kerce, J. Clayton, et al.
Publicado: (2026)

Linear Representation Transferability Hypothesis: Leveraging Small Models to Steer Large Models
por: Bello, Femi, et al.
Publicado: (2025)

Block Transformer: Global-to-Local Language Modeling for Fast Inference
por: Ho, Namgyu, et al.
Publicado: (2024)

KV Cache Transform Coding for Compact Storage in LLM Inference
por: Staniszewski, Konrad, et al.
Publicado: (2025)

A Differentiable Integer Linear Programming Solver for Explanation-Based Natural Language Inference
por: Thayaparan, Mokanarangan, et al.
Publicado: (2024)

Analyzing the Evaluation of Cross-Lingual Knowledge Transfer in Multilingual Language Models
por: Rajaee, Sara, et al.
Publicado: (2024)

ORPO-Distill: Mixed-Policy Preference Optimization for Cross-Architecture LLM Distillation
por: Singh, Aasheesh, et al.
Publicado: (2025)

Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts
por: Chen, Yingfa, et al.
Publicado: (2026)

Multi-Layer Transformers Gradient Can be Approximated in Almost Linear Time
por: Liang, Yingyu, et al.
Publicado: (2024)

LEAP: Layer-wise Exit-Aware Pretraining for Efficient Transformer Inference
por: Kapadia, Shashank, et al.
Publicado: (2026)

PromptIntern: Saving Inference Costs by Internalizing Recurrent Prompt during Large Language Model Fine-tuning
por: Zou, Jiaru, et al.
Publicado: (2024)

Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models
por: Bandarkar, Lucas, et al.
Publicado: (2024)

SwiftKV: Fast Prefill-Optimized Inference with Knowledge-Preserving Model Transformation
por: Qiao, Aurick, et al.
Publicado: (2024)

mini-vec2vec: Scaling Universal Geometry Alignment with Linear Transformations
por: Dar, Guy
Publicado: (2025)

ReplaceMe: Network Simplification via Depth Pruning and Transformer Block Linearization
por: Shopkhoev, Dmitriy, et al.
Publicado: (2025)

Efficient Systematic Reviews: Literature Filtering with Transformers & Transfer Learning
por: Hawkins, John, et al.
Publicado: (2024)

Culinary Class Wars: Evaluating LLMs using ASH in Cuisine Transfer Task
por: Lee, Hoonick, et al.
Publicado: (2024)

What Is the Minimum Architecture for Prolepsis? Early Irrevocable Commitment Across Tasks in Small Transformers
por: Jacopin, Éric
Publicado: (2026)

Linear Chain Transformation: Expanding Optimization Dynamics for Fine-Tuning Large Language Models
por: Wang, Yulong, et al.
Publicado: (2024)

M2R2: Mixture of Multi-Rate Residuals for Efficient Transformer Inference
por: Bhendawade, Nikhil, et al.
Publicado: (2025)

Grouped Sequency-arranged Rotation: Optimizing Rotation Transformation for Quantization for Free
por: Choi, Euntae, et al.
Publicado: (2025)

LEIA: Facilitating Cross-lingual Knowledge Transfer in Language Models with Entity-based Data Augmentation
por: Yamada, Ikuya, et al.
Publicado: (2024)

Few-Shot Cross-Lingual Transfer for Prompting Large Language Models in Low-Resource Languages
por: Toukmaji, Christopher
Publicado: (2024)

Identifying the Correlation Between Language Distance and Cross-Lingual Transfer in a Multilingual Representation Space
por: Philippy, Fred, et al.
Publicado: (2023)

Inference and Verbalization Functions During In-Context Learning
por: Tao, Junyi, et al.
Publicado: (2024)