Copertina

Functional Component Ablation Reveals Specialization Patterns in Hybrid Language Model Architectures

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Borobia, Hector, Seguí-Mas, Elies, Tormo-Carbó, Guillermina
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Computation and Language Artificial Intelligence Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Component-Aware Self-Speculative Decoding in Hybrid Language Models
di: Borobia, Hector, et al.
Pubblicazione: (2026)

Where Should LoRA Go? Component-Type Placement in Hybrid Language Models
di: Borobia, Hector, et al.
Pubblicazione: (2026)

How Pruning Reshapes Features: Sparse Autoencoder Analysis of Weight-Pruned Language Models
di: Borobia, Hector, et al.
Pubblicazione: (2026)

Linear-Readout Floors and Threshold Recovery in Computation in Superposition
di: Borobia, Hector, et al.
Pubblicazione: (2026)

Hymba: A Hybrid-head Architecture for Small Language Models
di: Dong, Xin, et al.
Pubblicazione: (2024)

Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models
di: Wang, Zihan, et al.
Pubblicazione: (2024)

A Nurse is Blue and Elephant is Rugby: Cross Domain Alignment in Large Language Models Reveal Human-like Patterns
di: Yehudai, Asaf, et al.
Pubblicazione: (2024)

Simplifying Outcomes of Language Model Component Analyses with ELIA
di: Eidt, Aaron Louis, et al.
Pubblicazione: (2026)

Are Human Conversations Special? A Large Language Model Perspective
di: Jawale, Toshish, et al.
Pubblicazione: (2024)

Prescriptive Scaling Reveals the Evolution of Language Model Capabilities
di: Zhang, Hanlin, et al.
Pubblicazione: (2026)

SCALPEL: Selective Capability Ablation via Low-rank Parameter Editing for Large Language Model Interpretability Analysis
di: Fu, Zihao, et al.
Pubblicazione: (2026)

The Compositional Architecture of Regret in Large Language Models
di: Cui, Xiangxiang, et al.
Pubblicazione: (2025)

Untangling Component Imbalance in Hybrid Linear Attention Conversion Methods
di: Benfeghoul, Martin, et al.
Pubblicazione: (2025)

Sequences of Logits Reveal the Low Rank Structure of Language Models
di: Golowich, Noah, et al.
Pubblicazione: (2025)

Derivational Morphology Reveals Analogical Generalization in Large Language Models
di: Hofmann, Valentin, et al.
Pubblicazione: (2024)

Lost in the Prompt Order: Revealing the Limitations of Causal Attention in Language Models
di: Ok, Hyunjong, et al.
Pubblicazione: (2026)

When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboards
di: Alzahrani, Norah, et al.
Pubblicazione: (2024)

Revealing Behavioral Plasticity in Large Language Models: A Token-Conditional Perspective
di: Mao, Liyuan, et al.
Pubblicazione: (2026)

The Dual-Stream Transformer: Channelized Architecture for Interpretable Language Modeling
di: Kerce, J. Clayton, et al.
Pubblicazione: (2026)

Small or Large? Zero-Shot or Finetuned? Guiding Language Model Choice for Specialized Applications in Healthcare
di: Gondara, Lovedeep, et al.
Pubblicazione: (2025)

Jet-Nemotron: Efficient Language Model with Post Neural Architecture Search
di: Gu, Yuxian, et al.
Pubblicazione: (2025)

Do Language Models Have Bayesian Brains? Distinguishing Stochastic and Deterministic Decision Patterns within Large Language Models
di: Cui, Andrea Yaoyun, et al.
Pubblicazione: (2025)

Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning
di: Ling Team, et al.
Pubblicazione: (2025)

Gumiho: A Hybrid Architecture to Prioritize Early Tokens in Speculative Decoding
di: Li, Jinze, et al.
Pubblicazione: (2025)

Synergy-of-Thoughts: Eliciting Efficient Reasoning in Hybrid Language Models
di: Shang, Yu, et al.
Pubblicazione: (2024)

LaFFi: Leveraging Hybrid Natural Language Feedback for Fine-tuning Language Models
di: Li, Qianxi, et al.
Pubblicazione: (2023)

PanGu-$π$ Pro:Rethinking Optimization and Architecture for Tiny Language Models
di: Tang, Yehui, et al.
Pubblicazione: (2024)

Multiscale Byte Language Models -- A Hierarchical Architecture for Causal Million-Length Sequence Modeling
di: Egli, Eric, et al.
Pubblicazione: (2025)

IAA: Inner-Adaptor Architecture Empowers Frozen Large Language Model with Multimodal Capabilities
di: Wang, Bin, et al.
Pubblicazione: (2024)

Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuning
di: Casademunt, Helena, et al.
Pubblicazione: (2025)

DASH: Fast Differentiable Architecture Search for Hybrid Attention in Minutes on a Single GPU
di: Chen, Weizhe, et al.
Pubblicazione: (2026)

Federated Learning-Enabled Hybrid Language Models for Communication-Efficient Token Transmission
di: Solat, Faranaksadat, et al.
Pubblicazione: (2025)

Pruning Large Language Models by Identifying and Preserving Functional Networks
di: Liu, Yiheng, et al.
Pubblicazione: (2025)

Correcting Large Language Model Behavior via Influence Function
di: Zhang, Han, et al.
Pubblicazione: (2024)

Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts
di: Chen, Yingfa, et al.
Pubblicazione: (2026)

Cognitive Architectures for Language Agents
di: Sumers, Theodore R., et al.
Pubblicazione: (2023)

Elastic Architecture Search for Efficient Language Models
di: Wang, Shang
Pubblicazione: (2025)

LayerIF: Estimating Layer Quality for Large Language Models using Influence Functions
di: Askari, Hadi, et al.
Pubblicazione: (2025)

Toward Adaptive Large Language Models Structured Pruning via Hybrid-grained Weight Importance Assessment
di: Liu, Jun, et al.
Pubblicazione: (2024)

Decoding Dark Matter: Specialized Sparse Autoencoders for Interpreting Rare Concepts in Foundation Models
di: Muhamed, Aashiq, et al.
Pubblicazione: (2024)