Imagen de Portada

MAGNET: Autonomous Expert Model Generation via Decentralized Autoresearch and BitNet Training

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Kim, Yongwan, Park, Sungchul
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Continual Quantization-Aware Pre-Training: When to transition from 16-bit to 1.58-bit pre-training for BitNet language models?
por: Nielsen, Jacob, et al.
Publicado: (2025)

BitNet Distillation
por: Wu, Xun, et al.
Publicado: (2025)

BitNet b1.58 Reloaded: State-of-the-art Performance Also on Smaller Networks
por: Nielsen, Jacob, et al.
Publicado: (2024)

Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMs
por: Panfilov, Alexander, et al.
Publicado: (2026)

PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference
por: Zhao, Yushu, et al.
Publicado: (2025)

Improving SMOTE via Fusing Conditional VAE for Data-adaptive Noise Filtering
por: Hong, Sungchul, et al.
Publicado: (2024)

LittleBit: Ultra Low-Bit Quantization via Latent Factorization
por: Lee, Banseok, et al.
Publicado: (2025)

LittleBit-2: Maximizing the Spectral Energy Gain in Sub-1-Bit LLMs via Latent Geometry Alignment
por: Lee, Banseok, et al.
Publicado: (2026)

Bilevel Autoresearch: Meta-Autoresearching Itself
por: Qu, Yaonan, et al.
Publicado: (2026)

Transcendence: Generative Models Can Outperform The Experts That Train Them
por: Zhang, Edwin, et al.
Publicado: (2024)

Outlier-Safe Pre-Training for Robust 4-Bit Quantization of Large Language Models
por: Park, Jungwoo, et al.
Publicado: (2025)

Federated Domain Generalization with Label Smoothing and Balanced Decentralized Training
por: Soltany, Milad, et al.
Publicado: (2024)

dFLMoE: Decentralized Federated Learning via Mixture of Experts for Medical Data Analysis
por: Xie, Luyuan, et al.
Publicado: (2025)

How Many Experts Are Enough? Towards Optimal Semantic Specialization for Mixture-of-Experts
por: Park, Sumin, et al.
Publicado: (2025)

Learning to Specialize: Joint Gating-Expert Training for Adaptive MoEs in Decentralized Settings
por: Farhat, Yehya, et al.
Publicado: (2023)

BitNet b1.58 2B4T Technical Report
por: Ma, Shuming, et al.
Publicado: (2025)

MixtureKit: A General Framework for Composing, Training, and Visualizing Mixture-of-Experts Models
por: Chamma, Ahmad, et al.
Publicado: (2025)

Decentralized Adversarial Training over Graphs
por: Cao, Ying, et al.
Publicado: (2023)

Local Mixtures of Experts: Essentially Free Test-Time Training via Model Merging
por: Bertolissi, Ryo, et al.
Publicado: (2025)

Task-Aware Virtual Training: Enhancing Generalization in Meta-Reinforcement Learning for Out-of-Distribution Tasks
por: Kim, Jeongmo, et al.
Publicado: (2025)

Uncovering Intra-expert Activation Sparsity for Efficient Mixture-of-Expert Model Execution
por: Park, Jongseok, et al.
Publicado: (2026)

Attn-QAT: 4-Bit Attention With Quantization-Aware Training
por: Zhang, Peiyuan, et al.
Publicado: (2026)

AdaQAT: Adaptive Bit-Width Quantization-Aware Training
por: Gernigon, Cédric, et al.
Publicado: (2024)

DAMEL: Dual-Axis Multi-Expert Learning for Class-Imbalanced Learning
por: Lee, Hyuck, et al.
Publicado: (2026)

Load-Aware Training Scheduling for Model Circulation-based Decentralized Federated Learning
por: Kainuma, Haruki, et al.
Publicado: (2025)

Quant-dLLM: Post-Training Extreme Low-Bit Quantization for Diffusion Large Language Models
por: Zhang, Tianao, et al.
Publicado: (2025)

BitNet a4.8: 4-bit Activations for 1-bit LLMs
por: Wang, Hongyu, et al.
Publicado: (2024)

Decentralized Autoregressive Generation
por: Maschan, Stepan, et al.
Publicado: (2026)

Loop Corrections to the Training Error and Generalization Gap of Random Feature Models
por: Kim, Taeyoung
Publicado: (2026)

Modeling Expert Interactions in Sparse Mixture of Experts via Graph Structures
por: Nguyen-Nhat, Minh-Khoi, et al.
Publicado: (2025)

Characterization of Multi-Model Agentic AI Systems on General Tasks via Trace-Driven Simulation
por: Kim, Donghwan, et al.
Publicado: (2026)

Exploring Expert Specialization through Unsupervised Training in Sparse Mixture of Experts
por: Nikolic, Strahinja, et al.
Publicado: (2025)

Enhancing Autonomous Vehicle Training with Language Model Integration and Critical Scenario Generation
por: Tian, Hanlin, et al.
Publicado: (2024)

PFedDST: Personalized Federated Learning with Decentralized Selection Training
por: Fan, Mengchen, et al.
Publicado: (2025)

Efficient Epistemic Uncertainty Estimation for Large Language Models via Knowledge Distillation
por: Park, Seonghyeon, et al.
Publicado: (2026)

HeRo-Q: A General Framework for Stable Low Bit Quantization via Hessian Conditioning
por: Zhang, Jinhao Zhang Yunquan, et al.
Publicado: (2026)

When are 1.58 bits enough? A Bottom-up Exploration of BitNet Quantization
por: Nielsen, Jacob, et al.
Publicado: (2024)

LCD: Advancing Extreme Low-Bit Clustering for Large Language Models via Knowledge Distillation
por: Liu, Fangxin, et al.
Publicado: (2025)

Decentralized Domain Generalization with Style Sharing: Formal Model and Convergence Analysis
por: Zehtabi, Shahryar, et al.
Publicado: (2025)

PTQ1.61: Push the Real Limit of Extremely Low-Bit Post-Training Quantization Methods for Large Language Models
por: Zhao, Jiaqi, et al.
Publicado: (2025)