Imagen de Portada

Wisdom of Committee: Distilling from Foundation Model to Specialized Application Model

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Liu, Zichang, Liu, Qingyun, Li, Yuening, Liu, Liang, Shrivastava, Anshumali, Bi, Shuchao, Hong, Lichan, Chi, Ed H., Zhao, Zhe
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

LEVI: Generalizable Fine-tuning via Layer-wise Ensemble of Different Views
por: Roh, Yuji, et al.
Publicado: (2024)

Bridging the Gap: Unpacking the Hidden Challenges in Knowledge Distillation for Online Ranking Systems
por: Khani, Nikhil, et al.
Publicado: (2024)

To Compress or Not? Pushing the Frontier of Lossless GenAI Model Weights Compression with Exponent Concentration
por: Yang, Zeyu, et al.
Publicado: (2025)

Superintelligent Retrieval Agent: The Next Frontier of Information Retrieval
por: Yang, Zeyu, et al.
Publicado: (2026)

STAR: A Simple Training-free Approach for Recommendations using Large Language Models
por: Lee, Dong-Ho, et al.
Publicado: (2024)

Reward Modeling with Ordinal Feedback: Wisdom of the Crowd
por: Liu, Shang, et al.
Publicado: (2024)

Less is More: Unlocking Specialization of Time Series Foundation Models via Structured Pruning
por: Zhao, Lifan, et al.
Publicado: (2025)

Dataset Distillation via Committee Voting
por: Cui, Jiacheng, et al.
Publicado: (2025)

RACE Attention: A Strictly Linear-Time Attention Layer for Training on Outrageously Large Contexts
por: Joshi, Sahil, et al.
Publicado: (2025)

Scout Before You Attend: Sketch-and-Walk Sparse Attention for Efficient LLM Inference
por: Le, Hoang Anh Duy, et al.
Publicado: (2026)

NoMAD-Attention: Efficient LLM Inference on CPUs Through Multiply-add-free Attention
por: Zhang, Tianyi, et al.
Publicado: (2024)

WisdomInterrogatory (LuWen): An Open-Source Legal Large Language Model Technical Report
por: Wu, Yiquan, et al.
Publicado: (2026)

Multimodal Robust Prompt Distillation for 3D Point Cloud Models
por: Gu, Xiang, et al.
Publicado: (2025)

REFRAG: Rethinking RAG based Decoding
por: Lin, Xiaoqiang, et al.
Publicado: (2025)

Survey of Specialized Large Language Model
por: Yang, Chenghan, et al.
Publicado: (2025)

FoMEMO: Towards Foundation Models for Expensive Multi-objective Optimization
por: Yao, Yiming, et al.
Publicado: (2025)

ModelWisdom: An Integrated Toolkit for TLA+ Model Visualization, Digest and Repair
por: Chen, Zhiyong, et al.
Publicado: (2026)

CIARD: Cyclic Iterative Adversarial Robustness Distillation
por: Lu, Liming, et al.
Publicado: (2025)

Data-Efficient Symbolic Regression via Foundation Model Distillation
por: Ying, Wangyang, et al.
Publicado: (2025)

MemCollab: Cross-Model Memory Collaboration via Contrastive Trajectory Distillation
por: Chang, Yurui, et al.
Publicado: (2026)

Gaining Wisdom from Setbacks: Aligning Large Language Models via Mistake Analysis
por: Chen, Kai, et al.
Publicado: (2023)

Visual Large Language Models for Generalized and Specialized Applications
por: Li, Yifan, et al.
Publicado: (2025)

Piculet: Specialized Models-Guided Hallucination Decrease for MultiModal Large Language Models
por: Wang, Kohou, et al.
Publicado: (2024)

Offline Training of Language Model Agents with Functions as Learnable Weights
por: Zhang, Shaokun, et al.
Publicado: (2024)

LLMs for User Interest Exploration in Large-scale Recommendation Systems
por: Wang, Jianling, et al.
Publicado: (2024)

scDD: Latent Codes Based scRNA-seq Dataset Distillation with Foundation Model Knowledge
por: Yu, Zhen, et al.
Publicado: (2025)

The Illusion of Specialization: Unveiling the Domain-Invariant "Standing Committee" in Mixture-of-Experts Models
por: Wang, Yan, et al.
Publicado: (2026)

Emergent Specialization: Rare Token Neurons in Language Models
por: Liu, Jing, et al.
Publicado: (2025)

Pairwise Judgment Formulation for Semantic Embedding Model in Web Search
por: Hong, Mengze, et al.
Publicado: (2024)

Advancing Financial Engineering with Foundation Models: Progress, Applications, and Challenges
por: Chen, Liyuan, et al.
Publicado: (2025)

Distributed Specialization: Rare-Token Neurons in Large Language Models
por: Liu, Jing, et al.
Publicado: (2025)

Foundation Models for Scientific Discovery: From Paradigm Enhancement to Paradigm Transition
por: Liu, Fan, et al.
Publicado: (2025)

Foundation Models Knowledge Distillation For Battery Capacity Degradation Forecast
por: Chan, Joey, et al.
Publicado: (2025)

LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Grid
por: Zhang, Tianyi, et al.
Publicado: (2024)

Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RL
por: Li, Weizhen, et al.
Publicado: (2025)

The Shape of Wisdom: Decision Trajectories in Language Models
por: Rana, Shailesh
Publicado: (2026)

PHYSICS: Benchmarking Foundation Models on University-Level Physics Problem Solving
por: Feng, Kaiyue, et al.
Publicado: (2025)

RAG-GFM: Overcoming In-Memory Bottlenecks in Graph Foundation Models via Retrieval-Augmented Generation
por: Yuan, Haonan, et al.
Publicado: (2026)

Wisdom of the Crowd: Reinforcement Learning from Coevolutionary Collective Feedback
por: Yuan, Wenzhen, et al.
Publicado: (2025)

Large Multimodal Model Compression via Efficient Pruning and Distillation at AntGroup
por: Wang, Maolin, et al.
Publicado: (2023)