Image de couverture de livre

Self-Adjust Softmax

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zheng, Chuanyang, Gao, Yihang, Chen, Guoxuan, Shi, Han, Xiong, Jing, Ren, Xiaozhe, Huang, Chao, Jiang, Xin, Li, Zhenguo, Li, Yu
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

DAPE: Data-Adaptive Positional Encoding for Length Extrapolation
par: Zheng, Chuanyang, et autres
Publié: (2024)

DAPE V2: Process Attention Score as Feature Map for Length Extrapolation
par: Zheng, Chuanyang, et autres
Publié: (2024)

SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator
par: Chen, Guoxuan, et autres
Publié: (2024)

Progressive-Hint Prompting Improves Reasoning in Large Language Models
par: Zheng, Chuanyang, et autres
Publié: (2023)

DESTEIN: Navigating Detoxification of Language Models via Universal Steering Pairs and Head-wise Activation Fusion
par: Li, Yu, et autres
Publié: (2024)

Lyra: Orchestrating Dual Correction in Automated Theorem Proving
par: Zheng, Chuanyang, et autres
Publié: (2023)

UniAttn: Reducing Inference Costs via Softmax Unification for Post-Training LLMs
par: Xiong, Yizhe, et autres
Publié: (2025)

Partially Recentralization Softmax Loss for Vision-Language Models Robustness
par: Wang, Hao, et autres
Publié: (2024)

Answer-Consistent Chain-of-thought Reinforcement Learning For Multi-modal Large Langauge Models
par: Huang, Minbin, et autres
Publié: (2025)

SAS: Simulated Attention Score
par: Zheng, Chuanyang, et autres
Publié: (2025)

Understanding the Mixture-of-Experts with Nadaraya-Watson Kernel
par: Zheng, Chuanyang, et autres
Publié: (2025)

CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibration
par: Gao, Jiahui, et autres
Publié: (2024)

Scaling Law for Language Models Training Considering Batch Size
par: Shuai, Xian, et autres
Publié: (2024)

GeoNorm: Unify Pre-Norm and Post-Norm with Geodesic Optimization
par: Zheng, Chuanyang, et autres
Publié: (2026)

Dream 7B: Diffusion Large Language Models
par: Ye, Jiacheng, et autres
Publié: (2025)

BYOM: Building Your Own Multi-Task Model For Free
par: Jiang, Weisen, et autres
Publié: (2023)

Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning
par: Ye, Jiacheng, et autres
Publié: (2024)

Query-focused and Memory-aware Reranker for Long Context Processing
par: Li, Yuqing, et autres
Publié: (2026)

DQ-LoRe: Dual Queries with Low Rank Approximation Re-ranking for In-Context Learning
par: Xiong, Jing, et autres
Publié: (2023)

ExDR: Explanation-driven Dynamic Retrieval Enhancement for Multimodal Fake News Detection
par: Ding, Guoxuan, et autres
Publié: (2026)

Efficient Multi-modal Large Language Models via Visual Token Grouping
par: Huang, Minbin, et autres
Publié: (2024)

BLASST: Dynamic BLocked Attention Sparsity via Softmax Thresholding
par: Yuan, Jiayi, et autres
Publié: (2025)

Cubit: Token Mixer with Kernel Ridge Regression
par: Zheng, Chuanyang, et autres
Publié: (2026)

Mixture of insighTful Experts (MoTE): The Synergy of Thought Chains and Expert Mixtures in Self-Alignment
par: Liu, Zhili, et autres
Publié: (2024)

FlyRoute: Self-Evolving Agent Profiling via Data Flywheel for Adaptive Task Routing
par: Li, Rongjun, et autres
Publié: (2026)

Forward-Backward Reasoning in Large Language Models for Mathematical Verification
par: Jiang, Weisen, et autres
Publié: (2023)

Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models
par: Ye, Jiacheng, et autres
Publié: (2024)

UncertaintyRAG: Span-Level Uncertainty Enhanced Long-Context Modeling for Retrieval-Augmented Generation
par: Li, Zixuan, et autres
Publié: (2024)

Align to the Pivot: Dual Alignment with Self-Feedback for Multilingual Math Reasoning
par: Zhao, Chunxu, et autres
Publié: (2026)

Forewarned is Forearmed: Leveraging LLMs for Data Synthesis through Failure-Inducing Exploration
par: Li, Qintong, et autres
Publié: (2024)

Modality Bias in LVLMs: Analyzing and Mitigating Object Hallucination via Attention Lens
par: Zheng, Haohan, et autres
Publié: (2025)

MM-LIMA: Less Is More for Alignment in Multi-Modal Datasets
par: Wei, Lai, et autres
Publié: (2023)

DeepDiver: Adaptive Search Intensity Scaling via Open-Web Reinforcement Learning
par: Shi, Wenxuan, et autres
Publié: (2025)

ParallelComp: Parallel Long-Context Compressor for Length Extrapolation
par: Xiong, Jing, et autres
Publié: (2025)

Rethinking Meeting Effectiveness: A Benchmark and Framework for Temporal Fine-grained Automatic Meeting Effectiveness Evaluation
par: Li, Yihang, et autres
Publié: (2026)

Personality Editing for Language Models through Adjusting Self-Referential Queries
par: Hwang, Seojin, et autres
Publié: (2025)

Softmax Transformers are Turing-Complete
par: Jiang, Hongjian, et autres
Publié: (2025)

MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models
par: Yu, Longhui, et autres
Publié: (2023)

SPICE: Self-Play In Corpus Environments Improves Reasoning
par: Liu, Bo, et autres
Publié: (2025)

Dream-Coder 7B: An Open Diffusion Language Model for Code
par: Xie, Zhihui, et autres
Publié: (2025)