Copertina

Self-Adjust Softmax

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zheng, Chuanyang, Gao, Yihang, Chen, Guoxuan, Shi, Han, Xiong, Jing, Ren, Xiaozhe, Huang, Chao, Jiang, Xin, Li, Zhenguo, Li, Yu
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

DAPE: Data-Adaptive Positional Encoding for Length Extrapolation
di: Zheng, Chuanyang, et al.
Pubblicazione: (2024)

DAPE V2: Process Attention Score as Feature Map for Length Extrapolation
di: Zheng, Chuanyang, et al.
Pubblicazione: (2024)

SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator
di: Chen, Guoxuan, et al.
Pubblicazione: (2024)

Progressive-Hint Prompting Improves Reasoning in Large Language Models
di: Zheng, Chuanyang, et al.
Pubblicazione: (2023)

DESTEIN: Navigating Detoxification of Language Models via Universal Steering Pairs and Head-wise Activation Fusion
di: Li, Yu, et al.
Pubblicazione: (2024)

Lyra: Orchestrating Dual Correction in Automated Theorem Proving
di: Zheng, Chuanyang, et al.
Pubblicazione: (2023)

UniAttn: Reducing Inference Costs via Softmax Unification for Post-Training LLMs
di: Xiong, Yizhe, et al.
Pubblicazione: (2025)

Partially Recentralization Softmax Loss for Vision-Language Models Robustness
di: Wang, Hao, et al.
Pubblicazione: (2024)

Answer-Consistent Chain-of-thought Reinforcement Learning For Multi-modal Large Langauge Models
di: Huang, Minbin, et al.
Pubblicazione: (2025)

SAS: Simulated Attention Score
di: Zheng, Chuanyang, et al.
Pubblicazione: (2025)

Understanding the Mixture-of-Experts with Nadaraya-Watson Kernel
di: Zheng, Chuanyang, et al.
Pubblicazione: (2025)

CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibration
di: Gao, Jiahui, et al.
Pubblicazione: (2024)

Scaling Law for Language Models Training Considering Batch Size
di: Shuai, Xian, et al.
Pubblicazione: (2024)

GeoNorm: Unify Pre-Norm and Post-Norm with Geodesic Optimization
di: Zheng, Chuanyang, et al.
Pubblicazione: (2026)

Dream 7B: Diffusion Large Language Models
di: Ye, Jiacheng, et al.
Pubblicazione: (2025)

BYOM: Building Your Own Multi-Task Model For Free
di: Jiang, Weisen, et al.
Pubblicazione: (2023)

Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning
di: Ye, Jiacheng, et al.
Pubblicazione: (2024)

Query-focused and Memory-aware Reranker for Long Context Processing
di: Li, Yuqing, et al.
Pubblicazione: (2026)

DQ-LoRe: Dual Queries with Low Rank Approximation Re-ranking for In-Context Learning
di: Xiong, Jing, et al.
Pubblicazione: (2023)

ExDR: Explanation-driven Dynamic Retrieval Enhancement for Multimodal Fake News Detection
di: Ding, Guoxuan, et al.
Pubblicazione: (2026)

Efficient Multi-modal Large Language Models via Visual Token Grouping
di: Huang, Minbin, et al.
Pubblicazione: (2024)

BLASST: Dynamic BLocked Attention Sparsity via Softmax Thresholding
di: Yuan, Jiayi, et al.
Pubblicazione: (2025)

Cubit: Token Mixer with Kernel Ridge Regression
di: Zheng, Chuanyang, et al.
Pubblicazione: (2026)

Mixture of insighTful Experts (MoTE): The Synergy of Thought Chains and Expert Mixtures in Self-Alignment
di: Liu, Zhili, et al.
Pubblicazione: (2024)

FlyRoute: Self-Evolving Agent Profiling via Data Flywheel for Adaptive Task Routing
di: Li, Rongjun, et al.
Pubblicazione: (2026)

Forward-Backward Reasoning in Large Language Models for Mathematical Verification
di: Jiang, Weisen, et al.
Pubblicazione: (2023)

Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models
di: Ye, Jiacheng, et al.
Pubblicazione: (2024)

UncertaintyRAG: Span-Level Uncertainty Enhanced Long-Context Modeling for Retrieval-Augmented Generation
di: Li, Zixuan, et al.
Pubblicazione: (2024)

Align to the Pivot: Dual Alignment with Self-Feedback for Multilingual Math Reasoning
di: Zhao, Chunxu, et al.
Pubblicazione: (2026)

Forewarned is Forearmed: Leveraging LLMs for Data Synthesis through Failure-Inducing Exploration
di: Li, Qintong, et al.
Pubblicazione: (2024)

Modality Bias in LVLMs: Analyzing and Mitigating Object Hallucination via Attention Lens
di: Zheng, Haohan, et al.
Pubblicazione: (2025)

MM-LIMA: Less Is More for Alignment in Multi-Modal Datasets
di: Wei, Lai, et al.
Pubblicazione: (2023)

DeepDiver: Adaptive Search Intensity Scaling via Open-Web Reinforcement Learning
di: Shi, Wenxuan, et al.
Pubblicazione: (2025)

ParallelComp: Parallel Long-Context Compressor for Length Extrapolation
di: Xiong, Jing, et al.
Pubblicazione: (2025)

Rethinking Meeting Effectiveness: A Benchmark and Framework for Temporal Fine-grained Automatic Meeting Effectiveness Evaluation
di: Li, Yihang, et al.
Pubblicazione: (2026)

Personality Editing for Language Models through Adjusting Self-Referential Queries
di: Hwang, Seojin, et al.
Pubblicazione: (2025)

Softmax Transformers are Turing-Complete
di: Jiang, Hongjian, et al.
Pubblicazione: (2025)

MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models
di: Yu, Longhui, et al.
Pubblicazione: (2023)

SPICE: Self-Play In Corpus Environments Improves Reasoning
di: Liu, Bo, et al.
Pubblicazione: (2025)

Dream-Coder 7B: An Open Diffusion Language Model for Code
di: Xie, Zhihui, et al.
Pubblicazione: (2025)