Copertina

Most Transformer Modifications Still Do Not Transfer at 1-3B: A 2020-2026 Update to Narang et al. (2021) with Downstream Evaluation and a Noise Floor

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhao, Yang, Lu, Jiahao, Huang, Bin, Zhang, Guhua, Zhou, Jie
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Downstream Transfer Attack: Adversarial Attacks on Downstream Models with Pre-trained Vision Transformers
di: Zheng, Weijie, et al.
Pubblicazione: (2024)

Industry Transformation 2026-2030: Which Sectors Will Be Most Transformed by AI
di: Ivchenko, Ihor
Pubblicazione: (2026)

Improving the Downstream Performance of Mixture-of-Experts Transformers via Weak Vanilla Transformers
di: Lu, Xin, et al.
Pubblicazione: (2024)

For Making the Most of College, It's Still Location, Location, Location
di: Carlson, Scott
Pubblicazione: (2013)

How Well Do Large-Scale Chemical Language Models Transfer to Downstream Tasks?
di: Sagawa, Tatsuya, et al.
Pubblicazione: (2026)

Editorial Commentary: Not Every Hill‐Sachs Needs Filling: Most Do, But Selective, Cost‐Conscious Remplissage Is Still Warranted
di: Philip J. Rosinsky
Pubblicazione: (2025)

The Small-Noise Limit of the Most Likely Element is the Most Likely Element in the Small-Noise Limit
di: Selk, Zachary, et al.
Pubblicazione: (2022)

Warm or Dry Springs (Still) Boost the Reproduction of Most Temperate Songbirds
di: Amaury Thepault, et al.
Pubblicazione: (2025)

Do Books Still Matter?
di: Aronson, Marc
Pubblicazione: (2007)

Measuring the Effect of Transcription Noise on Downstream Language Understanding Tasks
di: Shapira, Ori, et al.
Pubblicazione: (2025)

Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks
di: Chen, Hao, et al.
Pubblicazione: (2023)

From Abstract to Contextual: What LLMs Still Cannot Do in Mathematics
di: Cao, Bowen, et al.
Pubblicazione: (2026)

When Do LLM Preferences Predict Downstream Behavior?
di: Slama, Katarina, et al.
Pubblicazione: (2026)

Transferable Adversarial Attacks on SAM and Its Downstream Models
di: Xia, Song, et al.
Pubblicazione: (2024)

Pre-Trained Model Recommendation for Downstream Fine-tuning
di: Bai, Jiameng, et al.
Pubblicazione: (2024)

Empirical Study on Updating Key-Value Memories in Transformer Feed-forward Layers
di: Qiu, Zihan, et al.
Pubblicazione: (2024)

The COHERENT Experiment: 2026 Update
di: Adhikari, M., et al.
Pubblicazione: (2026)

Designing Preconditioners for SGD: Local Conditioning, Noise Floors, and Basin Stability
di: Scott, Mitchell, et al.
Pubblicazione: (2025)

Patrick-Mulholland/Martin_et_al_2026_Neuropharmacology: Martin_et_al_2026_Neuropharmacology
di: Patrick-Mulholland
Pubblicazione: (2026)

The Sexual Recidivism Rates of Women Are Still Low: An Updated Meta‐Analysis
di: R. Karl Hanson, et al.
Pubblicazione: (2025)

Cross-Correlation Periodograms with Decaying Noise Floor for Power Spectral Density Estimation
di: Magsino, Mark
Pubblicazione: (2026)

RF Desense significance and its impact on the EVM at Signal Near the Noise Floor
di: Vedala, Rajesh, et al.
Pubblicazione: (2025)

Post‐Translational Modifications in Cilia and Ciliopathies
di: Jie Ran, et al.
Pubblicazione: (2025)

In response to Shimizu et al. (2026)
di: Hayato Tsuno, et al.
Pubblicazione: (2026)

Aligning the Evaluation of Probabilistic Predictions with Downstream Value
di: Shahroudi, Novin, et al.
Pubblicazione: (2025)

The Still Point and the Liminal Character of Transformation
di: Steven Eliezer Zemmelman
Pubblicazione: (2025)

Lowering the Error Floor of Error Correction Code Transformer
di: Park, Taewoo, et al.
Pubblicazione: (2025)

NepfLab/Lee_et_al_2026: Lee_et_al_2026_data_v1.0
di: Ernie I. H. Lee
Pubblicazione: (2026)

Do Schools Still Need Brick-and-Mortar Libraries?
di: Johnson, Doug, et al.
Pubblicazione: (2009)

Transferring Knowledge from Large Foundation Models to Small Downstream Models
di: Qiu, Shikai, et al.
Pubblicazione: (2024)

The Assessment of Downstream Clinical Tasks in Cardiac Ultrasound Is Crucial for Segmentation Evaluation
di: Shihui Zhang, et al.
Pubblicazione: (2025)

Bayesian Floor Field: Transferring people flow predictions across environments
di: Verdoja, Francesco, et al.
Pubblicazione: (2022)

ozgeata/Kuzman_et_al_2026: Kuzman_et_al_2026_RuMP_v1.0.0
di: ozgeata
Pubblicazione: (2026)

Adam Converges Without Any Modification On Update Rules
di: Zhang, Yushun, et al.
Pubblicazione: (2026)

Still rethinking the J‐shaped curve: A commentary on Kember et al., 2024
di: Alex P. Miller
Pubblicazione: (2024)

DSevolve: Enabling Real-Time Adaptive Scheduling on Dynamic Shop Floor with LLM-Evolved Heuristic Portfolios
di: Huang, Jin, et al.
Pubblicazione: (2026)

Balancing Knowledge Updates: Toward Unified Modular Editing in LLMs
di: Liu, Jiahao, et al.
Pubblicazione: (2025)

Robust 6DoF Pose Estimation Against Depth Noise and a Comprehensive Evaluation on a Mobile Dataset
di: Huang, Zixun, et al.
Pubblicazione: (2023)

dataset bibliography Ciccone et al 2026
di: SIROLI, LORENZO, et al.
Pubblicazione: (2026)

Publication Resources for Adloff et al. 2026
di: Adloff, Markus, et al.
Pubblicazione: (2026)