Cover Image

Scaling Embedding Layers in Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yu, Da, Cohen, Edith, Ghazi, Badih, Huang, Yangsibo, Kamath, Pritish, Kumar, Ravi, Liu, Daogao, Zhang, Chiyuan
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuning
by: Chua, Lynn, et al.
Published: (2024)

Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Models
by: Chua, Lynn, et al.
Published: (2024)

Urania: Differentially Private Insights into AI Use
by: Liu, Daogao, et al.
Published: (2025)

Quantifying Cross-Modality Memorization in Vision-Language Models
by: Wen, Yuxin, et al.
Published: (2025)

On Convex Optimization with Semi-Sensitive Features
by: Ghazi, Badih, et al.
Published: (2024)

On Memorization of Large Language Models in Logical Reasoning
by: Xie, Chulin, et al.
Published: (2024)

Scalable DP-SGD: Shuffling vs. Poisson Subsampling
by: Chua, Lynn, et al.
Published: (2024)

How Private are DP-SGD Implementations?
by: Chua, Lynn, et al.
Published: (2024)

Denoising the US Census: Succinct Block Hierarchical Regression
by: Ghazi, Badih, et al.
Published: (2026)

Differentially Private Optimization with Sparse Gradients
by: Ghazi, Badih, et al.
Published: (2024)

Unlearn and Burn: Adversarial Machine Unlearning Requests Destroy Model Accuracy
by: Huang, Yangsibo, et al.
Published: (2024)

Linear-Time User-Level DP-SCO via Robust Statistics
by: Ghazi, Badih, et al.
Published: (2025)

Scaling Laws for Differentially Private Language Models
by: McKenna, Ryan, et al.
Published: (2025)

Balls-and-Bins Sampling for DP-SGD
by: Chua, Lynn, et al.
Published: (2024)

Evaluating Copyright Takedown Methods for Language Models
by: Wei, Boyi, et al.
Published: (2024)

Detecting Pretraining Data from Large Language Models
by: Shi, Weijia, et al.
Published: (2023)

PREM: Privately Answering Statistical Queries with Relative Error
by: Ghazi, Badih, et al.
Published: (2025)

Private Hyperparameter Tuning with Ex-Post Guarantee
by: Ghazi, Badih, et al.
Published: (2025)

Training Differentially Private Ad Prediction Models with Semi-Sensitive Features
by: Chua, Lynn, et al.
Published: (2024)

Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models
by: Bandarkar, Lucas, et al.
Published: (2024)

Localizing Paragraph Memorization in Language Models
by: Stoehr, Niklas, et al.
Published: (2024)

Computational Hardness of Private Coreset
by: Ghazi, Badih, et al.
Published: (2026)

MUSE: Machine Unlearning Six-Way Evaluation for Language Models
by: Shi, Weijia, et al.
Published: (2024)

Layer by Layer: Uncovering Where Multi-Task Learning Happens in Instruction-Tuned Large Language Models
by: Zhao, Zheng, et al.
Published: (2024)

Individualized Privacy Accounting via Subsampling with Applications in Combinatorial Optimization
by: Ghazi, Badih, et al.
Published: (2024)

On Computing Pairwise Statistics with Local Differential Privacy
by: Ghazi, Badih, et al.
Published: (2024)

Scaling Embeddings Outperforms Scaling Experts in Language Models
by: Liu, Hong, et al.
Published: (2026)

Sparse Layers are Critical to Scaling Looped Language Models
by: Lee, Ryan, et al.
Published: (2026)

Mixture of Chapters: Scaling Learnt Memory in Transformers
by: Tibrewal, Tasmay Pankaj, et al.
Published: (2026)

Differential Privacy on Trust Graphs
by: Ghazi, Badih, et al.
Published: (2024)

Investigating Layer Importance in Large Language Models
by: Zhang, Yang, et al.
Published: (2024)

Efficient Knowledge Probing of Large Language Models by Adapting Pre-trained Embeddings
by: Sharma, Kartik, et al.
Published: (2025)

Basis Sharing: Cross-Layer Parameter Sharing for Large Language Model Compression
by: Wang, Jingcun, et al.
Published: (2024)

Parallel Scaling Law for Language Models
by: Chen, Mouxiang, et al.
Published: (2025)

Repetition Improves Language Model Embeddings
by: Springer, Jacob Mitchell, et al.
Published: (2024)

Mechanistic Insights into Grokking from the Embedding Layer
by: AlquBoj, H. V., et al.
Published: (2025)

Generalizing Large Language Model Usability Across Resource-Constrained
by: Tsai, Yun-Da
Published: (2025)

Aligning Large Language Models to Low-Resource Languages through LLM-Based Selective Translation: A Systematic Study
by: Paul, Rakesh, et al.
Published: (2025)

How Unique is Whose Web Browser? The role of demographics in browser fingerprinting among US users
by: Berke, Alex, et al.
Published: (2024)

Spectral Insights into Data-Oblivious Critical Layers in Large Language Models
by: Liu, Xuyuan, et al.
Published: (2025)