Pre-trained Models Perform the Best When Token Distributions Follow Zipf's Law

Documenti analoghi

• When Models Can't Follow: Testing Instruction Adherence Across 256 LLMs
  di: Young, Richard J., et al.
  Pubblicazione: (2025)
• Entropy-Based Measurement of Value Drift and Alignment Work in Large Language Models
  di: Fadli, Samih
  Pubblicazione: (2025)
• Future Token Prediction -- Causal Language Modelling with Per-Token Semantic State Vector for Multi-Token Prediction
  di: Walker, Nicholas
  Pubblicazione: (2024)
• Kronecker Embeddings: Byte-Level Structured Token Representations for Parameter-Efficient Language Models
  di: Shravan, Rohan
  Pubblicazione: (2026)
• The Data Efficiency Frontier of Financial Foundation Models: Scaling Laws from Continued Pretraining
  di: Ponnock, Jesse
  Pubblicazione: (2025)