SpacTor-T5: Pre-training T5 Models with Span Corruption and Replaced Token Detection

Documenti analoghi

• SoftSRV: Learn to Generate Targeted Synthetic Data
  di: DeSalvo, Giulia, et al.
  Pubblicazione: (2024)
• Analyzing Similarity Metrics for Data Selection for Language Model Pretraining
  di: Sam, Dylan, et al.
  Pubblicazione: (2025)
• GIST: Greedy Independent Set Thresholding for Max-Min Diversification with Submodular Utility
  di: Fahrbach, Matthew, et al.
  Pubblicazione: (2024)
• DistillSpec: Improving Speculative Decoding via Knowledge Distillation
  di: Zhou, Yongchao, et al.
  Pubblicazione: (2023)
• Algorithms for Learning Kernels Based on Centered Alignment
  di: Cortes, Corinna, et al.
  Pubblicazione: (2012)