Copertina

Exploring Pretraining via Active Forgetting for Improving Cross Lingual Transfer for Decoder Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Aggarwal, Divyanshu, Sathe, Ashutosh, Sitaram, Sunayana
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Improving Self Consistency in LLMs through Probabilistic Tokenization
di: Sathe, Ashutosh, et al.
Pubblicazione: (2024)

MAPLE: Multilingual Evaluation of Parameter Efficient Finetuning of Large Language Models
di: Aggarwal, Divyanshu, et al.
Pubblicazione: (2024)

A Unified Framework and Dataset for Assessing Societal Bias in Vision-Language Models
di: Sathe, Ashutosh, et al.
Pubblicazione: (2024)

Exploring Continual Fine-Tuning for Enhancing Language Ability in Large Language Model
di: Aggarwal, Divyanshu, et al.
Pubblicazione: (2024)

MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks
di: Ahuja, Sanchit, et al.
Pubblicazione: (2023)

MAFIA: Multi-Adapter Fused Inclusive LanguAge Models
di: Jain, Prachi, et al.
Pubblicazione: (2024)

M5 -- A Diverse Benchmark to Assess the Performance of Large Multimodal Models Across Multilingual and Multicultural Vision-Language Tasks
di: Schneider, Florian, et al.
Pubblicazione: (2024)

Improving Language Plasticity via Pretraining with Active Forgetting
di: Chen, Yihong, et al.
Pubblicazione: (2023)

MS-HuBERT: Mitigating Pre-training and Inference Mismatch in Masked Language Modelling methods for learning Speech Representations
di: Yadav, Hemant, et al.
Pubblicazione: (2024)

Measuring Catastrophic Forgetting in Cross-Lingual Transfer Paradigms: Exploring Tuning Strategies
di: Koloski, Boshko, et al.
Pubblicazione: (2023)

Efficient Training of Language Models with Compact and Consistent Next Token Distributions
di: Sathe, Ashutosh, et al.
Pubblicazione: (2024)

mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Models
di: Lin, Peiqin, et al.
Pubblicazione: (2023)

Contamination Report for Multilingual Benchmarks
di: Ahuja, Sanchit, et al.
Pubblicazione: (2024)

HEALTH-PARIKSHA: Assessing RAG Models for Health Chatbots in Real-World Multilingual Settings
di: Gumma, Varun, et al.
Pubblicazione: (2024)

Speech Representation Learning Revisited: The Necessity of Separate Learnable Parameters and Robust Data Augmentation
di: Yadav, Hemant, et al.
Pubblicazione: (2024)

JOOCI: a Framework for Learning Comprehensive Speech Representations
di: Yadav, Hemant, et al.
Pubblicazione: (2024)

From FusHa to Folk: Exploring Cross-Lingual Transfer in Arabic Language Models
di: Khalak, Abdulmuizz, et al.
Pubblicazione: (2026)

Cross-Lingual Optimization for Language Transfer in Large Language Models
di: Lee, Jungseob, et al.
Pubblicazione: (2025)

Initial Decoding with Minimally Augmented Language Model for Improved Lattice Rescoring in Low Resource ASR
di: Murthy, Savitha, et al.
Pubblicazione: (2024)

Geometry of Decision Making in Language Models
di: Joshi, Abhinav, et al.
Pubblicazione: (2025)

CultureLLM: Incorporating Cultural Differences into Large Language Models
di: Li, Cheng, et al.
Pubblicazione: (2024)

DOSA: A Dataset of Social Artifacts from Different Indian Geographical Subcultures
di: Seth, Agrima, et al.
Pubblicazione: (2024)

Multilingual Pretraining and Instruction Tuning Improve Cross-Lingual Knowledge Alignment, But Only Shallowly
di: Gao, Changjiang, et al.
Pubblicazione: (2024)

An Efficient Approach for Studying Cross-Lingual Transfer in Multilingual Language Models
di: Faisal, Fahim, et al.
Pubblicazione: (2024)

Semantic Pivots Enable Cross-Lingual Transfer in Large Language Models
di: He, Kaiyu, et al.
Pubblicazione: (2025)

The Impact of Language Adapters in Cross-Lingual Transfer for NLU
di: Kunz, Jenny, et al.
Pubblicazione: (2024)

Cross-Lingual Transfer for Natural Language Inference via Multilingual Prompt Translator
di: Qiu, Xiaoyu, et al.
Pubblicazione: (2024)

Improving Zero-Shot Cross-Lingual Transfer via Progressive Code-Switching
di: Li, Zhuoran, et al.
Pubblicazione: (2024)

Model-Based Ranking of Source Languages for Zero-Shot Cross-Lingual Transfer
di: Ebrahimi, Abteen, et al.
Pubblicazione: (2025)

Analysis of Multi-Source Language Training in Cross-Lingual Transfer
di: Lim, Seong Hoon, et al.
Pubblicazione: (2024)

Cross-Lingual Transfer for Low-Resource Natural Language Processing
di: García-Ferrero, Iker
Pubblicazione: (2025)

Marvelous Agglutinative Language Effect on Cross Lingual Transfer Learning
di: Kim, Wooyoung, et al.
Pubblicazione: (2022)

Fluent but Foreign: Even Regional LLMs Lack Cultural Alignment
di: Agarwal, Dhruv, et al.
Pubblicazione: (2025)

Self-Translate-Train: Enhancing Cross-Lingual Transfer of Large Language Models via Inherent Capability
di: Ri, Ryokan, et al.
Pubblicazione: (2024)

Bridging the Language Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs
di: Kumar, Somnath, et al.
Pubblicazione: (2023)

Exploring Cross-Lingual Knowledge Transfer via Transliteration-Based MLM Fine-Tuning for Critically Low-resource Chakma Language
di: Khisa, Adity, et al.
Pubblicazione: (2025)

METAL: Towards Multilingual Meta-Evaluation
di: Hada, Rishav, et al.
Pubblicazione: (2024)

Cross-Lingual LLM-Judge Transfer via Evaluation Decomposition
di: Sheth, Ivaxi, et al.
Pubblicazione: (2026)

Why Better Cross-Lingual Alignment Fails for Better Cross-Lingual Transfer: Case of Encoders
di: Veitsman, Yana, et al.
Pubblicazione: (2026)

Cross-Lingual Transfer Robustness to Lower-Resource Languages on Adversarial Datasets
di: Manafi, Shadi, et al.
Pubblicazione: (2024)