Cover Image

Multi-Task Pre-Finetuning of Lightweight Transformer Encoders for Text Classification and NER

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhu, Junyi, Ozkan, Savas, Maracani, Andrea, Mutlu, Sinan, Min, Cho Jung, Ozay, Mete
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Decoding Text Spans for Efficient and Accurate Named-Entity Recognition
by: Maracani, Andrea, et al.
Published: (2026)

Accurate Scene Text Recognition with Efficient Model Scaling and Cloze Self-Distillation
by: Maracani, Andrea, et al.
Published: (2025)

Efficient and Accurate Scene Text Recognition with Cascaded-Transformers
by: Ozkan, Savas, et al.
Published: (2025)

Geometrically Consistent Multi-View Scene Generation from Freehand Sketches
by: Bourouis, Ahmed, et al.
Published: (2026)

Fine-tuning Transformer-based Encoder for Turkish Language Understanding Tasks
by: Yildirim, Savas
Published: (2024)

HOP to the Next Tasks and Domains for Continual Learning in NLP
by: Michieli, Umberto, et al.
Published: (2024)

Guided Model Merging for Hybrid Data Learning: Leveraging Centralized Data to Refine Decentralized Models
by: Zhu, Junyi, et al.
Published: (2025)

Efficient 3D Full-Body Motion Generation from Sparse Tracking Inputs with Temporal Windows
by: Angelis, Georgios Fotios, et al.
Published: (2025)

Mem-MLP: Real-Time 3D Human Motion Generation from Sparse Inputs
by: Mutlu, Sinan, et al.
Published: (2025)

DreamCache: Finetuning-Free Lightweight Personalized Image Generation via Feature Caching
by: Aiello, Emanuele, et al.
Published: (2024)

NuNER: Entity Recognition Encoder Pre-training via LLM-Annotated Data
by: Bogdanov, Sergei, et al.
Published: (2024)

GLiNER multi-task: Generalist Lightweight Model for Various Information Extraction Tasks
by: Stepanov, Ihor, et al.
Published: (2024)

Efficient Compositional Multi-tasking for On-device Large Language Models
by: Bohdal, Ondrej, et al.
Published: (2025)

Clustering-driven Memory Compression for On-device Large Language Models
by: Bohdal, Ondrej, et al.
Published: (2026)

LoRA-Guard: Parameter-Efficient Guardrail Adaptation for Content Moderation of Large Language Models
by: Elesedy, Hayder, et al.
Published: (2024)

GLiNER2: An Efficient Multi-Task Information Extraction System with Schema-Driven Interface
by: Zaratiana, Urchade, et al.
Published: (2025)

LSTM-Based Text Generation: A Study on Historical Datasets
by: Hussein, Mustafa Abbas Hussein, et al.
Published: (2024)

OpenMed NER: Open-Source, Domain-Adapted State-of-the-Art Transformers for Biomedical NER Across 12 Public Datasets
by: Panahi, Maziyar
Published: (2025)

K-Merge: Online Continual Merging of Adapters for On-device Large Language Models
by: Shenaj, Donald, et al.
Published: (2025)

WojoodNER 2024: The Second Arabic Named Entity Recognition Shared Task
by: Jarrar, Mustafa, et al.
Published: (2024)

LegalTurk Optimized BERT for Multi-Label Text Classification and NER
by: Zeidi, Farnaz, et al.
Published: (2024)

Robo-Instruct: Simulator-Augmented Instruction Alignment For Finetuning Code LLMs
by: Hu, Zichao, et al.
Published: (2024)

StaICC: Standardized Evaluation for Classification Task in In-context Learning
by: Cho, Hakaze, et al.
Published: (2025)

WikiNER-fr-gold: A Gold-Standard NER Corpus
by: Cao, Danrun, et al.
Published: (2024)

2M-NER: Contrastive Learning for Multilingual and Multimodal NER with Language and Modal Fusion
by: Wang, Dongsheng, et al.
Published: (2024)

GLiClass: Generalist Lightweight Model for Sequence Classification Tasks
by: Stepanov, Ihor, et al.
Published: (2025)

Improving Task Diversity in Label Efficient Supervised Finetuning of LLMs
by: Arabelly, Abhinav, et al.
Published: (2025)

CoEvol: Constructing Better Responses for Instruction Finetuning through Multi-Agent Cooperation
by: Li, Renhao, et al.
Published: (2024)

Do LLMs Surpass Encoders for Biomedical NER?
by: Obeidat, Motasem S, et al.
Published: (2025)

Zero-Shot Cross-Lingual NER Using Phonemic Representations for Low-Resource Languages
by: Sohn, Jimin, et al.
Published: (2024)

A Model for Every User and Budget: Label-Free and Personalized Mixed-Precision Quantization
by: Fish, Edward, et al.
Published: (2023)

LabelFusion: Fusing Large Language Models with Transformer Encoders for Robust Financial News Classification
by: Schlee, Michael, et al.
Published: (2025)

Jatmo: Prompt Injection Defense by Task-Specific Finetuning
by: Piet, Julien, et al.
Published: (2023)

Using Large Language Model for End-to-End Chinese ASR and NER
by: Li, Yuang, et al.
Published: (2024)

Towards Few-Shot Adaptation of Foundation Models via Multitask Finetuning
by: Xu, Zhuoyan, et al.
Published: (2024)

Data-driven Clustering and Merging of Adapters for On-device Large Language Models
by: Bohdal, Ondrej, et al.
Published: (2026)

Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
by: Warner, Benjamin, et al.
Published: (2024)

NER- RoBERTa: Fine-Tuning RoBERTa for Named Entity Recognition (NER) within low-resource languages
by: Abdullah, Abdulhady Abas, et al.
Published: (2024)

Token Masking Improves Transformer-Based Text Classification
by: Xu, Xianglong, et al.
Published: (2025)

EEG2TEXT: Open Vocabulary EEG-to-Text Decoding with EEG Pre-Training and Multi-View Transformer
by: Liu, Hanwen, et al.
Published: (2024)