Cover Image

Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zong, Yongshuo, Bohdal, Ondrej, Yu, Tingyang, Yang, Yongxin, Hospedales, Timothy
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

VL-ICL Bench: The Devil in the Details of Multimodal In-Context Learning
by: Zong, Yongshuo, et al.
Published: (2024)

Fool Your (Vision and) Language Model With Embarrassingly Simple Permutations
by: Zong, Yongshuo, et al.
Published: (2023)

Memorized Images in Diffusion Models share a Subspace that can be Located and Deleted
by: Chavhan, Ruchika, et al.
Published: (2024)

Navigating Noise: A Study of How Noise Influences Generalisation and Calibration of Neural Networks
by: Ferianc, Martin, et al.
Published: (2023)

On the Limitations of General Purpose Domain Generalisation Methods
by: Gouk, Henry, et al.
Published: (2022)

Self-Supervised Multimodal Learning: A Survey
by: Zong, Yongshuo, et al.
Published: (2023)

Feed-Forward Latent Domain Adaptation
by: Bohdal, Ondrej, et al.
Published: (2022)

FairTune: Optimizing Parameter Efficient Fine Tuning for Fairness in Medical Image Analysis
by: Dutt, Raman, et al.
Published: (2023)

Benchmarking Multi-Image Understanding in Vision and Language Models: Perception, Knowledge, Reasoning, and Multi-Hop Reasoning
by: Zhao, Bingchen, et al.
Published: (2024)

Clustering-driven Memory Compression for On-device Large Language Models
by: Bohdal, Ondrej, et al.
Published: (2026)

MedVision: Dataset and Benchmark for Quantitative Medical Image Analysis
by: Yao, Yongcheng, et al.
Published: (2025)

K-Merge: Online Continual Merging of Adapters for On-device Large Language Models
by: Shenaj, Donald, et al.
Published: (2025)

Efficient Compositional Multi-tasking for On-device Large Language Models
by: Bohdal, Ondrej, et al.
Published: (2025)

A Stochastic Approach to Bi-Level Optimization for Hyperparameter Optimization and Meta Learning
by: Kim, Minyoung, et al.
Published: (2024)

Data-driven Clustering and Merging of Adapters for On-device Large Language Models
by: Bohdal, Ondrej, et al.
Published: (2026)

Sparse Gradient Compression for Fine-Tuning Large Language Models
by: Yang, David H., et al.
Published: (2025)

Multi-Level Safety Continual Projection for Fine-Tuned Large Language Models without Retraining
by: Han, Bing, et al.
Published: (2025)

FedHB: Hierarchical Bayesian Federated Learning
by: Kim, Minyoung, et al.
Published: (2023)

CG-TTRL: Context-Guided Test-Time Reinforcement Learning for On-Device Large Language Models
by: Hosseini, Peyman, et al.
Published: (2025)

Dissecting Fine-Tuning Unlearning in Large Language Models
by: Hong, Yihuai, et al.
Published: (2024)

Model Merging is Secretly Certifiable: Non-Vacuous Generalisation Bounds for Low-Shot Learning
by: Kim, Taehoon, et al.
Published: (2025)

Resource-Efficient Federated Fine-Tuning Large Language Models for Heterogeneous Data
by: Liu, Jun, et al.
Published: (2025)

Capacity Control is an Effective Memorization Mitigation Mechanism in Text-Conditional Diffusion Models
by: Dutt, Raman, et al.
Published: (2024)

MemControl: Mitigating Memorization in Diffusion Models via Automated Parameter Selection
by: Dutt, Raman, et al.
Published: (2024)

DGLight: DQN-Guided GRPO Fine-Tuning of Large Language Models for Traffic Signal Control
by: Yu, Chenbo
Published: (2026)

PREFINE: Preference-Based Implicit Reward and Cost Fine-Tuning for Safety Alignment
by: Verma, Richa, et al.
Published: (2026)

Differentially Private Subspace Fine-Tuning for Large Language Models
by: Zheng, Lele, et al.
Published: (2026)

Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbation
by: Liu, Guozhi, et al.
Published: (2024)

Self-Generative Adversarial Fine-Tuning for Large Language Models
by: Wu, Shiguang, et al.
Published: (2026)

Decentralized Low-Rank Fine-Tuning of Large Language Models
by: Ghiasvand, Sajjad, et al.
Published: (2025)

Diversity in Large Language Models under Supervised Fine-Tuning
by: Klypa, Roman, et al.
Published: (2026)

Better Later Than Sooner: Neuro-Symbolic Knowledge Graph Construction via Ontology-grounded Post-extraction Correction
by: Loconte, Lorenzo, et al.
Published: (2026)

Testing the Limits of Fine-Tuning for Improving Visual Cognition in Vision Language Models
by: Buschoff, Luca M. Schulze, et al.
Published: (2025)

Few-Shot Adversarial Low-Rank Fine-Tuning of Vision-Language Models
by: Ghiasvand, Sajjad, et al.
Published: (2025)

On the Transferability of Large-Scale Self-Supervision to Few-Shot Audio Classification
by: Heggan, Calum, et al.
Published: (2024)

Linearization Explains Fine-Tuning in Large Language Models
by: Afzal, Zahra Rahimi, et al.
Published: (2026)

ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron Pruning
by: Chavhan, Ruchika, et al.
Published: (2024)

Large Language-Geometry Model: When LLM meets Equivariance
by: Li, Zongzhao, et al.
Published: (2025)

Communication-Efficient and Tensorized Federated Fine-Tuning of Large Language Models
by: Ghiasvand, Sajjad, et al.
Published: (2024)

SafeCOMM: A Study on Safety Degradation in Fine-Tuned Telecom Large Language Models
by: Djuhera, Aladin, et al.
Published: (2025)