Cover Image

Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Thangarasa, Vithursan, Saxena, Shreyas, Gupta, Abhay, Lie, Sean
Format:	Preprint
Published:	2023
Subjects:	Machine Learning Computation and Language Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MediSwift: Efficient Sparse Pre-trained Biomedical Language Models
by: Thangarasa, Vithursan, et al.
Published: (2024)

MASSV: Multimodal Adaptation and Self-Data Distillation for Speculative Decoding of Vision-Language Models
by: Ganesan, Mugilan, et al.
Published: (2025)

TapOut: A Bandit-Based Approach to Dynamic Speculative Decoding
by: Sridhar, Aditya, et al.
Published: (2025)

Self-Data Distillation for Recovering Quality in Pruned Large Language Models
by: Thangarasa, Vithursan, et al.
Published: (2024)

White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is?
by: Yu, Yaodong, et al.
Published: (2023)

Sparse-to-Sparse Training of Diffusion Models
by: Oliveira, Inês Cardoso, et al.
Published: (2025)

Localize-and-Stitch: Efficient Model Merging via Sparse Task Arithmetic
by: He, Yifei, et al.
Published: (2024)

Cyclic Sparse Training: Is it Enough?
by: Gadhikar, Advait, et al.
Published: (2024)

Towards Sparse Video Understanding and Reasoning
by: Xu, Chenwei, et al.
Published: (2026)

Dynamic Sparse Training with Structured Sparsity
by: Lasby, Mike, et al.
Published: (2023)

No Training Wheels: Steering Vectors for Bias Correction at Inference Time
by: Gupta, Aviral, et al.
Published: (2025)

SparseSwin: Swin Transformer with Sparse Transformer Block
by: Pinasthika, Krisna, et al.
Published: (2023)

Steering LVLMs via Sparse Autoencoder for Hallucination Mitigation
by: Hua, Zhenglin, et al.
Published: (2025)

SD$^2$: Self-Distilled Sparse Drafters
by: Lasby, Mike, et al.
Published: (2025)

Efficient Model Editing with Task-Localized Sparse Fine-tuning
by: Iurada, Leonardo, et al.
Published: (2025)

MoLEx: Mixture of Layer Experts for Finetuning with Sparse Upcycling
by: Teo, Rachel S. Y., et al.
Published: (2025)

MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts
by: Teo, Rachel S. Y., et al.
Published: (2024)

Problems with Chinchilla Approach 2: Systematic Biases in IsoFLOP Parabola Fits
by: Czech, Eric, et al.
Published: (2026)

Sign-In to the Lottery: Reparameterizing Sparse Training From Scratch
by: Gadhikar, Advait, et al.
Published: (2025)

Sparse Autoencoders as Plug-and-Play Firewalls for Adversarial Attack Detection in VLMs
by: Wang, Hao, et al.
Published: (2026)

Unsupervised Representation Learning from Sparse Transformation Analysis
by: Song, Yue, et al.
Published: (2024)

TWEO: Transformers Without Extreme Outliers Enables FP8 Training And Quantization For Dummies
by: Liang, Guang, et al.
Published: (2025)

LVSA: Training-Free Sparse Attention for Long Video Diffusion
by: Glorian, Gael, et al.
Published: (2026)

Beyond Adapter Retrieval: Latent Geometry-Preserving Composition via Sparse Task Projection
by: Jin, Pengfei, et al.
Published: (2024)

SSH: Sparse Spectrum Adaptation via Discrete Hartley Transformation
by: Shen, Yixian, et al.
Published: (2025)

From Sparse Decisions to Dense Reasoning: A Multi-attribute Trajectory Paradigm for Multimodal Moderation
by: Gu, Tianle, et al.
Published: (2026)

TinyTrain: Resource-Aware Task-Adaptive Sparse Training of DNNs at the Data-Scarce Edge
by: Kwon, Young D., et al.
Published: (2023)

Embracing Unknown Step by Step: Towards Reliable Sparse Training in Real World
by: Lei, Bowen, et al.
Published: (2024)

Robust Experts: the Effect of Adversarial Training on CNNs with Sparse Mixture-of-Experts Layers
by: Pavlitska, Svetlana, et al.
Published: (2025)

Sparsely Activated Networks
by: Bizopoulos, Paschalis, et al.
Published: (2019)

DiffMM: Efficient Method for Accurate Noisy and Sparse Trajectory Map Matching via One Step Diffusion
by: Han, Chenxu, et al.
Published: (2026)

Lung-DETR: Deformable Detection Transformer for Sparse Lung Nodule Anomaly Detection
by: Ramezani, Hooman, et al.
Published: (2024)

Mixture of Distributions Matters: Dynamic Sparse Attention for Efficient Video Diffusion Transformers
by: Liu, Yuxi, et al.
Published: (2026)

Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity
by: Xi, Haocheng, et al.
Published: (2025)

Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformers
by: Chen, Pengtao, et al.
Published: (2025)

Towards Better Alignment: Training Diffusion Models with Reinforcement Learning Against Sparse Rewards
by: Hu, Zijing, et al.
Published: (2025)

Transformer-VQ: Linear-Time Transformers via Vector Quantization
by: Lingle, Lucas D.
Published: (2023)

DiagrammerGPT: Generating Open-Domain, Open-Platform Diagrams via LLM Planning
by: Zala, Abhay, et al.
Published: (2023)

VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planning
by: Lin, Han, et al.
Published: (2023)

QeRL: Beyond Efficiency -- Quantization-enhanced Reinforcement Learning for LLMs
by: Huang, Wei, et al.
Published: (2025)