Cover Image

StrAE: Autoencoding for Pre-Trained Embeddings using Explicit Structure

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Opper, Mattia, Prokhorov, Victor, Siddharth, N.
Format:	Preprint
Published:	2023
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Self-StrAE at SemEval-2024 Task 1: Making Self-Structuring AutoEncoders Learn More With Less
by: Opper, Mattia, et al.
Published: (2024)

Banyan: Improved Representation Learning with Explicit Structure
by: Opper, Mattia, et al.
Published: (2024)

TRA: Better Length Generalisation with Threshold Relative Attention
by: Opper, Mattia, et al.
Published: (2025)

Autoencoding Conditional Neural Processes for Representation Learning
by: Prokhorov, Victor, et al.
Published: (2023)

Bootstrapping Embeddings for Low Resource Languages
by: Basoz, Merve, et al.
Published: (2026)

Embedding Geometries of Contrastive Language-Image Pre-Training
by: Chou, Jason Chuan-Chih, et al.
Published: (2024)

Training Superior Sparse Autoencoders for Instruct Models
by: Li, Jiaming, et al.
Published: (2025)

AE-LLM: Adaptive Efficiency Optimization for Large Language Models
by: Tanaka, Kaito, et al.
Published: (2026)

DEPT: Decoupled Embeddings for Pre-training Language Models
by: Iacob, Alex, et al.
Published: (2024)

Memory-efficient Energy-adaptive Inference of Pre-Trained Models on Batteryless Embedded Systems
by: Farina, Pietro, et al.
Published: (2024)

CURE: Controlled Unlearning for Robust Embeddings -- Mitigating Conceptual Shortcuts in Pre-Trained Language Models
by: Kocak, Aysenur, et al.
Published: (2025)

Explicitly Encoding Structural Symmetry is Key to Length Generalization in Arithmetic Tasks
by: Sabbaghi, Mahdi, et al.
Published: (2024)

ZClip: Adaptive Spike Mitigation for LLM Pre-Training
by: Kumar, Abhay, et al.
Published: (2025)

Pre-Trained Policy Discriminators are General Reward Models
by: Dou, Shihan, et al.
Published: (2025)

An Aspect Extraction Framework using Different Embedding Types, Learning Models, and Dependency Structure
by: Erkan, Ali, et al.
Published: (2025)

Efficient Knowledge Probing of Large Language Models by Adapting Pre-trained Embeddings
by: Sharma, Kartik, et al.
Published: (2025)

Language Model Training Paradigms for Clinical Feature Embeddings
by: Hu, Yurong, et al.
Published: (2023)

Attention with Trained Embeddings Provably Selects Important Tokens
by: Wu, Diyuan, et al.
Published: (2025)

MultiGPrompt for Multi-Task Pre-Training and Prompting on Graphs
by: Yu, Xingtong, et al.
Published: (2023)

FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale
by: Patel, Ajay, et al.
Published: (2026)

Domain-Adaptive Continued Pre-Training of Small Language Models
by: Faroz, Salman
Published: (2025)

Development of Pre-Trained Transformer-based Models for the Nepali Language
by: Thapa, Prajwal, et al.
Published: (2024)

Heterogeneous Self-Supervised Acoustic Pre-Training with Local Constraints
by: Cui, Xiaodong, et al.
Published: (2025)

MedFLIP: Medical Vision-and-Language Self-supervised Fast Pre-Training with Masked Autoencoder
by: Li, Lei, et al.
Published: (2024)

Optimizing Pre-Training Data Mixtures with Mixtures of Data Expert Models
by: Belenki, Lior, et al.
Published: (2025)

ProteinAE: Protein Diffusion Autoencoders for Structure Encoding
by: Li, Shaoning, et al.
Published: (2025)

Incorporating Hierarchical Semantics in Sparse Autoencoder Architectures
by: Muchane, Mark, et al.
Published: (2025)

Reinforcement Learning on Pre-Training Data
by: Li, Siheng, et al.
Published: (2025)

GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuning
by: Solatorio, Aivin V.
Published: (2024)

SynAE: A Framework for Measuring the Quality of Synthetic Data for Tool-Calling Agent Evaluations
by: Wang, Shuaiqi, et al.
Published: (2026)

PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runs
by: van der Wal, Oskar, et al.
Published: (2025)

Empirical Analysis of Efficient Fine-Tuning Methods for Large Pre-Trained Language Models
by: Doering, Nigel, et al.
Published: (2024)

$100K or 100 Days: Trade-offs when Pre-Training with Academic Resources
by: Khandelwal, Apoorv, et al.
Published: (2024)

Investigating the Pre-Training Dynamics of In-Context Learning: Task Recognition vs. Task Learning
by: Wang, Xiaolei, et al.
Published: (2024)

Zero2Text: Zero-Training Cross-Domain Inversion Attacks on Textual Embeddings
by: Kim, Doohyun, et al.
Published: (2026)

Improving Sentence Embeddings with Automatic Generation of Training Data Using Few-shot Examples
by: Sato, Soma, et al.
Published: (2024)

MapFormer: Self-Supervised Learning of Cognitive Maps with Input-Dependent Positional Embeddings
by: Rambaud, Victor, et al.
Published: (2025)

Safe Reinforcement Learning with Free-form Natural Language Constraints and Pre-Trained Language Models
by: Lou, Xingzhou, et al.
Published: (2024)

Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Models
by: Zhang, Jingyang, et al.
Published: (2024)

SVFit: Parameter-Efficient Fine-Tuning of Large Pre-Trained Models Using Singular Values
by: Sun, Chengwei, et al.
Published: (2024)