KARMA: Knowledge-Action Regularized Multimodal Alignment for Personalized Search at Taobao

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Sun, Zhi, Zhang, Wenming, Wei, Yi, Yu, Liren, Zhang, Zhixuan, Ou, Dan, Tang, Haihong
Format:	Preprint
Published:	2026
Subjects:	Information Retrieval Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866914432423034880
author	Sun, Zhi Zhang, Wenming Wei, Yi Yu, Liren Zhang, Zhixuan Ou, Dan Tang, Haihong
author_facet	Sun, Zhi Zhang, Wenming Wei, Yi Yu, Liren Zhang, Zhixuan Ou, Dan Tang, Haihong
contents	Large Language Models (LLMs) are equipped with profound semantic knowledge, making them a natural choice for injecting semantic generalization into personalized search systems. However, in practice we find that directly fine-tuning LLMs on industrial personalized tasks (e.g. next item prediction) often yields suboptimal results. We attribute this bottleneck to a critical Knowledge--Action Gap: the inherent conflict between preserving pre-trained semantic knowledge and aligning with specific personalized actions by discriminative objectives. Empirically, action-only training objectives induce Semantic Collapse, such as attention "sinks". This degradation severely cripples the LLM's generalization, failing to bring improvements to personalized search systems. We propose KARMA (Knowledge--Action Regularized Multimodal Alignment), a unified framework that treats semantic reconstruction as a train-only regularizer. KARMA optimizes a next-interest embedding for retrieval (Action) while enforcing semantic decodability (Knowledge) through two complementary objectives: (i) history-conditioned semantic generation, which anchors optimization to the LLM's native next-token distribution, and (ii) embedding-conditioned semantic reconstruction, which constrains the interest embedding to remain semantically recoverable. On Taobao search system, KARMA mitigates semantic collapse (attention-sink analysis) and improves both action metrics and semantic fidelity. In ablations, semantic decodability yields up to +22.5 HR@200. With KARMA, we achieve +0.25 CTR AUC in ranking, +1.86 HR in pre-ranking and +2.51 HR in recalling. Deployed online with low inference overhead at ranking & pre-ranking stage, KARMA drives +0.9% increase in GMV.
format	Preprint
id	arxiv_https___arxiv_org_abs_2603_22779
institution	arXiv
publishDate	2026
record_format	arxiv
spellingShingle	KARMA: Knowledge-Action Regularized Multimodal Alignment for Personalized Search at Taobao Sun, Zhi Zhang, Wenming Wei, Yi Yu, Liren Zhang, Zhixuan Ou, Dan Tang, Haihong Information Retrieval Artificial Intelligence Machine Learning Large Language Models (LLMs) are equipped with profound semantic knowledge, making them a natural choice for injecting semantic generalization into personalized search systems. However, in practice we find that directly fine-tuning LLMs on industrial personalized tasks (e.g. next item prediction) often yields suboptimal results. We attribute this bottleneck to a critical Knowledge--Action Gap: the inherent conflict between preserving pre-trained semantic knowledge and aligning with specific personalized actions by discriminative objectives. Empirically, action-only training objectives induce Semantic Collapse, such as attention "sinks". This degradation severely cripples the LLM's generalization, failing to bring improvements to personalized search systems. We propose KARMA (Knowledge--Action Regularized Multimodal Alignment), a unified framework that treats semantic reconstruction as a train-only regularizer. KARMA optimizes a next-interest embedding for retrieval (Action) while enforcing semantic decodability (Knowledge) through two complementary objectives: (i) history-conditioned semantic generation, which anchors optimization to the LLM's native next-token distribution, and (ii) embedding-conditioned semantic reconstruction, which constrains the interest embedding to remain semantically recoverable. On Taobao search system, KARMA mitigates semantic collapse (attention-sink analysis) and improves both action metrics and semantic fidelity. In ablations, semantic decodability yields up to +22.5 HR@200. With KARMA, we achieve +0.25 CTR AUC in ranking, +1.86 HR in pre-ranking and +2.51 HR in recalling. Deployed online with low inference overhead at ranking & pre-ranking stage, KARMA drives +0.9% increase in GMV.
title	KARMA: Knowledge-Action Regularized Multimodal Alignment for Personalized Search at Taobao
topic	Information Retrieval Artificial Intelligence Machine Learning
url	https://arxiv.org/abs/2603.22779