Selected Publications
2024
- The Fall of ROME: Understanding the Collapse of LLMs in Model EditingIn Findings of the Association for Computational Linguistics: EMNLP 2024, Nov 2024
Despite significant progress in model editing methods, their application in real-world scenarios remains challenging as they often cause large language models (LLMs) to collapse. Among them, ROME is particularly concerning, as it could disrupt LLMs with only a single edit. In this paper, we study the root causes of such collapse. Through extensive analysis, we identify two primary factors that contribute to the collapse: i) inconsistent handling of prefixed and unprefixed keys in the parameter update equation may result in very small denominators, causing excessively large parameter updates; ii) the subject of collapse cases is usually the first token, whose unprefixed key distribution significantly differs from the prefixed key distribution in autoregressive transformers, causing the aforementioned issue to materialize. To validate our findings, we propose a simple yet effective approach: uniformly using prefixed keys during editing phase and adding prefixes during testing phase to ensure the consistency between training and testing. The experimental results show that the proposed solution can prevent model collapse while maintaining the effectiveness of the edits.
@inproceedings{yang-etal-2024-fall, title = {The Fall of {ROME}: Understanding the Collapse of {LLM}s in Model Editing}, author = {Yang, Wanli and Sun, Fei and Tan, Jiajun and Ma, Xinyu and Su, Du and Yin, Dawei and Shen, Huawei}, editor = {Al-Onaizan, Yaser and Bansal, Mohit and Chen, Yun-Nung}, booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2024}, month = nov, year = {2024}, address = {Miami, Florida, USA}, publisher = {Association for Computational Linguistics}, url = {https://aclanthology.org/2024.findings-emnlp.236/}, google_scholar = {ziOE8S1-AIUC}, doi = {10.18653/v1/2024.findings-emnlp.236}, pages = {4079--4087} } - Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts?In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Aug 2024
While auxiliary information has become a key to enhancing Large Language Models (LLMs), relatively little is known about how LLMs merge these contexts, specifically contexts generated by LLMs and those retrieved from external sources.To investigate this, we formulate a systematic framework to identify whether LLMs’ responses are attributed to either generated or retrieved contexts.To easily trace the origin of the response, we construct datasets with conflicting contexts, i.e., each question is paired with both generated and retrieved contexts, yet only one of them contains the correct answer.Our experiments reveal a significant bias in several LLMs (GPT-4/3.5 and Llama2) to favor generated contexts, even when they provide incorrect information.We further identify two key factors contributing to this bias: i) contexts generated by LLMs typically show greater similarity to the questions, increasing their likelihood of being selected; ii) the segmentation process used in retrieved contexts disrupts their completeness, thereby hindering their full utilization in LLMs.Our analysis enhances the understanding of how LLMs merge diverse contexts, offers valuable insights for advancing current LLM augmentation methods, and highlights the risk of generated misinformation for retrieval-augmented LLMs.
@inproceedings{tan-etal-2024-blinded, google_scholar = {raTqNPD5sRQC}, title = {Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts?}, author = {Tan, Hexiang and Sun, Fei and Yang, Wanli and Wang, Yuanzhuo and Cao, Qi and Cheng, Xueqi}, editor = {Ku, Lun-Wei and Martins, Andre and Srikumar, Vivek}, booktitle = {Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)}, month = aug, year = {2024}, address = {Bangkok, Thailand}, publisher = {Association for Computational Linguistics}, url = {https://aclanthology.org/2024.acl-long.337/}, doi = {10.18653/v1/2024.acl-long.337}, pages = {6207--6227} } - The Butterfly Effect of Model Editing: Few Edits Can Trigger Large Language Models CollapseIn Findings of the Association for Computational Linguistics: ACL 2024, Aug 2024
Although model editing has shown promise in revising knowledge in Large Language Models (LLMs), its impact on the inherent capabilities of LLMs is often overlooked. In this work, we reveal a critical phenomenon: even a single edit can trigger model collapse, manifesting as significant performance degradation in various benchmark tasks. However, benchmarking LLMs after each edit, while necessary to prevent such collapses, is impractically time-consuming and resource-intensive. To mitigate this, we propose using perplexity as a surrogate metric, validated by extensive experiments demonstrating changes in an edited model‘s perplexity are strongly correlated with its downstream task performances. We further conduct an in-depth study on sequential editing, a practical setting for real-world scenarios, across various editing methods and LLMs, focusing on hard cases from our previous single edit studies. The results indicate that nearly all examined editing methods result in model collapse after only few edits. To facilitate further research, we have utilized GPT-3.5 to develop a new dataset, HardEdit, based on those hard cases. This dataset aims to establish the foundation for pioneering research in reliable model editing and the mechanisms underlying editing-induced model collapse. We hope this work can draw the community‘s attention to the potential risks inherent in model editing practices.
@inproceedings{yang-etal-2024-butterfly, google_scholar = {kWvqk_afx_IC}, title = {The Butterfly Effect of Model Editing: Few Edits Can Trigger Large Language Models Collapse}, author = {Yang, Wanli and Sun, Fei and Ma, Xinyu and Liu, Xun and Yin, Dawei and Cheng, Xueqi}, editor = {Ku, Lun-Wei and Martins, Andre and Srikumar, Vivek}, booktitle = {Findings of the Association for Computational Linguistics: ACL 2024}, month = aug, year = {2024}, address = {Bangkok, Thailand}, publisher = {Association for Computational Linguistics}, url = {https://aclanthology.org/2024.findings-acl.322/}, doi = {10.18653/v1/2024.findings-acl.322}, pages = {5419--5437} } - When to Trust LLMs: Aligning Confidence with Response QualityShuchang Tao , Liuyi Yao , Hanxing Ding , Yuexiang Xie , Qi Cao , Fei Sun , Jinyang Gao , Huawei Shen , and Bolin DingIn Findings of the Association for Computational Linguistics: ACL 2024, Aug 2024
Despite the success of large language models (LLMs) in natural language generation, much evidence shows that LLMs may produce incorrect or nonsensical text. This limitation highlights the importance of discerning when to trust LLMs, especially in safety-critical domains. Existing methods often express reliability by confidence level, however, their effectiveness is limited by the lack of objective guidance. To address this, we propose CONfidence-Quality-ORDer-preserving alignment approach (CONQORD), which leverages reinforcement learning guided by a tailored dual-component reward function. This function integrates quality reward and order-preserving alignment reward functions. Specifically, the order-preserving reward incentivizes the model to verbalize greater confidence for responses of higher quality to align the order of confidence and quality. Experiments demonstrate that CONQORD significantly improves the alignment performance between confidence and response accuracy, without causing over-cautious. Furthermore, the aligned confidence provided by CONQORD informs when to trust LLMs, and acts as a determinant for initiating the retrieval process of external knowledge. Aligning confidence with response quality ensures more transparent and reliable responses, providing better trustworthiness.
@inproceedings{tao-etal-2024-trust, google_scholar = {w1MjKQ0l0TYC}, title = {When to Trust {LLM}s: Aligning Confidence with Response Quality}, author = {Tao, Shuchang and Yao, Liuyi and Ding, Hanxing and Xie, Yuexiang and Cao, Qi and Sun, Fei and Gao, Jinyang and Shen, Huawei and Ding, Bolin}, editor = {Ku, Lun-Wei and Martins, Andre and Srikumar, Vivek}, booktitle = {Findings of the Association for Computational Linguistics: ACL 2024}, month = aug, year = {2024}, address = {Bangkok, Thailand}, publisher = {Association for Computational Linguistics}, url = {https://aclanthology.org/2024.findings-acl.357/}, doi = {10.18653/v1/2024.findings-acl.357}, pages = {5984--5996} } - Unlink to Unlearn: Simplifying Edge Unlearning in GNNsJiajun Tan , Fei Sun , Ruichen Qiu , Du Su , and Huawei ShenIn Companion Proceedings of the ACM Web Conference 2024, Singapore, Singapore, Aug 2024
As concerns over data privacy intensify, unlearning in Graph Neural Networks (GNNs) has emerged as a prominent research frontier in academia. This concept is pivotal in enforcing the right to be forgotten, which entails the selective removal of specific data from trained GNNs upon user request. Our research focuses on edge unlearning, a process of particular relevance to real-world applications. Current state-of-the-art approaches like GNNDelete can eliminate the influence of specific edges yet suffer from over-forgetting, which means the unlearning process inadvertently removes excessive information beyond needed, leading to a significant performance decline for remaining edges. Our analysis identifies the loss functions of GNNDelete as the primary source of over-forgetting and also suggests that loss functions may be redundant for effective edge unlearning. Building on these insights, we simplify GNNDelete to develop Unlink to Unlearn (UtU), a novel method that facilitates unlearning exclusively through unlinking the forget edges from graph structure. Our extensive experiments demonstrate that UtU delivers privacy protection on par with that of a retrained model while preserving high accuracy in downstream tasks, by upholding over 97.3% of the retrained model’s privacy protection capabilities and 99.8% of its link prediction accuracy. Meanwhile, UtU requires only constant computational demands, underscoring its advantage as a highly lightweight and practical edge unlearning solution.
@inproceedings{tan-etal-2024-Unlink, google_scholar = {1DsIQWDZLl8C}, author = {Tan, Jiajun and Sun, Fei and Qiu, Ruichen and Su, Du and Shen, Huawei}, title = {Unlink to Unlearn: Simplifying Edge Unlearning in GNNs}, year = {2024}, isbn = {9798400701726}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3589335.3651578}, doi = {10.1145/3589335.3651578}, booktitle = {Companion Proceedings of the ACM Web Conference 2024}, pages = {489–492}, numpages = {4}, keywords = {graph neural networks, machine unlearning, over-forgetting}, location = {Singapore, Singapore}, series = {WWW '24} }WebConf (Short) - Accelerating the Surrogate Retraining for Poisoning Attacks against Recommender SystemsYunfan Wu , Qi Cao , Shuchang Tao , Kaike Zhang , Fei Sun , and Huawei ShenIn Proceedings of the 18th ACM Conference on Recommender Systems, Bari, Italy, Aug 2024
Recent studies have demonstrated the vulnerability of recommender systems to data poisoning attacks, where adversaries inject carefully crafted fake user interactions into the training data of recommenders to promote target items. Current attack methods involve iteratively retraining a surrogate recommender on the poisoned data with the latest fake users to optimize the attack. However, this repetitive retraining is highly time-consuming, hindering the efficient assessment and optimization of fake users. To mitigate this computational bottleneck and develop a more effective attack in an affordable time, we analyze the retraining process and find that a change in the representation of one user/item will cause a cascading effect through the user-item interaction graph. Under theoretical guidance, we introduce Gradient Passing (GP), a novel technique that explicitly passes gradients between interacted user-item pairs during backpropagation, thereby approximating the cascading effect and accelerating retraining. With just a single update, GP can achieve effects comparable to multiple original training iterations. Under the same number of retraining epochs, GP enables a closer approximation of the surrogate recommender to the victim. This more accurate approximation provides better guidance for optimizing fake users, ultimately leading to enhanced data poisoning attacks. Extensive experiments on real-world datasets demonstrate the efficiency and effectiveness of our proposed GP.
@inproceedings{wu-2024-attack, google_scholar = {An6A6Jpfc1oC}, author = {Wu, Yunfan and Cao, Qi and Tao, Shuchang and Zhang, Kaike and Sun, Fei and Shen, Huawei}, title = {Accelerating the Surrogate Retraining for Poisoning Attacks against Recommender Systems}, year = {2024}, isbn = {9798400705052}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3640457.3688148}, doi = {10.1145/3640457.3688148}, booktitle = {Proceedings of the 18th ACM Conference on Recommender Systems}, pages = {701–711}, numpages = {11}, keywords = {Adversarial Learning, Poisoning Attacks, Recommender Systems}, location = {Bari, Italy}, series = {RecSys '24} } - Improving the Shortest Plank: Vulnerability-Aware Adversarial Training for Robust Recommender SystemKaike Zhang , Qi Cao , Yunfan Wu , Fei Sun , Huawei Shen , and Xueqi ChengIn Proceedings of the 18th ACM Conference on Recommender Systems, Bari, Italy, Aug 2024
Recommender systems play a pivotal role in mitigating information overload in various fields. Nonetheless, the inherent openness of these systems introduces vulnerabilities, allowing attackers to insert fake users into the system’s training data to skew the exposure of certain items, known as poisoning attacks. Adversarial training has emerged as a notable defense mechanism against such poisoning attacks within recommender systems. Existing adversarial training methods apply perturbations of the same magnitude across all users to enhance system robustness against attacks. Yet, in reality, we find that attacks often affect only a subset of users who are vulnerable. These perturbations of indiscriminate magnitude make it difficult to balance effective protection for vulnerable users without degrading recommendation quality for those who are not affected. To address this issue, our research delves into understanding user vulnerability. Considering that poisoning attacks pollute the training data, we note that the higher degree to which a recommender system fits users’ training data correlates with an increased likelihood of users incorporating attack information, indicating their vulnerability. Leveraging these insights, we introduce the Vulnerability-aware Adversarial Training (VAT), designed to defend against poisoning attacks in recommender systems. VAT employs a novel vulnerability-aware function to estimate users’ vulnerability based on the degree to which the system fits them. Guided by this estimation, VAT applies perturbations of adaptive magnitude to each user, not only reducing the success ratio of attacks but also preserving, and potentially enhancing, the quality of recommendations. Comprehensive experiments confirm VAT’s superior defensive capabilities across different recommendation models and against various types of attacks.
@inproceedings{zhang-2024-adv, google_scholar = {2l5NCbZemmgC}, author = {Zhang, Kaike and Cao, Qi and Wu, Yunfan and Sun, Fei and Shen, Huawei and Cheng, Xueqi}, title = {Improving the Shortest Plank: Vulnerability-Aware Adversarial Training for Robust Recommender System}, year = {2024}, isbn = {9798400705052}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3640457.3688120}, doi = {10.1145/3640457.3688120}, booktitle = {Proceedings of the 18th ACM Conference on Recommender Systems}, pages = {680–689}, numpages = {10}, keywords = {Adversarial Training, Poisoning Attack, Robust Recommender System}, location = {Bari, Italy}, series = {RecSys '24} } - LoRec: Combating Poisons with Large Language Model for Robust Sequential RecommendationKaike Zhang , Qi Cao , Yunfan Wu , Fei Sun , Huawei Shen , and Xueqi ChengIn Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, Washington DC, USA, Aug 2024
Sequential recommender systems stand out for their ability to capture users’ dynamic interests and the patterns of item transitions. However, the inherent openness of sequential recommender systems renders them vulnerable to poisoning attacks, where fraudsters are injected into the training data to manipulate learned patterns. Traditional defense methods predominantly depend on predefined assumptions or rules extracted from specific known attacks, limiting their generalizability to unknown attacks. To solve the above problems, considering the rich open-world knowledge encapsulated in Large Language Models (LLMs), we attempt to introduce LLMs into defense methods to broaden the knowledge beyond limited known attacks. We propose LoRec, an innovative framework that employs LLM-Enhanced Calibration to strengthen the robustness of sequential Recommender systems against poisoning attacks. LoRec integrates an LLM-enhanced CalibraTor (LCT) that refines the training process of sequential recommender systems with knowledge derived from LLMs, applying a user-wise reweighting to diminish the impact of attacks. Incorporating LLMs’ open-world knowledge, the LCT effectively converts the limited, specific priors or rules into a more general pattern of fraudsters, offering improved defenses against poisons. Our comprehensive experiments validate that LoRec, as a general framework, significantly strengthens the robustness of sequential recommender systems.
@inproceedings{zhang-2024-lorec, google_scholar = {QUX0mv85b1cC}, author = {Zhang, Kaike and Cao, Qi and Wu, Yunfan and Sun, Fei and Shen, Huawei and Cheng, Xueqi}, title = {LoRec: Combating Poisons with Large Language Model for Robust Sequential Recommendation}, year = {2024}, isbn = {9798400704314}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3626772.3657684}, doi = {10.1145/3626772.3657684}, booktitle = {Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval}, pages = {1733–1742}, numpages = {10}, keywords = {large language model, poisoning attack, robust sequential recommendation}, location = {Washington DC, USA}, series = {SIGIR '24} } - Understanding and Improving Adversarial Collaborative Filtering for Robust RecommendationKaike Zhang , Qi Cao , Yunfan Wu , Fei Sun , Huawei Shen , and Xueqi ChengIn The Thirty-eighth Annual Conference on Neural Information Processing Systems, Aug 2024
@inproceedings{zhang2024understanding, google_scholar = {DkZNVXde3BIC}, title = {Understanding and Improving Adversarial Collaborative Filtering for Robust Recommendation}, author = {Zhang, Kaike and Cao, Qi and Wu, Yunfan and Sun, Fei and Shen, Huawei and Cheng, Xueqi}, booktitle = {The Thirty-eighth Annual Conference on Neural Information Processing Systems}, year = {2024}, url = {https://openreview.net/forum?id=k8AYft5ED1} }
2023
- Studying the Impact of Data Disclosure Mechanism in Recommender Systems via SimulationZiqian Chen , Fei Sun , Yifan Tang , Haokun Chen , Jinyang Gao , and Bolin DingACM Trans. Inf. Syst., Feb 2023
Recently, privacy issues in web services that rely on users’ personal data have raised great attention. Despite that recent regulations force companies to offer choices for each user to opt-in or opt-out of data disclosure, real-world applications usually only provide an “all or nothing” binary option for users to either disclose all their data or preserve all data with the cost of no personalized service. In this article, we argue that such a binary mechanism is not optimal for both consumers and platforms. To study how different privacy mechanisms affect users’ decisions on information disclosure and how users’ decisions affect the platform’s revenue, we propose a privacy-aware recommendation framework that gives users fine control over their data. In this new framework, users can proactively control which data to disclose based on the tradeoff between anticipated privacy risks and potential utilities. Then we study the impact of different data disclosure mechanisms via simulation with reinforcement learning due to the high cost of real-world experiments. The results show that the platform mechanisms with finer split granularity and more unrestrained disclosure strategy can bring better results for both consumers and platforms than the “all or nothing” mechanism adopted by most real-world applications.
@article{Chen-2023-privacy, google_scholar = {7Hz3ACDFbsoC}, author = {Chen, Ziqian and Sun, Fei and Tang, Yifan and Chen, Haokun and Gao, Jinyang and Ding, Bolin}, title = {Studying the Impact of Data Disclosure Mechanism in Recommender Systems via Simulation}, year = {2023}, issue_date = {July 2023}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {41}, number = {3}, issn = {1046-8188}, url = {https://doi.org/10.1145/3569452}, doi = {10.1145/3569452}, journal = {ACM Trans. Inf. Syst.}, month = feb, articleno = {60}, numpages = {26}, keywords = {GDPR, privacy, Recommender system} }
2022
- Graph Neural Networks in Recommender Systems: A SurveyShiwen Wu , Fei Sun , Wentao Zhang , Xu Xie , and Bin CuiACM Comput. Surv., Dec 2022
With the explosive growth of online information, recommender systems play a key role to alleviate such information overload. Due to the important application value of recommender systems, there have always been emerging works in this field. In recommender systems, the main challenge is to learn the effective user/item representations from their interactions and side information (if any). Recently, graph neural network (GNN) techniques have been widely utilized in recommender systems since most of the information in recommender systems essentially has graph structure and GNN has superiority in graph representation learning. This article aims to provide a comprehensive review of recent research efforts on GNN-based recommender systems. Specifically, we provide a taxonomy of GNN-based recommendation models according to the types of information used and recommendation tasks. Moreover, we systematically analyze the challenges of applying GNN on different types of data and discuss how existing works in this field address these challenges. Furthermore, we state new perspectives pertaining to the development of this field. We collect the representative papers along with their open-source implementations in .
@article{Wu-2022-GNN, google_scholar = {NyGDZy8z5eUC}, author = {Wu, Shiwen and Sun, Fei and Zhang, Wentao and Xie, Xu and Cui, Bin}, title = {Graph Neural Networks in Recommender Systems: A Survey}, year = {2022}, issue_date = {May 2023}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {55}, number = {5}, issn = {0360-0300}, url = {https://doi.org/10.1145/3535101}, doi = {10.1145/3535101}, journal = {ACM Comput. Surv.}, month = dec, articleno = {97}, numpages = {37}, keywords = {survey, graph neural network, Recommender system} }CSURTop 10 cited CSUR (5Y) - Contrastive Learning for Sequential RecommendationXu Xie , Fei Sun , Zhaoyang Liu , Shiwen Wu , Jinyang Gao , Jiandong Zhang , Bolin Ding , and Bin CuiIn 2022 IEEE 38th International Conference on Data Engineering (ICDE), Dec 2022
@inproceedings{xie-2022-Contrastive, google_scholar = {Mojj43d5GZwC}, author = {Xie, Xu and Sun, Fei and Liu, Zhaoyang and Wu, Shiwen and Gao, Jinyang and Zhang, Jiandong and Ding, Bolin and Cui, Bin}, booktitle = {2022 IEEE 38th International Conference on Data Engineering (ICDE)}, title = {Contrastive Learning for Sequential Recommendation}, year = {2022}, volume = {}, number = {}, pages = {1259-1273}, keywords = {Computer vision;Conferences;Multitasking;Data engineering;Data models;Behavioral sciences;Data mining;Contrastive Learning;Deep Learning;Recom-mender Systems}, doi = {10.1109/ICDE53745.2022.00099} }ICDETop 5 cited ICDE (5Y) - Recommendation UnlearningChong Chen , Fei Sun , Min Zhang , and Bolin DingIn Proceedings of the ACM Web Conference 2022, Virtual Event, Lyon, France, Dec 2022
Recommender systems provide essential web services by learning users’ personal preferences from collected data. However, in many cases, systems also need to forget some training data. From the perspective of privacy, users desire a tool to erase the impacts of their sensitive data from the trained models. From the perspective of utility, if a system’s utility is damaged by some bad data, the system needs to forget such data to regain utility. While unlearning is very important, it has not been well-considered in existing recommender systems. Although there are some researches have studied the problem of machine unlearning, existing methods can not be directly applied to recommendation as they are unable to consider the collaborative information. In this paper, we propose RecEraser, a general and efficient machine unlearning framework tailored to recommendation tasks. The main idea of RecEraser is to divide the training set into multiple shards and train submodels with these shards. Specifically, to keep the collaborative information of the data, we first design three novel data partition algorithms to divide training data into balanced groups. We then further propose an adaptive aggregation method to improve the global model utility. Experimental results on three public benchmarks show that RecEraser can not only achieve efficient unlearning but also outperform the state-of-the-art unlearning methods in terms of model utility. The source code can be found at https://github.com/chenchongthu/Recommendation-Unlearning
@inproceedings{Chen-2022-unlearn, google_scholar = {OP4eGU-M3BUC}, author = {Chen, Chong and Sun, Fei and Zhang, Min and Ding, Bolin}, title = {Recommendation Unlearning}, year = {2022}, isbn = {9781450390965}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3485447.3511997}, doi = {10.1145/3485447.3511997}, booktitle = {Proceedings of the ACM Web Conference 2022}, pages = {2768–2777}, numpages = {10}, keywords = {Collaborative Filtering;, Machine Unlearning, Recommender Systems, Selective Deletion}, location = {Virtual Event, Lyon, France}, series = {WWW '22} }WebConf
2021
- Factual Consistency Evaluation for Text Summarization via Counterfactual EstimationYuexiang Xie , Fei Sun , Yang Deng , Yaliang Li , and Bolin DingIn Findings of the Association for Computational Linguistics: EMNLP 2021, Nov 2021
Despite significant progress has been achieved in text summarization, factual inconsistency in generated summaries still severely limits its practical applications. Among the key factors to ensure factual consistency, a reliable automatic evaluation metric is the first and the most crucial one. However, existing metrics either neglect the intrinsic cause of the factual inconsistency or rely on auxiliary tasks, leading to an unsatisfied correlation with human judgments or increasing the inconvenience of usage in practice. In light of these challenges, we propose a novel metric to evaluate the factual consistency in text summarization via counterfactual estimation, which formulates the causal relationship among the source document, the generated summary, and the language prior. We remove the effect of language prior, which can cause factual inconsistency, from the total causal effect on the generated summary, and provides a simple yet effective way to evaluate consistency without relying on other auxiliary tasks. We conduct a series of experiments on three public abstractive text summarization datasets, and demonstrate the advantages of the proposed metric in both improving the correlation with human judgments and the convenience of usage. The source code is available at \urlhttps://github.com/xieyxclack/factual_coco.
@inproceedings{xie-etal-2021-factual-consistency, google_scholar = {4X0JR2_MtJMC}, title = {Factual Consistency Evaluation for Text Summarization via Counterfactual Estimation}, author = {Xie, Yuexiang and Sun, Fei and Deng, Yang and Li, Yaliang and Ding, Bolin}, editor = {Moens, Marie-Francine and Huang, Xuanjing and Specia, Lucia and Yih, Scott Wen-tau}, booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2021}, month = nov, year = {2021}, address = {Punta Cana, Dominican Republic}, publisher = {Association for Computational Linguistics}, url = {https://aclanthology.org/2021.findings-emnlp.10/}, doi = {10.18653/v1/2021.findings-emnlp.10}, pages = {100--110} } - Variation Control and Evaluation for Generative Slate RecommendationsShuchang Liu , Fei Sun , Yingqiang Ge , Changhua Pei , and Yongfeng ZhangIn Proceedings of the Web Conference 2021, Ljubljana, Slovenia, Nov 2021
Slate recommendation generates a list of items as a whole instead of ranking each item individually, so as to better model the intra-list positional biases and item relations. In order to deal with the enormous combinatorial space of slates, recent work considers a generative solution so that a slate distribution can be directly modeled. However, we observe that such approaches—despite their proved effectiveness in computer vision—suffer from a trade-off dilemma in recommender systems: when focusing on reconstruction, they easily over-fit the data and hardly generate satisfactory recommendations; on the other hand, when focusing on satisfying the user interests, they get trapped in a few items and fail to cover the item variation in slates. In this paper, we propose to enhance the accuracy-based evaluation with slate variation metrics to estimate the stochastic behavior of generative models. We illustrate that instead of reaching to one of the two undesirable extreme cases in the dilemma, a valid generative solution resides in a narrow “elbow” region in between. And we show that item perturbation can enforce slate variation and mitigate the over-concentration of generated slates, which expand the “elbow” performance to an easy-to-find region. We further propose to separate a pivot selection phase from the generation process so that the model can apply perturbation before generation. Empirical results show that this simple modification can provide even better variance with the same level of accuracy compared to post-generation perturbation methods.
@inproceedings{Liu-2021-Slate, google_scholar = {Y5dfb0dijaUC}, author = {Liu, Shuchang and Sun, Fei and Ge, Yingqiang and Pei, Changhua and Zhang, Yongfeng}, title = {Variation Control and Evaluation for Generative Slate Recommendations}, year = {2021}, isbn = {9781450383127}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3442381.3449864}, doi = {10.1145/3442381.3449864}, booktitle = {Proceedings of the Web Conference 2021}, pages = {436–448}, numpages = {13}, keywords = {Conditional Variational Auto-Encoder, Generative Recommendation, Slate Recommendation}, location = {Ljubljana, Slovenia}, series = {WWW '21} }WebConf - Unified Conversational Recommendation Policy Learning via Graph-based Reinforcement LearningYang Deng , Yaliang Li , Fei Sun , Bolin Ding , and Wai LamIn Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, Nov 2021
Conversational recommender systems (CRS) enable the traditional recommender systems to explicitly acquire user preferences towards items and attributes through interactive conversations. Reinforcement learning (RL) is widely adopted to learn conversational recommendation policies to decide what attributes to ask, which items to recommend, and when to ask or recommend, at each conversation turn. However, existing methods mainly target at solving one or two of these three decision-making problems in CRS with separated conversation and recommendation components, which restrict the scalability and generality of CRS and fall short of preserving a stable training procedure. In the light of these challenges, we propose to formulate these three decision-making problems in CRS as a unified policy learning task. In order to systematically integrate conversation and recommendation components, we develop a dynamic weighted graph based RL method to learn a policy to select the action at each conversation turn, either asking an attribute or recommending items. Further, to deal with the sample efficiency issue, we propose two action selection strategies for reducing the candidate action space according to the preference and entropy information. Experimental results on two benchmark CRS datasets and a real-world E-Commerce application show that the proposed method not only significantly outperforms state-of-the-art methods but also enhances the scalability and stability of CRS.
@inproceedings{deng-2021-conv, google_scholar = {BwyfMAYsbu0C}, author = {Deng, Yang and Li, Yaliang and Sun, Fei and Ding, Bolin and Lam, Wai}, title = {Unified Conversational Recommendation Policy Learning via Graph-based Reinforcement Learning}, year = {2021}, isbn = {9781450380379}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3404835.3462913}, doi = {10.1145/3404835.3462913}, booktitle = {Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval}, pages = {1431–1441}, numpages = {11}, keywords = {conversational recommendation, graph representation learning, reinforcement learning}, location = {Virtual Event, Canada}, series = {SIGIR '21} } - Towards Long-term Fairness in RecommendationYingqiang Ge , Shuchang Liu , Ruoyuan Gao , Yikun Xian , Yunqi Li , Xiangyu Zhao , Changhua Pei , Fei Sun , Junfeng Ge , Wenwu Ou , and Yongfeng ZhangIn Proceedings of the 14th ACM International Conference on Web Search and Data Mining, Virtual Event, Israel, Nov 2021
As Recommender Systems (RS) influence more and more people in their daily life, the issue of fairness in recommendation is becoming more and more important. Most of the prior approaches to fairness-aware recommendation have been situated in a static or one-shot setting, where the protected groups of items are fixed, and the model provides a one-time fairness solution based on fairness-constrained optimization. This fails to consider the dynamic nature of the recommender systems, where attributes such as item popularity may change over time due to the recommendation policy and user engagement. For example, products that were once popular may become no longer popular, and vice versa. As a result, the system that aims to maintain long-term fairness on the item exposure in different popularity groups must accommodate this change in a timely fashion.Novel to this work, we explore the problem of long-term fairness in recommendation and accomplish the problem through dynamic fairness learning. We focus on the fairness of exposure of items in different groups, while the division of the groups is based on item popularity, which dynamically changes over time in the recommendation process. We tackle this problem by proposing a fairness-constrained reinforcement learning algorithm for recommendation, which models the recommendation problem as a Constrained Markov Decision Process (CMDP), so that the model can dynamically adjust its recommendation policy to make sure the fairness requirement is always satisfied when the environment changes. Experiments on several real-world datasets verify our framework’s superiority in terms of recommendation performance, short-term fairness, and long-term fairness.
@inproceedings{ge-2021-fairness, google_scholar = {t6usbXjVLHcC}, author = {Ge, Yingqiang and Liu, Shuchang and Gao, Ruoyuan and Xian, Yikun and Li, Yunqi and Zhao, Xiangyu and Pei, Changhua and Sun, Fei and Ge, Junfeng and Ou, Wenwu and Zhang, Yongfeng}, title = {Towards Long-term Fairness in Recommendation}, year = {2021}, isbn = {9781450382977}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3437963.3441824}, doi = {10.1145/3437963.3441824}, booktitle = {Proceedings of the 14th ACM International Conference on Web Search and Data Mining}, pages = {445–453}, numpages = {9}, keywords = {unbiased recommendation, reinforcement learning, recommender system, long-term fairness, constrained policy optimization}, location = {Virtual Event, Israel}, series = {WSDM '21} }
2020
- Intent Preference Decoupling for User Representation on Online Recommender SystemZhaoyang Liu , Haokun Chen , Fei Sun , Xu Xie , Jinyang Gao , Bolin Ding , and Yanyan ShenIn Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, Jul 2020Main track
@inproceedings{liu-2020-Intent, google_scholar = {08ZZubdj9fEC}, title = {Intent Preference Decoupling for User Representation on Online Recommender System}, author = {Liu, Zhaoyang and Chen, Haokun and Sun, Fei and Xie, Xu and Gao, Jinyang and Ding, Bolin and Shen, Yanyan}, booktitle = {Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, {IJCAI-20}}, publisher = {International Joint Conferences on Artificial Intelligence Organization}, editor = {Bessiere, Christian}, pages = {2575--2582}, year = {2020}, month = jul, note = {Main track}, doi = {10.24963/ijcai.2020/357}, url = {https://doi.org/10.24963/ijcai.2020/357} } - Privileged Features Distillation at Taobao RecommendationsChen Xu , Quan Li , Junfeng Ge , Jinyang Gao , Xiaoyong Yang , Changhua Pei , Fei Sun , Jian Wu , Hanxiao Sun , and Wenwu OuIn Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Virtual Event, CA, USA, Jul 2020
Features play an important role in the prediction tasks of e-commerce recommendations. To guarantee the consistency of off-line training and on-line serving, we usually utilize the same features that are both available. However, the consistency in turn neglects some discriminative features. For example, when estimating the conversion rate (CVR), i.e., the probability that a user would purchase the item if she clicked it, features like dwell time on the item detailed page are informative. However, CVR prediction should be conducted for on-line ranking before the click happens. Thus we cannot get such post-event features during serving.We define the features that are discriminative but only available during training as the privileged features. Inspired by the distillation techniques which bridge the gap between training and inference, in this work, we propose privileged features distillation (PFD). We train two models, i.e., a student model that is the same as the original one and a teacher model that additionally utilizes the privileged features. Knowledge distilled from the more accurate teacher is transferred to the student, which helps to improve its prediction accuracy. During serving, only the student part is extracted and it relies on no privileged features. We conduct experiments on two fundamental prediction tasks at Taobao recommendations, i.e., click-through rate (CTR) at coarse-grained ranking and CVR at fine-grained ranking. By distilling the interacted features that are prohibited during serving for CTR and the post-event features for CVR, we achieve significant improvements over their strong baselines. During the on-line A/B tests, the click metric is improved by +5.0% in the CTR task. And the conversion metric is improved by +2.3% in the CVR task. Besides, by addressing several issues of training PFD, we obtain comparable training speed as the baselines without any distillation.
@inproceedings{Chen-2020-Distillation, google_scholar = {vRqMK49ujn8C}, author = {Xu, Chen and Li, Quan and Ge, Junfeng and Gao, Jinyang and Yang, Xiaoyong and Pei, Changhua and Sun, Fei and Wu, Jian and Sun, Hanxiao and Ou, Wenwu}, title = {Privileged Features Distillation at Taobao Recommendations}, year = {2020}, isbn = {9781450379984}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3394486.3403309}, doi = {10.1145/3394486.3403309}, booktitle = {Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery \& Data Mining}, pages = {2590–2598}, numpages = {9}, keywords = {ctr, cvr, distillation, e-commerce recommendations, privileged features}, location = {Virtual Event, CA, USA}, series = {KDD '20} } - Understanding Echo Chambers in E-commerce Recommender SystemsYingqiang Ge , Shuya Zhao , Honglu Zhou , Changhua Pei , Fei Sun , Wenwu Ou , and Yongfeng ZhangIn Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, China, Jul 2020
Personalized recommendation benefits users in accessing contents of interests effectively. Current research on recommender systems mostly focuses on matching users with proper items based on user interests. However, significant efforts are missing to understand how the recommendations influence user preferences and behaviors, e.g., if and how recommendations result in echo chambers. Extensive efforts have been made in examining the phenomenon in online media and social network systems. Meanwhile, there are growing concerns that recommender systems might lead to the self-reinforcing of user’s interests due to narrowed exposure of items, which may be the potential cause of echo chamber. In this paper, we aim to analyze the echo chamber phenomenon in Alibaba Taobao — one of the largest e-commerce platforms in the world.Echo chamber means the effect of user interests being reinforced through repeated exposure to similar contents. Based on the definition, we examine the presence of echo chamber in two steps. First, we explore whether user interests have been reinforced. Second, we check whether the reinforcement results from the exposure of similar contents. Our evaluations are enhanced with robust metrics, including cluster validity and statistical significance. Experiments are performed on extensive collections of real-world data consisting of user clicks, purchases, and browse logs from Alibaba Taobao. Evidence suggests the tendency of echo chamber in user click behaviors, while it is relatively mitigated in user purchase behaviors. Insights from the results guide the refinement of recommendation algorithms in real-world e-commerce systems.
@inproceedings{ge-2020-echo, google_scholar = {738O_yMBCRsC}, author = {Ge, Yingqiang and Zhao, Shuya and Zhou, Honglu and Pei, Changhua and Sun, Fei and Ou, Wenwu and Zhang, Yongfeng}, title = {Understanding Echo Chambers in E-commerce Recommender Systems}, year = {2020}, isbn = {9781450380164}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3397271.3401431}, doi = {10.1145/3397271.3401431}, booktitle = {Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval}, pages = {2261–2270}, numpages = {10}, keywords = {e-commerce, echo chamber, filter bubble, recommender systems}, location = {Virtual Event, China}, series = {SIGIR '20} } - Graph-based Regularization on Embedding Layers for RecommendationYuan Zhang , Fei Sun , Xiaoyong Yang , Chen Xu , Wenwu Ou , and Yan ZhangACM Trans. Inf. Syst., Sep 2020
Neural networks have been extensively used in recommender systems. Embedding layers are not only necessary but also crucial for neural models in recommendation as a typical discrete task. In this article, we argue that the widely used l2 regularization for normal neural layers (e.g., fully connected layers) is not ideal for embedding layers from the perspective of regularization theory in Reproducing Kernel Hilbert Space. More specifically, the l2 regularization corresponds to the inner product and the distance in the Euclidean space where correlations between discrete objects (e.g., items) are not well captured. Inspired by this observation, we propose a graph-based regularization approach to serve as a counterpart of the l2 regularization for embedding layers. The proposed regularization incurs almost no extra computational overhead especially when being trained with mini-batches. We also discuss its relationships to other approaches (namely, data augmentation, graph convolution, and joint learning) theoretically. We conducted extensive experiments on five publicly available datasets from various domains with two state-of-the-art recommendation models. Results show that given a kNN (k-nearest neighbor) graph constructed directly from training data without external information, the proposed approach significantly outperforms the l2 regularization on all the datasets and achieves more notable improvements for long-tail users and items.
@article{Zhang-2020-graph, google_scholar = {tS2w5q8j5-wC}, author = {Zhang, Yuan and Sun, Fei and Yang, Xiaoyong and Xu, Chen and Ou, Wenwu and Zhang, Yan}, title = {Graph-based Regularization on Embedding Layers for Recommendation}, year = {2020}, issue_date = {January 2021}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {39}, number = {1}, issn = {1046-8188}, url = {https://doi.org/10.1145/3414067}, doi = {10.1145/3414067}, journal = {ACM Trans. Inf. Syst.}, month = sep, articleno = {2}, numpages = {27}, keywords = {neural recommender system, graph-based regularization, Embedding} }
2019
- A pareto-efficient algorithm for multiple objective optimization in e-commerce recommendationXiao Lin , Hongjie Chen , Changhua Pei , Fei Sun , Xuanji Xiao , Hanxiao Sun , Yongfeng Zhang , Wenwu Ou , and Peng JiangIn Proceedings of the 13th ACM Conference on Recommender Systems, Copenhagen, Denmark, Sep 2019
Recommendation with multiple objectives is an important but difficult problem, where the coherent difficulty lies in the possible conflicts between objectives. In this case, multi-objective optimization is expected to be Pareto efficient, where no single objective can be further improved without hurting the others. However existing approaches to Pareto efficient multi-objective recommendation still lack good theoretical guarantees.In this paper, we propose a general framework for generating Pareto efficient recommendations. Assuming that there are formal differentiable formulations for the objectives, we coordinate these objectives with a weighted aggregation. Then we propose a condition ensuring Pareto efficiency theoretically and a two-step Pareto efficient optimization algorithm. Meanwhile the algorithm can be easily adapted for Pareto Frontier generation and fair recommendation selection. We specifically apply the proposed framework on E-Commerce recommendation to optimize GMV and CTR simultaneously. Extensive online and offline experiments are conducted on the real-world E-Commerce recommender system and the results validate the Pareto efficiency of the framework.To the best of our knowledge, this work is among the first to provide a Pareto efficient framework for multi-objective recommendation with theoretical guarantees. Moreover, the framework can be applied to any other objectives with differentiable formulations and any model with gradients, which shows its strong scalability.
@inproceedings{Lin-2019-pareto, google_scholar = {yD5IFk8b50cC}, author = {Lin, Xiao and Chen, Hongjie and Pei, Changhua and Sun, Fei and Xiao, Xuanji and Sun, Hanxiao and Zhang, Yongfeng and Ou, Wenwu and Jiang, Peng}, title = {A pareto-efficient algorithm for multiple objective optimization in e-commerce recommendation}, year = {2019}, isbn = {9781450362436}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3298689.3346998}, doi = {10.1145/3298689.3346998}, booktitle = {Proceedings of the 13th ACM Conference on Recommender Systems}, pages = {20–28}, numpages = {9}, keywords = {learning to rank, multiple obecjtive optimization, pareto efficiency, recommendation}, location = {Copenhagen, Denmark}, series = {RecSys '19} }RecSysBest Long Paper Runner Up - BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from TransformerFei Sun , Jun Liu , Jian Wu , Changhua Pei , Xiao Lin , Wenwu Ou , and Peng JiangIn Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, China, Sep 2019
Modeling users’ dynamic preferences from their historical behaviors is challenging and crucial for recommendation systems. Previous methods employ sequential neural networks to encode users’ historical interactions from left to right into hidden representations for making recommendations. Despite their effectiveness, we argue that such left-to-right unidirectional models are sub-optimal due to the limitations including: begin enumerate* [label=seriesitshapealph*upshape)] item unidirectional architectures restrict the power of hidden representation in users’ behavior sequences; item they often assume a rigidly ordered sequence which is not always practical. end enumerate* To address these limitations, we proposed a sequential recommendation model called BERT4Rec, which employs the deep bidirectional self-attention to model user behavior sequences. To avoid the information leakage and efficiently train the bidirectional model, we adopt the Cloze objective to sequential recommendation, predicting the random masked items in the sequence by jointly conditioning on their left and right context. In this way, we learn a bidirectional representation model to make recommendations by allowing each item in user historical behaviors to fuse information from both left and right sides. Extensive experiments on four benchmark datasets show that our model outperforms various state-of-the-art sequential models consistently.
@inproceedings{fei-2019-BERT4REC, google_scholar = {RYcK_YlVTxYC}, author = {Sun, Fei and Liu, Jun and Wu, Jian and Pei, Changhua and Lin, Xiao and Ou, Wenwu and Jiang, Peng}, title = {BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer}, year = {2019}, isbn = {9781450369763}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3357384.3357895}, doi = {10.1145/3357384.3357895}, booktitle = {Proceedings of the 28th ACM International Conference on Information and Knowledge Management}, pages = {1441–1450}, numpages = {10}, keywords = {sequential recommendation, cloze, bidirectional sequential model}, location = {Beijing, China}, series = {CIKM '19} }CIKMTop 5 most cited in CIKM (all time)