Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

Search 250 million research papers

Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.

About 10,000 papers for “Preference learning”

Preference Learning

Johannes Fürnkranz, Eyke Hüllermeier · Encyclopedia of Machine Learning and Data Mining · 2017

Cited by 0

Hindsight Preference Learning for Offline Preference-based Reinforcement Learning

Chen-Xiao Gao, Shengjun Fang, Chenjun Xiao et al. · arXiv (Cornell University) · 2024

Offline preference-based reinforcement learning (RL), which focuses on optimizing policies using human preferences between pairs of trajectory segments selected from an offline dataset, has emerged as a practical avenue for RL applications. Existing works rely on extracting step-wise reward signals from trajectory-wise preference annotations, assuming that preferences correlate with the cumulative Markovian rewards. However, such methods fail to capture the holistic perspective of data annotation: Humans often assess the desirability of a sequence of actions by considering the overall outcome rather than the immediate rewards. To address this challenge, we propose to model human preferences using rewards conditioned on future outcomes of the trajectory segments, i.e. the hindsight information. For downstream RL optimization, the reward of each step is calculated by marginalizing over possible future outcomes, the distribution of which is approximated by a variational auto-encoder trained using the offline dataset. Our proposed method, Hindsight Preference Learning (HPL), can facilitate credit assignment by taking full advantage of vast trajectory data available in massive unlabeled datasets. Comprehensive empirical studies demonstrate the benefits of HPL in delivering robust and advantageous rewards across various domains. Our code is publicly released at https://github.com/typoverflow/WiseRL.

Cited by 0Open access

Preference Learning Methods

Ota Pichl · Digital Repository (National Repository of Grey Literature) · 2013

The diploma thesis is focused on preference learning. Preferences can be analyzed in many areas starting from economics, over the statistics to informatics. This thesis is focused on informatics point of view on preferences. At the beginning it is focusing on preferences in general and analyzing its origin in economical science. Then this knowledge base is used for analysis of informatics methods employed in preference learning which also includes machine learning and describes how these sciences are connected with each other. Practical part of work is focused on employing informatics preferences in practice. The basic tasks and methods are described at the beginning and followed by more detailed analysis of one the methods (UTA NM). The result consists of description and implementation of a REST web service that can be used for one of the preference learning tasks.

Cited by 0

Learning about Preference

Evan Piermont · D-Scholarship@Pitt (University of Pittsburgh) · 2017

This dissertation includes three essays, each investigating the formation and evolution of preference, with particular attention paid to epistemological concerns, dynamics, and learning. Loosely speaking, each chapter provides a different approach to modeling how a Decision Maker (DM) might learn about her own preference. The work as a whole adheres to the classical notions of revealed preference and utility maximization and the chapters are ordered by their level of abstraction. The first chapter considers DM who learns by observing the environment in which decisions must be made. The second chapter, written jointly with Roee Teper, considers a DM who learns by observing the consequences of her prior choices. Finally, the third chapter considers a DM who learns novel of which she previously had not conception.

Cited by 0Open access

\nSparse Preference Learning

Tsivtsivadze, E., Heskes, T. · Radboud Repository (Radboud University) · 2010

\n Contains fulltext :\n 83983.pdf (Author’s version preprint ) (Open Access)\n

Cited by 0

Abductive Preference Learning

Ni, Yijin, Qi, Peng · ArXiv.org · 2025

Frontier large language models such as GPT-5 and Claude Sonnet remain prone to overconfidence even after alignment through Reinforcement Learning with Human Feedback (RLHF) and Direct Preference Optimization (DPO). For instance, they tend to offer the same conservative answer "No" to both questions "Can I eat the [food / potato chips] that has been left out overnight?" despite the latter requiring no refridgeration for safe consumption. We find that this failure is potentially attributed to a limitation of existing preference learning: it emphasizes selecting the correct response for a given prompt, while neglecting counterfactual prompts that should alter the response. To address this limitation, we propose abductive preference learning, a fine-tuning paradigm that reverses the conventional conditioning by learning preferences over prompts given a response. To validate this idea, we construct an abductive dataset derived from the HaluEval QA benchmark with 1,001 entries, implementing abductive DPO and its variant DPOP. Experiments reveal complementary strengths: standard methods improve response selection, abductive methods improve prompt discrimination, while a multitask objective unifies both. On the abductive dataset, multitask DPOP boosts accuracy from $90.0\%$ to $99.5\%$ in response selection and $54.7\%$ to $85.0\%$ in prompt discrimination, with qualitative evidence highlighting improved sensitivity to prompt differences. Finally, evaluation on AlpacaEval shows multitask DPOP improves win rate (from $5.26\%$ to $6.17\%$), confirming that abductive preference learning preserves the benefits of conventional preference optimization while addressing the overlooked challenge of counterfactual prompts.

Cited by 0Open access

Learning Preference Inventory

Arndt, Glenda J. Gorsuch · e-Publications@Marquette (Marquette University) · 1979

The purpose of this study was to develop a Learning Preference Inventory (LPI) which reflects a child's unique learning style. This LPI will assist educators to individualize educational programs of handicapped children as mandated by Public Law 94-142. The investigative method used to construct this essay was a literature review which focused on the importance of selected social-environmental variables in relation to learning style. Analysis of the literature review revealed that the environment plays a crucial role in the learning process, specifically, the learning style preference. The outcome of this study was the development of a Learning Preference Inventory (LPI) based on a need for a viable instrument to assess the learning environment. The LPI will inform educators of the child's preference for instructional procedures, presentations, and interpersonal communication interactions.

Cited by 0

2.1.1 Preference Learning Tasks...................................

Präferenzbasiertes Lernen, Tag Der Einreichung, Gutachten Prof Fürnkranz et al. · 2015

Hiermit versichere ich, die vorliegende Master-Thesis ohne Hilfe Dritter nur mit den angegebenen Quellen und Hilfsmitteln angefertigt zu haben. Alle Stellen, die aus Quellen entnommen wurden, sind als solche kenntlich gemacht. Diese Arbeit hat in gleicher oder ähnlicher Form noch keiner Prüfungs-behörde vorgelegen.

Cited by 0

The Preference Learning Toolbox

Vincent-Emmanuel Farrugia, Hèctor Martínez, Georgios N. Yannakakis · arXiv (Cornell University) · 2015

Preference learning (PL) is a core area of machine learning that handles datasets with ordinal relations. As the number of generated data of ordinal nature is increasing, the importance and role of the PL field becomes central within machine learning research and practice. This paper introduces an open source, scalable, efficient and accessible preference learning toolbox that supports the key phases of the data training process incorporating various popular data preprocessing, feature selection and preference learning methods.

Cited by 19Open access

Contextual Preference Distribution Learning

Benjamin Hudson, Laurent Charlin, Emma Frejinger · arXiv (Cornell University) · 2026

Decision-making problems often feature uncertainty stemming from heterogeneous and context-dependent human preferences. To address this, we propose a sequential learning-and-optimization pipeline to learn preference distributions and leverage them to solve downstream problems, for example risk-averse formulations. We focus on human choice settings that can be formulated as (integer) linear programs. In such settings, existing inverse optimization and choice modelling methods infer preferences from observed choices but typically produce point estimates or fail to capture contextual shifts, making them unsuitable for risk-averse decision-making. Using a bounded-variance score function gradient estimator, we train a predictive model mapping contextual features to a rich class of parameterizable distributions. This approach yields a maximum likelihood estimate. The model generates scenarios for unseen contexts in the subsequent optimization phase. In a synthetic ridesharing environment, our approach reduces average post-decision surprise by up to 114$\times$ compared to a risk-neutral approach with perfect predictions and up to 25$\times$ compared to leading risk-averse baselines.

Cited by 0Open access

Contextual Preference Distribution Learning

Benjamin Hudson, Laurent Charlin, Emma Frejinger · ArXiv.org · 2026

Decision-making problems often feature uncertainty stemming from heterogeneous and context-dependent human preferences. To address this, we propose a sequential learning-and-optimization pipeline to learn preference distributions and leverage them to solve downstream problems, for example risk-averse formulations. We focus on human choice settings that can be formulated as (integer) linear programs. In such settings, existing inverse optimization and choice modelling methods infer preferences from observed choices but typically produce point estimates or fail to capture contextual shifts, making them unsuitable for risk-averse decision-making. Using a bounded-variance score function gradient estimator, we train a predictive model mapping contextual features to a rich class of parameterizable distributions. This approach yields a maximum likelihood estimate. The model generates scenarios for unseen contexts in the subsequent optimization phase. In a synthetic ridesharing environment, our approach reduces average post-decision surprise by up to 114$\times$ compared to a risk-neutral approach with perfect predictions and up to 25$\times$ compared to leading risk-averse baselines.

Cited by 0Open access

Unsupervised Human Preference Learning

Sumuk Shashidhar, Abhinav Chinta, Vaibhav Sahai et al. · 2024

Large language models demonstrate impressive reasoning abilities but struggle to provide personalized content due to their lack of individual user preference information.Existing methods, such as in-context learning and parameter-efficient fine-tuning, fall short in capturing the complexity of human preferences, especially given the small, personal datasets individuals possess.In this paper, we propose a novel approach utilizing small parameter models as preference agents to generate natural language rules that guide a larger, pre-trained model, enabling efficient personalization.Our method involves a small, local "steering wheel" model that directs the outputs of a much larger foundation model, producing content tailored to an individual's preferences while leveraging the extensive knowledge and capabilities of the large model.Importantly, this personalization is achieved without the need to fine-tune the large model.Experimental results demonstrate that our technique significantly outperforms baseline personalization methods.By allowing foundation models to adapt to individual preferences in a data and compute-efficient manner, our approach paves the way for highly personalized language model applications.

Cited by 1Open access

Debiasing Online Preference Learning via Preference Feature Preservation

Dongyoung Kim, Jinsung Yoon, Jinwoo Shin et al. · 2025

Recent preference learning frameworks for large language models (LLMs) simplify human preferences with binary pairwise comparisons and scalar rewards.This simplification could make LLMs' responses biased to mostly preferred features, and would be exacerbated during the iterations of online preference learning steps.To address these challenges, we propose a novel framework coined PFP (Preference Feature Preservation).The key idea of PFP is maintaining the distribution of human preference features and utilizing such rich signals throughout the online preference learning process.Specifically, PFP first extract preference features from offline pairwise human preference data and trains a feature classifier.Then, using trained classifier and the distribution preserving optimization, PFP maps appropriate preference features for a new input instruction during online learning.Lastly, PFP trains LLM using the existing preference learning method, by incorporating the preference feature into system prompts and enabling LLM to explicitly handle various human preferences.Our experiments demonstrate that PFP successfully mitigates the bias in preference features during online learning, and hence achieves superior performance compared to previous preference learning methods on standard benchmarks to evaluate LLM alignment. 1

Cited by 0Open access