| Title | : | Active Learning with a Budget to Rank Candidates Rated by Disjoint Assessors. |
| Speaker | : | Phule Tushar Jaywant (IITM) |
| Details | : | Wed, 5 Apr, 2023 10:00 AM @ SSB 223 |
| Abstract: | : | We consider the problem of learning to rank a set of candidates rated by a set of assessors. In several practical settings, the assessors are divided into panels, each assessing a set of candidates. In this setup, standard matrix completion algorithms fail to recover the entire matrix meaningfully, so the ranking obtained from such a completed matrix is inaccurate. We consider the case where a small extra budget is available, which an algorithm can actively use to choose a set of (candidate, assessor) pairs to query to obtain good rankings of the candidates. We propose two novel algorithms, Query by Object Probability and Local Coherence Probability (OPLP) and Query by Base Factor and Local Coherence Probability (BFLP). These algorithms learn good rankings in an active query-based model and are inspired by two natural but different human assessor models. Specifically, they decide to query a candidate-assessor pair by first choosing a candidate to query using a certain probabilistic scheme and then choosing the best assessor to query for the selected candidate based on a local coherence-based probabilistic scheme. We conduct extensive experiments on synthetic and real-world datasets to test our algorithms against several baselines. Our experimental results show that the proposed algorithms outperform existing baselines for this problem. |
