ARC/ACO Student Seminar
Jingyan Wang (CMU)
Friday, April 2, 2021
Virtual via Bluejeans - 12:00 pm
Title: Towards Understanding and Mitigating Biases
Abstract: There are many problems in real life that involve aggregating evaluation from people, such as hiring, peer grading and conference peer review. In this talk, I describe three types of biases that may arise in such problems, and propose methods to mitigate them. (1) We consider miscalibration, that is, different people have different calibration scales. We propose randomized algorithms that provably extract useful information under arbitrary miscalibration. (2) We consider the bias induced by the outcome experienced by people. For example, student ratings in teaching evaluation are affected by the grading leniency of the instructors. We propose an adaptive algorithm that debiases people's ratings under very mild assumptions of the biases. (3) Estimation bias arises when algorithms yield different performance on different subgroups of the population. We analyze the statistical bias (defined as the expected value of the estimate minus the true value) when using the maximum-likelihood estimator on pairwise comparison data, and then propose a simple modification of the estimator to reduce the bias.
----------------------------------
Videos of recent talks are available at: http://arc.gatech.edu/node/121
Click here to subscribe to the seminar email list: arc-colloq@Klauscc.gatech.edu
