서브메뉴
검색
The Roads Not Taken: Model Multiplicity in Machine Learning
The Roads Not Taken: Model Multiplicity in Machine Learning
상세정보
- 자료유형
- 학위논문 서양
- 최종처리일시
- 20250211151425
- ISBN
- 9798382786476
- DDC
- 004
- 서명/저자
- The Roads Not Taken: Model Multiplicity in Machine Learning
- 발행사항
- [Sl] : Harvard University, 2024
- 발행사항
- Ann Arbor : ProQuest Dissertations & Theses, 2024
- 형태사항
- 138 p
- 주기사항
- Source: Dissertations Abstracts International, Volume: 85-12, Section: B.
- 주기사항
- Advisor: Parkes, David C.
- 학위논문주기
- Thesis (Ph.D.)--Harvard University, 2024.
- 초록/해제
- 요약In machine learning, model multiplicity is the existence of multiple models that perform equally well for a given prediction task (also known as the "Rashomon effect" ). The set of near-optimal models is referred to as the "Rashomon set." Predictive multiplicity examines how predictions change over this set of near-optimal models. If model outputs vary significantly across similar models, this information can offer insight into predictive arbitrariness. In this thesis, I introduce frameworks for evaluating and leveraging predictive multiplicity in different settings.First, I present methods to measure predictive multiplicity in probabilistic classification (predicting the probability of a positive outcome) and develop optimization-based methods to compute these measures efficiently and reliably for convex empirical risk minimization problems. Empirical results show that real-world probabilistic classification tasks can in fact admit competing models that assign substantially different risk estimates. Additionally, I provide insight into how predictive multiplicity arises by analyzing dataset characteristics.Second, I formulate predictive multiplicity analysis in a resource constrained setting recognizing that predictive allocation tasks are governed by a resource budget. I also extend the multiplicity framing, outlining the concept of multi-target multiplicity for quantifying the impact of choices made in regard to target specification for a given predictive allocation task. With this framework, I demonstrate how to fit separate models that are useful for predicting the three outcomes of interest independently and arriving at a way of ranking patients that results in a more equitable allocation.Third, I investigate the connections between predictive multiplicity and predictive churn which is the change in predictions pre- and post- model update in response to a change in training data. I present empirical and theoretical results on characterizing churn in terms of the Rashomon set. Results show that churn unstable points overlap by more than 50 percent with ambiguity points. This points to similarities in the two concepts. Theoretical results to characterize predictive churn between two Rashomon sets as well as churn between models within one Rashomon set hinges on the type of Rashomon set.I focus on predictive multiplicity to advocate for transparency in the prediction model training procedure. These methods to evaluate predictive multiplicity, as well as connections with predictive churn, contribute to a larger effort for machine learning researchers to be accountable to the individuals affected by model predictions. Similar to a person deciding between roads to take while travelling, insight into alternative options (i.e., roads not taken) may provide insight into the significance of the decisions made.
- 일반주제명
- Computer science
- 일반주제명
- Applied mathematics
- 키워드
- Fairness
- 키워드
- Machine learning
- 키워드
- Rashomon set
- 키워드
- Rashomon effect
- 기타저자
- Harvard University Engineering and Applied Sciences - Applied Math
- 기본자료저록
- Dissertations Abstracts International. 85-12B.
- 전자적 위치 및 접속
- 로그인 후 원문을 볼 수 있습니다.
MARC
008250123s2024 us c eng d■001000017161642
■00520250211151425
■006m o d
■007cr#unu||||||||
■020 ▼a9798382786476
■035 ▼a(MiAaPQ)AAI31294605
■040 ▼aMiAaPQ▼cMiAaPQ
■0820 ▼a004
■1001 ▼aWatson-Daniels, Jamelle D.▼0(orcid)0000-0003-4711-8789
■24510▼aThe Roads Not Taken: Model Multiplicity in Machine Learning
■260 ▼a[Sl]▼bHarvard University▼c2024
■260 1▼aAnn Arbor▼bProQuest Dissertations & Theses▼c2024
■300 ▼a138 p
■500 ▼aSource: Dissertations Abstracts International, Volume: 85-12, Section: B.
■500 ▼aAdvisor: Parkes, David C.
■5021 ▼aThesis (Ph.D.)--Harvard University, 2024.
■520 ▼aIn machine learning, model multiplicity is the existence of multiple models that perform equally well for a given prediction task (also known as the "Rashomon effect" ). The set of near-optimal models is referred to as the "Rashomon set." Predictive multiplicity examines how predictions change over this set of near-optimal models. If model outputs vary significantly across similar models, this information can offer insight into predictive arbitrariness. In this thesis, I introduce frameworks for evaluating and leveraging predictive multiplicity in different settings.First, I present methods to measure predictive multiplicity in probabilistic classification (predicting the probability of a positive outcome) and develop optimization-based methods to compute these measures efficiently and reliably for convex empirical risk minimization problems. Empirical results show that real-world probabilistic classification tasks can in fact admit competing models that assign substantially different risk estimates. Additionally, I provide insight into how predictive multiplicity arises by analyzing dataset characteristics.Second, I formulate predictive multiplicity analysis in a resource constrained setting recognizing that predictive allocation tasks are governed by a resource budget. I also extend the multiplicity framing, outlining the concept of multi-target multiplicity for quantifying the impact of choices made in regard to target specification for a given predictive allocation task. With this framework, I demonstrate how to fit separate models that are useful for predicting the three outcomes of interest independently and arriving at a way of ranking patients that results in a more equitable allocation.Third, I investigate the connections between predictive multiplicity and predictive churn which is the change in predictions pre- and post- model update in response to a change in training data. I present empirical and theoretical results on characterizing churn in terms of the Rashomon set. Results show that churn unstable points overlap by more than 50 percent with ambiguity points. This points to similarities in the two concepts. Theoretical results to characterize predictive churn between two Rashomon sets as well as churn between models within one Rashomon set hinges on the type of Rashomon set.I focus on predictive multiplicity to advocate for transparency in the prediction model training procedure. These methods to evaluate predictive multiplicity, as well as connections with predictive churn, contribute to a larger effort for machine learning researchers to be accountable to the individuals affected by model predictions. Similar to a person deciding between roads to take while travelling, insight into alternative options (i.e., roads not taken) may provide insight into the significance of the decisions made.
■590 ▼aSchool code: 0084.
■650 4▼aComputer science
■650 4▼aApplied mathematics
■653 ▼aFairness
■653 ▼aMachine learning
■653 ▼aPredictive multiplicity
■653 ▼aRashomon set
■653 ▼aRashomon effect
■690 ▼a0984
■690 ▼a0364
■71020▼aHarvard University▼bEngineering and Applied Sciences - Applied Math.
■7730 ▼tDissertations Abstracts International▼g85-12B.
■790 ▼a0084
■791 ▼aPh.D.
■792 ▼a2024
■793 ▼aEnglish
■85640▼uhttp://www.riss.kr/pdu/ddodLink.do?id=T17161642▼nKERIS▼z이 자료의 원문은 한국교육학술정보원에서 제공합니다.
