서브메뉴
검색
Principles of Uncertainty in Probabilistic Machine Learning
Principles of Uncertainty in Probabilistic Machine Learning
상세정보
- 자료유형
- 학위논문 서양
- 최종처리일시
- 20260202104854
- ISBN
- 9798288814181
- DDC
- 330
- 서명/저자
- Principles of Uncertainty in Probabilistic Machine Learning
- 발행사항
- [Sl] : Stanford University, 2025
- 발행사항
- Ann Arbor : ProQuest Dissertations & Theses, 2025
- 형태사항
- 117 p
- 주기사항
- Source: Dissertations Abstracts International, Volume: 87-02, Section: B.
- 주기사항
- Advisor: Ermon, Stefano.
- 학위논문주기
- Thesis (Ph.D.)--Stanford University, 2025.
- 초록/해제
- 요약Reliable uncertainty quantification is fundamental to the safe and effective deployment of machine learning systems in high-stakes settings, where predictions inform decisions ranging from medical diagnoses to infrastructure management and scientific discovery. This dissertation presents a study of probabilistic prediction with a focus on making uncertainty estimates trustworthy by achieving calibration - wherein predicted probabilities align with the empirical frequencies of events, such as a 90% confidence interval including the observed outcome 90% of the time. I propose interventions to improve calibration across the model lifecycle: training objectives to encourage calibration, post-processing methods to correct miscalibration, and online techniques for adaptively preserving calibration during deployment in nonstationary environments. The first part addresses post-hoc recalibration. I introduce modular conformal calibration, a general framework that encompasses and extends existing post-hoc uncertainty quantification techniques such as isotonic regression and conformal prediction. This framework identifies a design space for recalibration procedures and provides finite-sample calibration guarantees for any model recalibrated using these strategies. This allows practitioners to trade off between computational cost, meaningful likelihoods, deterministic behavior, and stronger calibration guarantees. In the second part, I turn to training-time calibration with the goal of encouraging calibration while maintaining sharpness - the degree to which predictions are confident and informative. I propose a class of differentiable calibration measures that serve as regularization objectives, enabling co-optimization of calibration and sharpness during training. These objectives encompass many popular notions of calibration for regression and classification previously enforced only after training, instead incorporating them into standard empirical risk minimization. They also enable task-specific calibration objectives, allowing probabilistic models whose uncertainty estimates are both statistically coherent and aligned with the practical needs of downstream decision-making. The third part investigates calibration under distribution shift, a central challenge in real-world deployments. I consider an online forecasting setting where data may evolve over time or be selected adversarially. Building on Blackwell approachability theory, I develop a general strategy for enforcing calibration guarantees across arbitrary observation sequences under minimal assumptions. This framework supports diverse calibration notions, including distribution and decision calibration, through both oracle-based and computationally tractable algorithms. I further present gradient-based approaches that relax the guarantees while enabling broader applicability. Empirical evaluations demonstrate that these methods maintain calibrated forecasts while achieving vanishing regret with respect to expert predictors. Collectively, this dissertation provides principled strategies for uncertainty estimation with increased flexibility by enforcing many forms of calibration at each stage of model development. This comprehensive approach to calibration across the model lifecycle enables practitioners to tailor uncertainty quantification to their specific applications while reliably informing decisions in high-stakes settings.
- 일반주제명
- Forecasting
- 일반주제명
- Decision making
- 일반주제명
- Computer science
- 기타저자
- Stanford University.
- 기본자료저록
- Dissertations Abstracts International. 87-02B.
- 전자적 위치 및 접속
- 로그인 후 원문을 볼 수 있습니다.
MARC
008260126s2025 us c eng d■001000017359241
■00520260202104854
■006m o d
■007cr#unu||||||||
■020 ▼a9798288814181
■035 ▼a(MiAaPQ)AAI32200994
■035 ▼a(MiAaPQ)Stanfordsm978sh0523
■040 ▼aMiAaPQ▼cMiAaPQ
■0820 ▼a330
■1001 ▼aMarx, Charles Thomas.
■24510▼aPrinciples of Uncertainty in Probabilistic Machine Learning
■260 ▼a[Sl]▼bStanford University▼c2025
■260 1▼aAnn Arbor▼bProQuest Dissertations & Theses▼c2025
■300 ▼a117 p
■500 ▼aSource: Dissertations Abstracts International, Volume: 87-02, Section: B.
■500 ▼aAdvisor: Ermon, Stefano.
■5021 ▼aThesis (Ph.D.)--Stanford University, 2025.
■520 ▼aReliable uncertainty quantification is fundamental to the safe and effective deployment of machine learning systems in high-stakes settings, where predictions inform decisions ranging from medical diagnoses to infrastructure management and scientific discovery. This dissertation presents a study of probabilistic prediction with a focus on making uncertainty estimates trustworthy by achieving calibration - wherein predicted probabilities align with the empirical frequencies of events, such as a 90% confidence interval including the observed outcome 90% of the time. I propose interventions to improve calibration across the model lifecycle: training objectives to encourage calibration, post-processing methods to correct miscalibration, and online techniques for adaptively preserving calibration during deployment in nonstationary environments. The first part addresses post-hoc recalibration. I introduce modular conformal calibration, a general framework that encompasses and extends existing post-hoc uncertainty quantification techniques such as isotonic regression and conformal prediction. This framework identifies a design space for recalibration procedures and provides finite-sample calibration guarantees for any model recalibrated using these strategies. This allows practitioners to trade off between computational cost, meaningful likelihoods, deterministic behavior, and stronger calibration guarantees. In the second part, I turn to training-time calibration with the goal of encouraging calibration while maintaining sharpness - the degree to which predictions are confident and informative. I propose a class of differentiable calibration measures that serve as regularization objectives, enabling co-optimization of calibration and sharpness during training. These objectives encompass many popular notions of calibration for regression and classification previously enforced only after training, instead incorporating them into standard empirical risk minimization. They also enable task-specific calibration objectives, allowing probabilistic models whose uncertainty estimates are both statistically coherent and aligned with the practical needs of downstream decision-making. The third part investigates calibration under distribution shift, a central challenge in real-world deployments. I consider an online forecasting setting where data may evolve over time or be selected adversarially. Building on Blackwell approachability theory, I develop a general strategy for enforcing calibration guarantees across arbitrary observation sequences under minimal assumptions. This framework supports diverse calibration notions, including distribution and decision calibration, through both oracle-based and computationally tractable algorithms. I further present gradient-based approaches that relax the guarantees while enabling broader applicability. Empirical evaluations demonstrate that these methods maintain calibrated forecasts while achieving vanishing regret with respect to expert predictors. Collectively, this dissertation provides principled strategies for uncertainty estimation with increased flexibility by enforcing many forms of calibration at each stage of model development. This comprehensive approach to calibration across the model lifecycle enables practitioners to tailor uncertainty quantification to their specific applications while reliably informing decisions in high-stakes settings.
■590 ▼aSchool code: 0212.
■650 4▼aForecasting
■650 4▼aDecision making
■650 4▼aComputer science
■690 ▼a0984
■690 ▼a0800
■71020▼aStanford University.
■7730 ▼tDissertations Abstracts International▼g87-02B.
■790 ▼a0212
■791 ▼aPh.D.
■792 ▼a2025
■793 ▼aEnglish
■85640▼uhttp://www.riss.kr/pdu/ddodLink.do?id=T17359241▼nKERIS▼z이 자료의 원문은 한국교육학술정보원에서 제공합니다.


