서브메뉴
검색
On the Importance of Inherent Structural Properties for Learning in Markov Decision Processes
On the Importance of Inherent Structural Properties for Learning in Markov Decision Processes
상세정보
- 자료유형
- 학위논문 서양
- 최종처리일시
- 20250211152107
- ISBN
- 9798382741130
- DDC
- 519
- 저자명
- Adler, Saghar.
- 서명/저자
- On the Importance of Inherent Structural Properties for Learning in Markov Decision Processes
- 발행사항
- [Sl] : University of Michigan, 2024
- 발행사항
- Ann Arbor : ProQuest Dissertations & Theses, 2024
- 형태사항
- 158 p
- 주기사항
- Source: Dissertations Abstracts International, Volume: 85-12, Section: B.
- 주기사항
- Advisor: Subramanian, Vijay Gautam.
- 학위논문주기
- Thesis (Ph.D.)--University of Michigan, 2024.
- 초록/해제
- 요약Recently, reinforcement learning methodologies have been applied to solve sequential decision-making problems in various fields, such as robotics and autonomous control, communication and networking, and resource allocation and scheduling. Despite great practical success, there has been less progress in developing theoretical performance guarantees for such complex systems. This dissertation aims to address the limitations of current theoretical frameworks and extend the applicability of learning-based control methods to more complex, real-life domains discussed above. This objective is achieved in two different settings using the inherent structural properties of the Markov decision processes used to model such systems. For admission control in systems modeled by the Erlang-B blocking model with unknown arrival and service rates, in the first setting, we use model knowledge to compensate for the lack of reward signals. Here, we propose a learning algorithm based on the self-tuning adaptive control and not only prove that our algorithm is asymptotically optimal but also provide finite-time regret guarantees. The second setting develops a framework to address the challenge of applying reinforcement learning methods to Markov decision processes with countably infinite state spaces and unbounded cost functions. An existing learning algorithm based on Thompson sampling with dynamically-sized episodes is extended to countably infinite state space using the ergodicity properties of Markov decision processes. We establish asymptotic optimality of our learning-based control policy by providing a sub-linear (in time-horizon) regret guarantee. Our framework is focused on models that arise in queueing system models of communication networks, computing systems, and processing networks. Hence, to demonstrate the applicability of our method, we also apply it to the problem of controlling two queueing systems with unknown dynamics.
- 일반주제명
- Applied mathematics
- 일반주제명
- Engineering
- 기타저자
- University of Michigan Electrical and Computer Engineering
- 기본자료저록
- Dissertations Abstracts International. 85-12B.
- 전자적 위치 및 접속
- 로그인 후 원문을 볼 수 있습니다.
MARC
008250123s2024 us c eng d■001000017162876
■00520250211152107
■006m o d
■007cr#unu||||||||
■020 ▼a9798382741130
■035 ▼a(MiAaPQ)AAI31349131
■035 ▼a(MiAaPQ)umichrackham005525
■040 ▼aMiAaPQ▼cMiAaPQ
■0820 ▼a519
■1001 ▼aAdler, Saghar.
■24510▼aOn the Importance of Inherent Structural Properties for Learning in Markov Decision Processes
■260 ▼a[Sl]▼bUniversity of Michigan▼c2024
■260 1▼aAnn Arbor▼bProQuest Dissertations & Theses▼c2024
■300 ▼a158 p
■500 ▼aSource: Dissertations Abstracts International, Volume: 85-12, Section: B.
■500 ▼aAdvisor: Subramanian, Vijay Gautam.
■5021 ▼aThesis (Ph.D.)--University of Michigan, 2024.
■520 ▼aRecently, reinforcement learning methodologies have been applied to solve sequential decision-making problems in various fields, such as robotics and autonomous control, communication and networking, and resource allocation and scheduling. Despite great practical success, there has been less progress in developing theoretical performance guarantees for such complex systems. This dissertation aims to address the limitations of current theoretical frameworks and extend the applicability of learning-based control methods to more complex, real-life domains discussed above. This objective is achieved in two different settings using the inherent structural properties of the Markov decision processes used to model such systems. For admission control in systems modeled by the Erlang-B blocking model with unknown arrival and service rates, in the first setting, we use model knowledge to compensate for the lack of reward signals. Here, we propose a learning algorithm based on the self-tuning adaptive control and not only prove that our algorithm is asymptotically optimal but also provide finite-time regret guarantees. The second setting develops a framework to address the challenge of applying reinforcement learning methods to Markov decision processes with countably infinite state spaces and unbounded cost functions. An existing learning algorithm based on Thompson sampling with dynamically-sized episodes is extended to countably infinite state space using the ergodicity properties of Markov decision processes. We establish asymptotic optimality of our learning-based control policy by providing a sub-linear (in time-horizon) regret guarantee. Our framework is focused on models that arise in queueing system models of communication networks, computing systems, and processing networks. Hence, to demonstrate the applicability of our method, we also apply it to the problem of controlling two queueing systems with unknown dynamics.
■590 ▼aSchool code: 0127.
■650 4▼aApplied mathematics
■650 4▼aEngineering
■653 ▼aReinforcement learning
■653 ▼aLearning in queueing systems
■653 ▼aMarkov decision processes
■653 ▼aAsymptotic optimality
■690 ▼a0537
■690 ▼a0800
■690 ▼a0364
■71020▼aUniversity of Michigan▼bElectrical and Computer Engineering.
■7730 ▼tDissertations Abstracts International▼g85-12B.
■790 ▼a0127
■791 ▼aPh.D.
■792 ▼a2024
■793 ▼aEnglish
■85640▼uhttp://www.riss.kr/pdu/ddodLink.do?id=T17162876▼nKERIS▼z이 자료의 원문은 한국교육학술정보원에서 제공합니다.
