서브메뉴
검색
Formalizing Tools for Meta-Analysis and Measurement Validity in the Social Sciences
Formalizing Tools for Meta-Analysis and Measurement Validity in the Social Sciences
상세정보
- 자료유형
- 학위논문 서양
- 최종처리일시
- 20250211152705
- ISBN
- 9798384015673
- DDC
- 320
- 서명/저자
- Formalizing Tools for Meta-Analysis and Measurement Validity in the Social Sciences
- 발행사항
- [Sl] : Northwestern University, 2024
- 발행사항
- Ann Arbor : ProQuest Dissertations & Theses, 2024
- 형태사항
- 166 p
- 주기사항
- Source: Dissertations Abstracts International, Volume: 86-02, Section: B.
- 주기사항
- Advisor: Arjona, Ana.
- 학위논문주기
- Thesis (Ph.D.)--Northwestern University, 2024.
- 초록/해제
- 요약This dissertation consists of three essays that contribute to the political science scholarship on meta-analysis and measurement validity. In the first paper, I develop a multi-method approach to meta-analysis in political science. Meta-analysis is a sophisticated and essential tool for cumulative knowledge production. However, traditional approaches to meta-analysis systematically exclude qualitative scholarship. This project proposes a systematic framework to deal with this shortcoming of meta-analysis: Bayesian Integrative Meta-Analysis (BIMA). Drawing on existing literature in Bayesian elicitation, I show how information from qualitative manuscripts can be systematically converted into Bayesian prior distribution information via a process I call conversion elicitation. This converted qualitative information can be combined and integrated into a Bayesian meta-analysis. The resulting posterior distribution of the standardized effect size results from a qualitatively informed prior and a likelihood distribution composed of the effect sizes from quantitative studies. I explicate the framework for BIMA using a toy case with simulated data and an applied example related to the effect of competitive wages on bureaucratic corruption. In addition to providing a framework for meta-analysis of diverse evidence types, this research contributes to the methodological scholarship on multi-method research tools. Strategies for mixed methods political science research have primarily focused on combining methods for causal research. This contribution presents an avenue for mixed methods approaches for other objectives, such as knowledge synthesis. In the second paper, I develop a framework for the informed use of proxy variables. Throughout the social sciences, practical challenges to measurement may frustrate scholars' attempts to measure their concepts of interest directly. In these cases of costly concepts, researchers may be compelled to use proxy variables that substitute values for the true concept measurement instead, often only assuming that the chosen proxy is good enough. In this paper, I propose an intermediate-range solution for discovering and reporting the potential measurement divergence between misaligned proxies and the true measures they intend to capture. I suggest obtaining a validation sample for which true measurements are obtained for the concept of interest is always preferable to assuming that a chosen proxy is sufficient. I outline three useful metrics to estimate, report, and potentially incorporate into additional statistical inference strategies. Furthermore, I use a simulation study to demonstrate which sampling procedures are most effective for obtaining these quantities. For more complex population structures, random or traditional stratified samples are informative in estimating metrics related to proxy-costly concept divergence. However, other stratified sampling approaches that consider the potential likelihood of measurement divergence across a population can also be informative of measurement divergence. This guide to validating proxies and reporting measurement divergence provides scholars with a practical and straightforward tool to account for the limitations of measurement choices.In the third paper, I develop a novel categorization of the distinct practical challenges that social scientists may encounter when attempting to measure concepts: resource constraints, ethical obligations, confidence in indicators, and unit divergence. I illustrate these practical challenges of concept measurement using various concepts throughout political science and discuss how these challenges may intersect with more theoretical considerations related to translating concepts into operational dimensions. I develop a cost-benefit roadmap for devising alternative concept measurements when researchers encounter insurmountable challenges to pursuing a preferred, theoretically-driven measure. Though this paper's primary contribution is to the scholarship on concept measurement, it also has implications for transparency in social science. The roadmap I develop provides scholars with a conceptual vocabulary to justify their concept measurement choices, allowing scholars to be more explicit about how different logistical costs of research contour methodological and design choices.
- 일반주제명
- Political science
- 일반주제명
- Sociology
- 일반주제명
- Statistics
- 키워드
- Measurement
- 키워드
- Meta-analysis
- 기타저자
- Northwestern University Political Science
- 기본자료저록
- Dissertations Abstracts International. 86-02B.
- 전자적 위치 및 접속
- 로그인 후 원문을 볼 수 있습니다.
MARC
008250123s2024 us c eng d■001000017163419
■00520250211152705
■006m o d
■007cr#unu||||||||
■020 ▼a9798384015673
■035 ▼a(MiAaPQ)AAI31488285
■040 ▼aMiAaPQ▼cMiAaPQ
■0820 ▼a320
■1001 ▼aMoore, Sarah Elizabeth.▼0(orcid)0000-0003-0903-4277
■24510▼aFormalizing Tools for Meta-Analysis and Measurement Validity in the Social Sciences
■260 ▼a[Sl]▼bNorthwestern University▼c2024
■260 1▼aAnn Arbor▼bProQuest Dissertations & Theses▼c2024
■300 ▼a166 p
■500 ▼aSource: Dissertations Abstracts International, Volume: 86-02, Section: B.
■500 ▼aAdvisor: Arjona, Ana.
■5021 ▼aThesis (Ph.D.)--Northwestern University, 2024.
■520 ▼aThis dissertation consists of three essays that contribute to the political science scholarship on meta-analysis and measurement validity. In the first paper, I develop a multi-method approach to meta-analysis in political science. Meta-analysis is a sophisticated and essential tool for cumulative knowledge production. However, traditional approaches to meta-analysis systematically exclude qualitative scholarship. This project proposes a systematic framework to deal with this shortcoming of meta-analysis: Bayesian Integrative Meta-Analysis (BIMA). Drawing on existing literature in Bayesian elicitation, I show how information from qualitative manuscripts can be systematically converted into Bayesian prior distribution information via a process I call conversion elicitation. This converted qualitative information can be combined and integrated into a Bayesian meta-analysis. The resulting posterior distribution of the standardized effect size results from a qualitatively informed prior and a likelihood distribution composed of the effect sizes from quantitative studies. I explicate the framework for BIMA using a toy case with simulated data and an applied example related to the effect of competitive wages on bureaucratic corruption. In addition to providing a framework for meta-analysis of diverse evidence types, this research contributes to the methodological scholarship on multi-method research tools. Strategies for mixed methods political science research have primarily focused on combining methods for causal research. This contribution presents an avenue for mixed methods approaches for other objectives, such as knowledge synthesis. In the second paper, I develop a framework for the informed use of proxy variables. Throughout the social sciences, practical challenges to measurement may frustrate scholars' attempts to measure their concepts of interest directly. In these cases of costly concepts, researchers may be compelled to use proxy variables that substitute values for the true concept measurement instead, often only assuming that the chosen proxy is good enough. In this paper, I propose an intermediate-range solution for discovering and reporting the potential measurement divergence between misaligned proxies and the true measures they intend to capture. I suggest obtaining a validation sample for which true measurements are obtained for the concept of interest is always preferable to assuming that a chosen proxy is sufficient. I outline three useful metrics to estimate, report, and potentially incorporate into additional statistical inference strategies. Furthermore, I use a simulation study to demonstrate which sampling procedures are most effective for obtaining these quantities. For more complex population structures, random or traditional stratified samples are informative in estimating metrics related to proxy-costly concept divergence. However, other stratified sampling approaches that consider the potential likelihood of measurement divergence across a population can also be informative of measurement divergence. This guide to validating proxies and reporting measurement divergence provides scholars with a practical and straightforward tool to account for the limitations of measurement choices.In the third paper, I develop a novel categorization of the distinct practical challenges that social scientists may encounter when attempting to measure concepts: resource constraints, ethical obligations, confidence in indicators, and unit divergence. I illustrate these practical challenges of concept measurement using various concepts throughout political science and discuss how these challenges may intersect with more theoretical considerations related to translating concepts into operational dimensions. I develop a cost-benefit roadmap for devising alternative concept measurements when researchers encounter insurmountable challenges to pursuing a preferred, theoretically-driven measure. Though this paper's primary contribution is to the scholarship on concept measurement, it also has implications for transparency in social science. The roadmap I develop provides scholars with a conceptual vocabulary to justify their concept measurement choices, allowing scholars to be more explicit about how different logistical costs of research contour methodological and design choices.
■590 ▼aSchool code: 0163.
■650 4▼aPolitical science
■650 4▼aSociology
■650 4▼aStatistics
■653 ▼aConstruct validity
■653 ▼aMeasurement
■653 ▼aMeta-analysis
■653 ▼aSocial science methodology
■653 ▼aBayesian Integrative Meta-Analysis
■690 ▼a0615
■690 ▼a0626
■690 ▼a0463
■690 ▼a0601
■71020▼aNorthwestern University▼bPolitical Science.
■7730 ▼tDissertations Abstracts International▼g86-02B.
■790 ▼a0163
■791 ▼aPh.D.
■792 ▼a2024
■793 ▼aEnglish
■85640▼uhttp://www.riss.kr/pdu/ddodLink.do?id=T17163419▼nKERIS▼z이 자료의 원문은 한국교육학술정보원에서 제공합니다.


