서브메뉴
검색
Interpreting the Inner-Workings of Vision Models
Interpreting the Inner-Workings of Vision Models
상세정보
- 자료유형
- 학위논문 서양
- 최종처리일시
- 20260202103516
- ISBN
- 9798288862403
- DDC
- 004
- 서명/저자
- Interpreting the Inner-Workings of Vision Models
- 발행사항
- [Sl] : University of California, Berkeley, 2025
- 발행사항
- Ann Arbor : ProQuest Dissertations & Theses, 2025
- 형태사항
- 87 p
- 주기사항
- Source: Dissertations Abstracts International, Volume: 87-01, Section: B.
- 주기사항
- Advisor: Efros, Alexei A.
- 학위논문주기
- Thesis (Ph.D.)--University of California, Berkeley, 2025.
- 초록/해제
- 요약The field of computer vision has recently transitioned from hand-engineering systems to learning them from large-scale datasets via deep learning. This shift motivates a new kind of observational science - closer in spirit to experimental biology than traditional engineering - which aims to discover what is being learned by deep learning models and why these models work. This science analyzes the emergent internal computation in deep vision models, hoping to discover the basic computational blocks that enable visual intelligence.This thesis presents my initial steps in this observational AI science, focusing on interpreting the internal mechanisms of deep vision models. It showcases how this understanding is used to improve model generalization and unlock new tasks without any additional learning.I begin with an in-depth analysis of a single vision-language model, CLIP-ViT, and attempt to explain the functionality of two main components in its vision encoder --- the attention heads and the neurons. I show that automatic characterization of components is attainable and reveals surprisingly structured and interpretable behavior, such as heads specializing and polysemantic neuron roles. These interpretations enable the removal of spurious features from CLIP, zero-shot image segmentation, and automatic generation of adversarial images. Next, I show that some similar computational components, "Rosetta Neurons", emerge across a diverse set of models trained with different architectures, objectives, and supervision. These findings suggest that certain visual concepts and structures are inherently embedded in the natural world and can be learned by different models regardless of the specific task or architecture. That provides a path to a scalable understanding of vision models that can be used to repair and improve future models.
- 일반주제명
- Computer science
- 일반주제명
- Computer engineering
- 일반주제명
- Information technology
- 키워드
- Computer vision
- 키워드
- Deep learning
- 키워드
- Interpretability
- 키워드
- Rosetta Neurons
- 기타저자
- University of California, Berkeley Computer Science
- 기본자료저록
- Dissertations Abstracts International. 87-01B.
- 전자적 위치 및 접속
- 로그인 후 원문을 볼 수 있습니다.
MARC
008260126s2025 us c eng d■001000017357464
■00520260202103516
■006m o d
■007cr#unu||||||||
■020 ▼a9798288862403
■035 ▼a(MiAaPQ)AAI32038079
■040 ▼aMiAaPQ▼cMiAaPQ
■0820 ▼a004
■1001 ▼aGandelsman, Yossi.
■24510▼aInterpreting the Inner-Workings of Vision Models
■260 ▼a[Sl]▼bUniversity of California, Berkeley▼c2025
■260 1▼aAnn Arbor▼bProQuest Dissertations & Theses▼c2025
■300 ▼a87 p
■500 ▼aSource: Dissertations Abstracts International, Volume: 87-01, Section: B.
■500 ▼aAdvisor: Efros, Alexei A.
■5021 ▼aThesis (Ph.D.)--University of California, Berkeley, 2025.
■520 ▼aThe field of computer vision has recently transitioned from hand-engineering systems to learning them from large-scale datasets via deep learning. This shift motivates a new kind of observational science - closer in spirit to experimental biology than traditional engineering - which aims to discover what is being learned by deep learning models and why these models work. This science analyzes the emergent internal computation in deep vision models, hoping to discover the basic computational blocks that enable visual intelligence.This thesis presents my initial steps in this observational AI science, focusing on interpreting the internal mechanisms of deep vision models. It showcases how this understanding is used to improve model generalization and unlock new tasks without any additional learning.I begin with an in-depth analysis of a single vision-language model, CLIP-ViT, and attempt to explain the functionality of two main components in its vision encoder --- the attention heads and the neurons. I show that automatic characterization of components is attainable and reveals surprisingly structured and interpretable behavior, such as heads specializing and polysemantic neuron roles. These interpretations enable the removal of spurious features from CLIP, zero-shot image segmentation, and automatic generation of adversarial images. Next, I show that some similar computational components, "Rosetta Neurons", emerge across a diverse set of models trained with different architectures, objectives, and supervision. These findings suggest that certain visual concepts and structures are inherently embedded in the natural world and can be learned by different models regardless of the specific task or architecture. That provides a path to a scalable understanding of vision models that can be used to repair and improve future models.
■590 ▼aSchool code: 0028.
■650 4▼aComputer science
■650 4▼aComputer engineering
■650 4▼aInformation technology
■653 ▼aComputer vision
■653 ▼aDeep learning
■653 ▼aInterpretability
■653 ▼aRosetta Neurons
■653 ▼aZero-shot segmentation
■690 ▼a0984
■690 ▼a0489
■690 ▼a0464
■690 ▼a0800
■71020▼aUniversity of California, Berkeley▼bComputer Science.
■7730 ▼tDissertations Abstracts International▼g87-01B.
■790 ▼a0028
■791 ▼aPh.D.
■792 ▼a2025
■793 ▼aEnglish
■85640▼uhttp://www.riss.kr/pdu/ddodLink.do?id=T17357464▼nKERIS▼z이 자료의 원문은 한국교육학술정보원에서 제공합니다.
