서브메뉴
검색
Training Deep Neural Networks With In-Memory Computing- [electronic resource]
![Training Deep Neural Networks With In-Memory Computing - [electronic resource]](/Users/Baul/Images/book.png)
Training Deep Neural Networks With In-Memory Computing- [electronic resource]
상세정보
- 자료유형
- 학위논문파일 국외
- 최종처리일시
- 20240214101922
- ISBN
- 9798380850780
- DDC
- 621.3
- 서명/저자
- Training Deep Neural Networks With In-Memory Computing - [electronic resource]
- 발행사항
- [S.l.]: : Princeton University., 2023
- 발행사항
- Ann Arbor : : ProQuest Dissertations & Theses,, 2023
- 형태사항
- 1 online resource(140 p.)
- 주기사항
- Source: Dissertations Abstracts International, Volume: 85-05, Section: B.
- 주기사항
- Advisor: Verma, Naveen.
- 학위논문주기
- Thesis (Ph.D.)--Princeton University, 2023.
- 사용제한주기
- This item must not be sold to any third party vendors.
- 초록/해제
- 요약Deep learning has advanced machine capabilities in a variety of fields typically associated with human intelligence, including image recognition, object detection, natural language processing, healthcare, and competitive games. The models behind deep learning called Deep Neural Networks (DNNs) generally require billions of operations for inference, and a hundred if not thousand fold more operations for training. These operations are typically dominated by high dimensionality Matrix-Vector Multiplies (MVMs). Due to the dominance of MVMs, a number of hardware accelerators have been developed to enhance the compute efficiency of MVMs, but data movement and accessing typically remain key bottlenecks. In-Memory Computing (IMC) has the potential to overcome these key bottlenecks by performing computations in-place within dense 2-D memory. However, IMC has challenges of its own, as it fundamentally trades efficiency and throughput gains for dynamic range. This tradeoff is especially challenging for training where higher dynamic range is typically involved in the form of higher compute precision requirements and greater noise sensitivity compared to inferencing. In this work, we will discuss how to enable deep learning training with IMC by mitigating these challenges while substantially mapping operations to IMC. Key advancements in this work include: (1) leveraging new high-precision IMC techniques, like charge-based and analog-input IMC to reduce noise sources; (2) mapping aggressive quantization techniques such as radix-4 gradients to IMC; and (3) ADC range adjustments to better capture output distributions and mitigate biases during training. These methods enable training of DNNs on IMC capable of over 400 x energy savings and of achieving high levels of testing accuracy on a variety of DNN models.
- 일반주제명
- Computer engineering.
- 일반주제명
- Electrical engineering.
- 키워드
- Analog computing
- 키워드
- Deep learning
- 기타저자
- Princeton University Electrical and Computer Engineering
- 기본자료저록
- Dissertations Abstracts International. 85-05B.
- 기본자료저록
- Dissertation Abstract International
- 전자적 위치 및 접속
- 로그인 후 원문을 볼 수 있습니다.
MARC
008240612s2023 us |||||||||||||||c||eng d■001000016935355
■00520240214101922
■006m o d
■007cr#unu||||||||
■020 ▼a9798380850780
■035 ▼a(MiAaPQ)AAI30691094
■040 ▼aMiAaPQ▼cMiAaPQ
■0820 ▼a621.3
■1001 ▼aGrimm, Christopher L., Jr.
■24510▼aTraining Deep Neural Networks With In-Memory Computing▼h[electronic resource]
■260 ▼a[S.l.]:▼bPrinceton University. ▼c2023
■260 1▼aAnn Arbor :▼bProQuest Dissertations & Theses, ▼c2023
■300 ▼a1 online resource(140 p.)
■500 ▼aSource: Dissertations Abstracts International, Volume: 85-05, Section: B.
■500 ▼aAdvisor: Verma, Naveen.
■5021 ▼aThesis (Ph.D.)--Princeton University, 2023.
■506 ▼aThis item must not be sold to any third party vendors.
■520 ▼aDeep learning has advanced machine capabilities in a variety of fields typically associated with human intelligence, including image recognition, object detection, natural language processing, healthcare, and competitive games. The models behind deep learning called Deep Neural Networks (DNNs) generally require billions of operations for inference, and a hundred if not thousand fold more operations for training. These operations are typically dominated by high dimensionality Matrix-Vector Multiplies (MVMs). Due to the dominance of MVMs, a number of hardware accelerators have been developed to enhance the compute efficiency of MVMs, but data movement and accessing typically remain key bottlenecks. In-Memory Computing (IMC) has the potential to overcome these key bottlenecks by performing computations in-place within dense 2-D memory. However, IMC has challenges of its own, as it fundamentally trades efficiency and throughput gains for dynamic range. This tradeoff is especially challenging for training where higher dynamic range is typically involved in the form of higher compute precision requirements and greater noise sensitivity compared to inferencing. In this work, we will discuss how to enable deep learning training with IMC by mitigating these challenges while substantially mapping operations to IMC. Key advancements in this work include: (1) leveraging new high-precision IMC techniques, like charge-based and analog-input IMC to reduce noise sources; (2) mapping aggressive quantization techniques such as radix-4 gradients to IMC; and (3) ADC range adjustments to better capture output distributions and mitigate biases during training. These methods enable training of DNNs on IMC capable of over 400 x energy savings and of achieving high levels of testing accuracy on a variety of DNN models.
■590 ▼aSchool code: 0181.
■650 4▼aComputer engineering.
■650 4▼aElectrical engineering.
■653 ▼aAnalog computing
■653 ▼aDeep learning
■653 ▼aIn-Memory Computing
■653 ▼aSignal processing
■653 ▼aDeep Neural Networks
■690 ▼a0800
■690 ▼a0464
■690 ▼a0544
■71020▼aPrinceton University▼bElectrical and Computer Engineering.
■7730 ▼tDissertations Abstracts International▼g85-05B.
■773 ▼tDissertation Abstract International
■790 ▼a0181
■791 ▼aPh.D.
■792 ▼a2023
■793 ▼aEnglish
■85640▼uhttp://www.riss.kr/pdu/ddodLink.do?id=T16935355▼nKERIS▼z이 자료의 원문은 한국교육학술정보원에서 제공합니다.
■980 ▼a202402▼f2024
