F9 IN Minnesystemer og cache. Yngve Hafting,

Transkript

1 F9 IN Minnesystemer og cache Yngve Hafting,

2 Kort om emnet Formål Emnet tar for seg prinsipper i digital design, som kombinatorisk og sekvensiell logikk, tilstandsmaskiner og digitale byggeblokker, og bygger på dette for å introdusere prosessorarkitekturer, pipelining, cache, og grensesnittet mellom maskinvare og programkode. Hva lærer du? Etter å ha tatt IN26 har du: kunnskaper om hvordan en datamaskin er satt sammen og fungerer, fra logiske porter til prosessor kunnskaper om grensesnittet mellom maskinvare og programvare lært å kunne analysere og konstruere digitale kretser Delmål Kunne forstå hvordan en cache virker prinsipielt- hvordan struktur og størrelse og plassering av en cache påvirker ytelse hvordan (små) endringer i maskinkode kan gi forskjeller i kjøretid Kunne bruke til terminologi knyttet til minne og cachesystemer Cache typer og oppbygning Hit, miss, konflikter,.. Kunne forstå hvordan man typisk organiserer virtuelt minne Adresseoversetting Page table Hvordan? Forelesning + selvstudium: Minnestørrelse / ytelse Minnehierarkier Amdahls lov Cache Oppbygning, varianter Hit/miss rate (Eksempler med maskinkode) Hvordan type/størrelse påvirker ytelse Virtuelt minne Virtuelle og fysiske adresser Oppbygning» Page table» TLB Ukeoppgaver Øvelse i å tolke og forstå betydningen av oppbygning av fysisk og virtuell minnearkitektur Gruppetimer Gjennomgang av øvelser Translation lookaside buffer, TLB 3

3 Oversikt Bakgrunn Utfordringer for minnearkitektur Ytelse for minnetyper Lokalitet Hit/ miss rate Cache Virtuelt minne 4

4 Processor-Memory Gap In prior chapters, assumed memory access takes 1 clock cycle but hasn t been true since the 198 s Chapter 8 <5>

5 Memory System Challenge Make memory system appear as fast as processor Use hierarchy of memories Ideal memory: Fast Cheap (inexpensive) Large (capacity) But can only choose two! Chapter 8 <6>

6 Memory Hierarchy Technology Price / GB Access Time (ns) Bandwidth (GB/s) Cache SRAM $1, Speed Main Memory DRAM $ SSD $1 1,.5 Virtual Memory HDD $.1 1,,.1 Capacity Chapter 8 <7>

7 Locality Exploit locality to make memory accesses fast Temporal Locality: Locality in time If data used recently, likely to use it again soon How to exploit: keep recently accessed data in higher levels of memory hierarchy Spatial Locality: Locality in space If data used recently, likely to use nearby data soon How to exploit: when access data, bring nearby data into higher levels of memory hierarchy too Chapter 8 <8>

8 Memory Performance Hit: data found in that level of memory hierarchy Miss: data not found (must go to next level) Hit Rate = Miss Rate (MR) = # hits # memory accesses = 1 Miss Rate # misses = 1 Hit Rate # memory accesses Average memory access time (AMAT): average time for processor to access data AMAT = t cache + MR cache [t MM + MR MM (t VM )] MM = main memory, VM = virtual memory Chapter 8 <9>

9 Memory Performance Example 1 A program has 2, loads and stores 1,25 of these data values in cache Rest supplied by other levels of memory hierarchy What are the hit and miss rates for the cache? Hit Rate = 125/2 =.625 Miss Rate = 75/2 =.375 = 1 Hit Rate Chapter 8 <1>

10 Memory Performance Example 2 Suppose processor has 2 levels of hierarchy: cache and main memory t cache = 1 cycle, t MM = 1 cycles What is the AMAT of the program from Example 1? AMAT = t cache + MR cache (t MM ) = [ (1)] cycles = 38.5 cycles Chapter 8 <11>

11 Gene Amdahl, Amdahl s Law: the effort spent increasing the performance of a subsystem is wasted unless the subsystem affects a large percentage of overall performance S Latency 1 1 p S Latency is the theoretical maximal speedup p is proportion of the part than can be sped up Chapter 8 <12>

12 Oversikt Bakgrunn Cache Hva er en cache Terminologi Hvordan virker ulike cache-typer Direct-mapped N-way set associative Fully associative Ulike blokkstørrelser Cache miss typer Miss rate vs cache typer Flernivå cache Virtuelt minne 13

13 Cache Cache, caching, or caché may refer to: Hoard or cache, a collection of artifacts A hoard or "wealth deposit" [1] is an archaeological term for a collection of valuable objects or artifacts, sometimes purposely buried in the ground, in which case it is sometimes also known as a cache. Highest level in memory hierarchy Fast (typically ~ 1 cycle access time) Ideally supplies most data to processor Usually holds most recently accessed data Chapter 8 <14>

14 Cache Design Questions What data is held in the cache? How is data found? What data is replaced? Focus on data loads, but stores follow same principles Chapter 8 <15>

15 What data is held in the cache? Ideally, cache anticipates needed data and puts it in cache But impossible to predict future Use past to predict future temporal and spatial locality: Temporal locality: copy newly accessed data into cache Spatial locality: copy neighboring data into cache too Chapter 8 <16>

16 Cache Terminology Capacity (C): number of data bytes in cache Block size (b): bytes of data brought into cache at once Number of blocks (B = C/b): number of blocks in cache: B = C/b Degree of associativity (N): number of blocks in a set Number of sets (S = B/N): each memory address maps to exactly one cache set Number of sets (S) Number of Blocks (B=S N) N-way set associative Cache Capacity C= B b Block size in bytes (b) byte block Chapter 8 <17>

17 How is data found? Cache organized into S sets Each memory address maps to exactly one set Caches categorized by # of blocks in a set: Direct mapped: 1 block per set N-way set associative: N blocks per set Fully associative: all cache blocks in 1 set Examine each organization for a cache with: Capacity (C = 8 words) Block size (b = 1 word) So, number of blocks (B = 8) Number of sets (S) Number of Blocks (B=S) Direct mapped Cache Capacity C= B b Block size in bytes (b) byte block Chapter 8 <18>

18 Example Cache Parameters Direct mapped ex: C = 8 words (capacity) b = 1 word (block size) So, B = 8 (# of blocks) S = B = 8 Ridiculously small, but will illustrate organizations Address mem[xff...fc] mem[xff...f8] mem[xff...f4] mem[xff...f] mem[xff...ec] mem[xff...e8] mem[xff...e4] mem[xff...e] mem[x...24] mem[x..2] mem[x..1c] mem[x...18] mem[x...14] mem[x...1] mem[x...c] mem[x...8] mem[x...4] mem[x...] Set Number 7 (111) 6 (11) 5 (11) 4 (1) 3 (11) 2 (1) 1 (1) () 2 3 Word Main Memory 2 3 Word Cache Chapter 8 <19>

19 Direct Mapped Cache Hardware Memory Address Tag Set 27 3 Byte Offset V Tag Data 8-entry x ( )-bit SRAM = Hit Data Chapter 8 <2>

20 Direct Mapped Cache Performance Memory Address Tag... ARM Assembly Code MOV R, #5 MOV R1, # LOOP CMP R, # BEQ DONE LDR R2, [R1, #4] LDR R3, [R1, #12] LDR R4, [R1, #8] DONE SUB R, R, #1 B LOOP Set 1 3 Byte Offset V Tag Data 1... mem[x...c] 1... mem[x...8] 1... mem[x...4] Miss Rate =? Set 7 (111) Set 6 (11) Set 5 (11) Set 4 (1) Set 3 (11) Set 2 (1) Set 1 (1) Set () Chapter 8 <21>

21 Direct Mapped Cache Performance Memory Address Tag... ARM Assembly Code MOV R, #5 MOV R1, # LOOP CMP R, # BEQ DONE LDR R2, [R1, #4] LDR R3, [R1, #12] LDR R4, [R1, #8] DONE SUB R, R, #1 B LOOP Set 1 3 Byte Offset V Tag Data 1... mem[x...c] 1... mem[x...8] 1... mem[x...4] Miss Rate = 3/15 = 2% Temporal Locality Compulsory Misses Set 7 (111) Set 6 (11) Set 5 (11) Set 4 (1) Set 3 (11) Set 2 (1) Set 1 (1) Set () Chapter 8 <22>

22 Direct Mapped Cache: Conflict Memory Address ARM Assembly Code MOV R, #5 MOV R1, # LOOP CMP R, # DONE BEQ DONE Tag...1 LDR R2, [R1, #x4] LDR R3, [R1, #x24] SUB R, R, #1 B LOOP Set 1 3 Byte Offset V 1 Tag... Data mem[x...4] mem[x...24] Miss Rate = 1/1 = 1% Set 7 (111) Set 6 (11) Set 5 (11) Set 4 (1) Set 3 (11) Set 2 (1) Set 1 (1) Set () Chapter 8 <23>

23 1 N-Way Set Associative Cache Memory Address Tag Set 28 2 Byte Offset V Tag Way 1 Way Data V Tag Data = = Hit 1 Hit 32 Hit 1 Hit Data Chapter 8 <24>

24 N-Way Set Associative Performance ARM Assembly Code MOV R, #5 MOV R1, # LOOP CMP R, DONE BEQ DONE LDR R2, [R1, #x4] LDR R3, [R1, #x24] SUB R, R, #1 B LOOP V Tag Way 1 Way Data V Miss Rate =? Tag Data Set 3 Set 2 Set 1 Set Chapter 8 <25>

25 N-Way Set Associative Performance ARM Assembly Code MOV R, #5 MOV R1, # LOOP CMP R, DONE BEQ DONE LDR R2, [R1, #x4] LDR R3, [R1, #x24] SUB R, R, #1 B LOOP V Tag Way 1 Way Data Miss Rate = 2/1 V Tag Data mem[x...24] 1... mem[x...4] = 2% Associativity reduces conflict misses Set 3 Set 2 Set 1 Set Chapter 8 <26>

26 Fully Associative Cache V Tag Data V Tag Data V Tag Data V Tag Data V Tag Data V Tag Data V Tag Data V Tag Data Reduces conflict misses Expensive to build Chapter 8 <27>

27 Spatial Locality? Increase block size: Block size, b = 4 words C = 8 words Direct mapped (1 block per set) Number of blocks, B = 2 (C/b = 8/4 = 2) Memory Address Block Byte Tag Set Offset Offset 27 2 V Tag 27 Data Set 1 Set = 32 Hit Data Chapter 8 <28>

28 Cache with Larger Block Size Memory Address Block Byte Tag Set Offset Offset 27 2 V Tag 27 Data Set 1 Set = 32 Hit Data Chapter 8 <29>

29 Direct Mapped Cache Performance ARM assembly code MOV R, #5 MOV R1, # LOOP CMP R, DONE BEQ DONE LDR R2, [R1, #4] LDR R3, [R1, #12] LDR R4, [R1, #8] SUB R, R, #1 B LOOP Memory Address Tag Block Byte Set Offset Offset Miss Rate = 1/15 Larger blocks V Tag 1... mem[x...c] 27 = 6.67% reduce compulsory misses through spatial locality Data mem[x...8] mem[x...4] mem[x...] Set 1 Set = 32 Hit Data Chapter 8 <3>

30 Cache Organization Recap Capacity: C Block size: b Number of blocks in cache: B = C/b Number of blocks in a set: N Number of sets: S = B/N Organization Number of Ways (N) Direct Mapped 1 B Number of Sets (S = B/N) N-Way Set Associative 1 < N < B B / N Fully Associative B 1 Chapter 8 <31>

31 Capacity Misses Cache is too small to hold all data of interest at once If cache full: program accesses data X & evicts data Y Capacity miss when access Y again How to choose Y to minimize chance of needing it again? Least recently used (LRU) replacement: the least recently used block in a set evicted Chapter 8 <32>

32 LRU Replacement ARM Assembly Code MOV R, # LDR R1, [R, #4] LDR R2, [R, #x24] LDR R3, [R, #x54] (a) V U Tag Way 1 Way Data V Tag mem[x...24] 1... Way 1 Way Data mem[x...4] Set 3 (11) Set 2 (1) Set 1 (1) Set () (b) V U Tag Data V Tag mem[x...24] Data mem[x 4] mem[x...54] Set 3 (11) Set 2 (1) Set 1 (1) Set () Chapter 8 <33>

33 Types of Misses Compulsory: first time data accessed Capacity: cache too small to hold all data of interest Conflict: data of interest maps to same location in cache Miss penalty: time it takes to retrieve a block from lower level of hierarchy Chapter 8 <34>

34 Cache Summary What data is held in the cache? Recently used data (temporal locality) Nearby data (spatial locality) How is data found? Set is determined by address of data Word within block also determined by address In associative caches, data could be in one of several ways What data is replaced? Least-recently used way in the set Chapter 8 <35>

35 Miss Rate Trends Bigger caches reduce capacity misses Greater associativity reduces conflict misses Adapted from Patterson & Hennessy, Computer Architecture: A Quantitative Approach, 211 Chapter 8 <36>

36 Miss Rate Trends Bigger blocks reduce compulsory misses Bigger blocks increase conflict misses Chapter 8 <37>

37 Multilevel Caches Larger caches have lower miss rates, longer access times Expand memory hierarchy to multiple levels of caches Level 1: small and fast (e.g. 16 KB, 1 cycle) Level 2: larger and slower (e.g. 256 KB, 2-6 cycles) Most modern PCs have L1, L2, and L3 cache Chapter 8 <38>

38 Intel Pentium III Die Chapter 8 <39>

39 Oversikt Bakgrunn Cache Virtuelt mine Hva er virtuelt mine Terminologi Virtuelle og fysiske addresser Oversetting av addresser Sidetabell TLB: sidetabellcache Minnebeskyttelse 4

40 Virtual Memory Gives the illusion of bigger memory Main memory (DRAM) acts as cache for hard disk Chapter 8 <41>

41 Memory Hierarchy Technology Price / GB Access Time (ns) Bandwidth (GB/s) Cache SRAM $1, Speed Main Memory DRAM $ SSD $1 1,.5 Virtual Memory HDD $.1 1,,.1 Capacity Physical Memory: DRAM (Main Memory) Virtual Memory: Hard drive Slow, Large, Cheap Chapter 8 <42>

42 Hard Disk Magnetic Disks Read/Write Head Takes milliseconds to seek correct location on disk Chapter 8 <43>

43 Virtual Memory Virtual addresses Programs use virtual addresses Entire virtual address space stored on a hard drive Subset of virtual address data in DRAM CPU translates virtual addresses into physical addresses (DRAM addresses) Data not in DRAM fetched from hard drive Memory Protection Each program has own virtual to physical mapping Two programs can use same virtual address for different data Programs don t need to be aware others are running One program (or virus) can t corrupt memory used by another Chapter 8 <44>

44 Cache/Virtual Memory Analogues Cache Block Block Size Block Offset Miss Tag Virtual Memory Page Page Size Page Offset Page Fault Virtual Page Number Physical memory acts as cache for virtual memory Chapter 8 <45>

45 Virtual Memory Definitions Page size: amount of memory transferred from hard disk to DRAM at once Address translation: determining physical address from virtual address Page table: lookup table used to translate virtual addresses to physical addresses Chapter 8 <46>

46 Virtual & Physical Addresses Most accesses hit in physical memory But programs have the large capacity of virtual memory Chapter 8 <47>

47 Address Translation Chapter 8 <48>

48 Virtual Memory Example System: Virtual memory size: 2 GB = 2 31 bytes Physical memory size: 128 MB = 2 27 bytes Page size: 4 KB = 2 12 bytes Chapter 8 <49>

49 Virtual Memory Example System: Virtual memory size: 2 GB = 2 31 bytes Physical memory size: 128 MB = 2 27 bytes Page size: 4 KB = 2 12 bytes Organization: Virtual address: 31 bits Physical address: 27 bits Page offset: 12 bits # Virtual pages = 2 31 /2 12 = 2 19 (VPN = 19 bits) # Physical pages = 2 27 /2 12 = 2 15 (PPN = 15 bits) Chapter 8 <5>

50 Virtual Memory Example 19-bit virtual page numbers 15-bit physical page numbers Chapter 8 <51>

51 Virtual Memory Example What is the physical address of virtual address x247c? VPN = x2 VPN x2 maps to PPN x7fff 12-bit page offset: x47c Physical address = x7fff47c Chapter 8 <52>

52 How to perform translation? Page table Entry for each virtual page Entry fields: Valid bit: 1 if page in physical memory Physical page number: where the page is located Chapter 8 <53>

53 Page Table Example Virtual Address Virtual Page Number x2 19 Page Offset 47C 12 VPN is index into page table V 1 x 1 x7ffe 1 x1 1 x7fff Hit Physical Address Physical Page Number x7fff Page Table C Chapter 8 <54>

54 Page Table Example 1 What is the physical address of virtual address x5f2? V Physical Page Number 1 x 1 x7ffe 1 x1 1 x7fff Page Table Chapter 8 <55>

55 Page Table Example 1 What is the physical address of virtual address x5f2? VPN = 5 Entry 5 in page table VPN 5 => physical page 1 Physical address: x1f2 Virtual Address Virtual Page Number x5 19 Page Offset F2 12 V 1 x 1 x7ffe 1 x1 1 x7fff Hit Physical Address Physical Page Number Page Table x1 F2 Chapter 8 <56>

56 Page Table Example 2 What is the physical address of virtual address x73e? Virtual Address Virtual Page Number x7 19 Page Offset 3E V Physical Page Number 1 x 1 x7ffe 1 x1 1 x7fff Page Table Hit 15 Chapter 8 <57>

57 Page Table Example 2 What is the physical address of virtual address x73e? VPN = 7 Entry 7 is invalid Virtual page must be paged into physical memory from disk Virtual Address Virtual Page Number x7 19 Page Offset 3E V Physical Page Number 1 x 1 x7ffe 1 x1 1 x7fff Page Table Hit 15 Chapter 8 <58>

58 Page Table Challenges Page table is large usually located in physical memory Load/store requires 2 main memory accesses: one for translation (page table read) one to access data (after translation) Cuts memory performance in half Unless we get clever Chapter 8 <59>

59 Translation Lookaside Buffer (TLB) Small cache of most recent translations Reduces # of memory accesses for most loads/stores from 2 to 1 Chapter 8 <6>

60 TLB Page table accesses: high temporal locality Large page size, so consecutive loads/stores likely to access same page TLB Small: accessed in < 1 cycle Typically entries Fully associative > 99 % hit rates typical Reduces # of memory accesses for most loads/stores from 2 to 1 Chapter 8 <61>

61 1 Example 2-Entry TLB Virtual Address Virtual Page Number x2 19 Page Offset 47C 12 Entry 1 Entry V Virtual Page Number Physical Page Number 1 x7fffd x 1 x2 x7fff V Virtual Page Number Physical Page Number TLB = = Hit 1 Hit Hit 1 Hit Physical Address 15 x7fff 47C 12 Chapter 8 <62>

62 Memory Protection Multiple processes (programs) run at once How to make sure they don t overwrite each other s data? Each process has its own page table Each process can use entire virtual address space A process can only access physical pages mapped in its own page table Chapter 8 <63>

63 Virtual Memory Summary Virtual memory increases capacity A subset of virtual pages in physical memory Page table maps virtual pages to physical pages address translation A TLB speeds up address translation Different page tables for different programs provides memory protection Chapter 8 <64>

64

65 Anbefalte oppgaver Oppgaver 8.2, 8.5, 8.6, 8.7, 8.8, 8.9, 8.14, 8.15, 8.16, 8.18, 8.2, 8.22, 8.24,