{"componentChunkName":"component---src-templates-post-template-js","path":"/posts/operatingsystems/caching","result":{"data":{"markdownRemark":{"id":"95f28f6d-a3c9-598d-9fac-3f0ff50bd8d5","html":"

Examples of Caches

\n
    \n
  1. TLBs: Caches recent results of address translation from page tables.
    2. \n
  2. Virtually addressed caches: Stores memory value associated with a virtual address
    3. \n
  3. Physically addressed caches: This is a part of the virtually addressed cache. Each entry stores the memory value associated with a physical memory location.
    4. \n
  4. DNS caches, Web content caches, web search, email clients, incremental compilation, just in time translation, virtual memory, file systems, conditional branch prediction all use caches.
    5. \n
\n

Design Challenges of Caches

\n\n

Naive Cache Implementation

\n\n

Memory Hierarchy + When to use Caches

\n\n

Working Sets

\n\n

Zipf Model

\n\n

Cache Design

\n\n

Replacement Policies

\n

Motivation: If we look up an address and find that it is a miss, how do we choose which memory block to replace?

\n

Random

\n\n

First in First Out(FIFO)

\n\n

Optimal Cache Replacement (MIN)

\n\n

Least Recently Used (LRU)

\n\n

Least Frequently Used(LFU)

\n\n

Belady’s Anomaly

\n

Intuitively, adding space to a cache should always improve or maintain the cache hit rate. However, Belady’s Anomaly says this isn’t the case.

\n

For some replacement policies like FIFO, adding more space will drastically change the contents of the cache, thus making the hit rate worse.

\n

Memory Mapped Files

\n

Demand Paging: Applications can access more memory than is physically present on the machine by using memory pages as a cache for disk blocks.\nMemory Mapped File: The program directly accesses and modifies the contents of the file, treating memory as a write-back cache for disk.

\n

Pros

\n\n

Implementation

\n

OS provides a system call where the kernel initializes a set of page table entries for that region of virtual address space, setting each entry to invalid.

\n

When the process gives an instruction that touches an invalid mapped address:

\n
    \n
  1. The TLB miss occurs, triggering a full page table lookup in hardware.
    2. \n
  2. The hardware finds the page table entry and finds that it is invalid, causing a hardware page fault into the OS kernel.
    3. \n
  3. The OS kernel converts the address to a file offset.
    4. \n
  4. The OS kernel allocates an empty page frame and reads the required file block into the empty page frame.
    5. \n
  5. The disk interrupts the processor when the file block has been read and the scheduler resumes handling the page fault exception.
    6. \n
  6. The kernel updates the page table entry to point to the page frame allocated for the block and sets the entry to valid.
    7. \n
  7. The OS resumes execution of the process.
    8. \n
  8. The TLB misses, trigger a full page table lookup.
    9. \n
  9. The hardware returns the page frame to the processor, loads the TLB with the new translation, evicts the previous TLB entry, then uses the translation to construct a physical address.
    10. \n
\n

To create the empty page frame that holds the incoming page from disk, the OS:

\n
    \n
  1. Selects a page to evict using the current replacement policy.
    2. \n
  2. Finds page table entries that point to the evicted page, usually through a core map.
    3. \n
  3. Set each page table entry to invalid, to prevent other processes from using the evicted page while it is being evicted.
    4. \n
  4. Copy back any changes to the evicted page.
    5. \n
\n

Approximating LRU

\n

Keeping a linked list to track when pages were last used in hardware is prohibitively expensive, thus, we will use the clock algorithm as an approximation instead.

\n

Clock algorithm

\n

The OS scans through the core map of physical memory pages. 2. For each page frame, it records the value of the use bit and clears the use bit. 3. The OS does a shootdown for each page frame that has its use bit cleared.

\n

A common usage of the collected values of the use bit is the not recently used/kth chance algorithm where the kernel picks a page that has not been set for the last k sweeps of the clock algorithm.

\n

Virtual Memory

\n

By backing every memory segment with a file on disk, we create virtual memory.

\n

Self Paging

\n

Problem: There is a malicious program that tries to grab as much memory as possible by using multiple threads to cycle through memory and add those pages.\nSolution: Self paging assigns every process its fair share of page frames. As the system starts to page, it evicts the page from whichever process has the most pages allocated to it.\nCons: If there are two processes, one with a working set of 2/3 of memory and the other with 1/3 of memory, the ideal split is for the first process to get 2/3 of memory and the other to get 1/3 of memory. Unfortunately, self-paging will results in both process splitting the memory, causing the first process to thrash

\n

Swapping

\n

Problem: Sometimes, there is simply too much data in all of the process’ working sets to effectively run them at once. If we try to run things normally, we will end up thrashing with every process.\nSolution: We can evict an entire process from memory (aka swapping)\nCons: The process that just got evicted from memory will be sad :(

\n

Future

\n
    \n
  1. Low latency backing store: Things that were once on hard disk are moving onto SSDs, things that are on SSDs will move onto even faster storage devices.
    2. \n
  2. Variable page sizes: We will move toward smaller effective page sizes due to low latency solid state storage and cluster memory.
    3. \n
  3. Memory aware applications: Applications can be tuned to how much memory is in the first level cache to increase performance.
    4. \n
\n

Questions

\n
    \n
  1. How does set associative caches work?
    2. \n
  2. How does virtual memory work?
    3. \n
","fields":{"slug":"/posts/operatingsystems/caching","tagSlugs":["/tag/notes/","/tag/textbook/","/tag/operating-system/"]},"frontmatter":{"date":"2021-10-22T23:46:37.121Z","description":"Notes on caching for operating systems","tags":["Notes","Textbook","Operating System"],"title":"Caching"}}},"pageContext":{"slug":"/posts/operatingsystems/caching"}},"staticQueryHashes":["251939775","401334301","825871152"]}