Lecture 7 Outline

  1. Previously
    • Enforced modularity on a single machine via virtualization.
      • Virtual memory, bounded buffers, threads.
    • Saw monolithic vs. microkernels.
    • Talked about VMs as a means to run multiple instances of an OS on a single machine with enforced modularity (bug in one OS won't crash the others).
      • Big thing to solve was how to implement the VMM. Solution: Trap and emulate. How the emulation works depends on the situation.
        • Another key problem: How to trap instructions that don't generate interrupts.
  2. What's left? Performance
    • Performance requirements significantly influence a system's design.
    • Today: General techniques for improving performance.
  3. Technique 1: Buy New Hardware
    • Why? Moore's law => processing power doubles every 1.5 years, DRAM density increase over time, disk price (per GB) decreases, ...
    • But:
      • Not all aspects improve at the same pace.
      • Moore's Law is plateauing.
      • Hardware improvements don't always keep pace with load increases.
    • Conclusion: Need to design for performance, potentially re-design as load increases.
  4. General Approach
    • Measure the system and find the bottleneck (the portion that limits performance).
    • Relax (improve) the bottleneck.
  5. Measurement
    • To measure, need metrics:
      • Throughput: Number of requests over a unit of time.
      • Latency: Amount of time for a single request.
      • Relationship between these changes depending on the context.
      • As system becomes heavily-loaded:
        • Latency and throughput start low. Throughput increases as users enter, latency stays flat...
        • ..until system is at maximum throughput. Then throughput plateaus, latency increases.
      • For heavily-loaded systems: Focus on improving throughput.
    • Need to compare measured throughput to possible throughput: Utilization.
    • Utilization sometimes makes bottleneck obvious (CPU is 100% utilized vs. disk is 20% utilized), sometimes not (CPU and disk are 50% utilized, and at alternating times).
    • Helpful to have a model in place: What do we expect from each component?
    • When bottleneck is not obvious, use measurements to locate candidates for bottlenecks, fix them, see what happens (iterate).
  6. How to Relax the Bottleneck
    • Better algorithms, etc. These are application-specific. 6.033 focuses on generally-applicable techniques.
    • Batching, caching, concurrency, scheduling.
    • Examples of these techniques follow. The examples related to operating systems (that's what you know), but techniques apply to all systems.
  7. Disk Throughput
    • How does an HDD (magnetic disk) work?
      • Several platters on a rotating axle.
      • Platters have circular tracks on either side, divided into sectors.
        • Cylinder: Group of aligned tracks.
      • Disk arm has one head for each surface, all move together.
      • Each disk head reads/writes sectors as they rotate past. Size of a sector = unit of read/write operation (typically 512B).
      • To read/write:
        • Seek arm to desired track.
        • Wait for platter to rotate the desired sector under the head.
        • Read/write as the platter rotates.
    • What about SSDs?
      • Organized into cells, each of which hold one (or 2, or 3) bits.
      • Cells organized into pages; pages into blocks.
      • Reads happen at page-level. Writes also at page-level, but to new pages (no overwrites of pages).
      • Erases (and thus overwrites) are at block-level.
        • Takes a high voltage to erase.
    • How long does R/W take on HDD?
      • Example disk specs:
        • Capacity: 400GB
        • Platters: 5
        • # heads: 10
        • # sectors per track: 567–1170 (inner to outer)
        • # bytes per sector: 512
        • Rotational speed: 7200 RPM => 8.3ms per revolution
      • Seek time: Avg read seek 8.2ms, avg write seek 9.2ms.
        • Given as part of disk specs
      • Rotation time: 0–8.3ms.
        • Platters only rotate in one direction.
      • R/W as platter rotates: 35–62MB/sec.
        • Also given in disk specs.
      • So reading random 4KB block: 8.2ms + 4.1ms + ~.1ms = 12.4
      • 4096 B / 12.4 ms = 322KB/s.
        => 99% of the time is spent moving the disk.
    • Can we do better?
      • Use flash? For this particular random-access of reads, yes; SSDs would help if available.
      • Batch individual transfers?
        • .8ms to seek to next track + 8.3ms to read entire track = 9.1ms.
          • .8ms is single-track seek time for our disk (again, from specs).
        • 1 track contains ~1000sectors * 512B = 512KB.
        • Throughput: 512KB/9.1ms = 55MB/s.
    • Lesson: Avoid random access. Try to do long sequential reads.
      • But how?
        • If your system reads/writes entire big files, lay them out contiguously on disk. Hard to achieve in practice!
        • If your system reads lots of small pieces of data, group them.
  8. Caching
    • Already saw in DNS. Common performance-enhancement for systems.
    • How do we measure how well it works?
      • Average access time: Hit_time * hit_rate + miss_time * miss_rate.
    • Want high hit rate. How do we know what to put in the cache?
      • Can't keep everything.
      • So really: How do we know what to *evict* from the cache?
    • Popular eviction policy: Least-recently used.
      • Evict data that was used the least recently.
      • Works well for popular data.
      • Bad for sequential access (think: Sequentially accessing a dataset that is larger than the cache).
    • Caching is good when:
      • All data fits in the cache.
      • There is locality, temporal or spatial.
    • Caching is bad for:
      • Writes (writes have to go to cache and disk; cache needs to be consistent, but disk is non-volatile).
    • Moral: To build a good cache, need to understand access patterns
      • Like disk performance: To relax disk as bottleneck, needed to understand details of how it works
  9. Concurrency/Scheduling
    • Suppose server alternates between CPU and disk: 
       CPU: --A--     --B--     --C--
 Disk:     --A--     --B--     --C--
    • Apply concurrency, can get: 
       CPU: --A----B----C-- ...
 Disk:     --A----B-- ..
    • This is a scheduling problem: Different orders of execution can lead to different performance.
    • Example:
      • 5 concurrent threads issue concurrent reads to sectors 71, 10, 92, 45, and 29.
      • Naive algorithm: Seek to each sector in turn.
      • Better algorithm: Sort by track and perform reads in order. Gets even higher throughput as load increases.
        • Drawback: It's unfair.
    • No one right answer to scheduling. Tradeoff between performance and fairness.
  10. Parallelism
    • Goal: Have multiple disks, want to access them in parallel.
    • Problem: How do we divide data across the disks?
    • Depends on bottleneck:
      • Case 1: Many requests for many small files. Limited by disk seeks. Put each file on a single disk, and allow multiple disks to seek multiple records in parallel.
      • Case 2: Few large reads. Limited by sequential throughput. Stripe files across disks.
    • Another case: Parallelism across many computers.
      • Problem: How do we deal with machine failures?
      • (One) Solution: Go to recitation tomorrow!
  11. Summary
    • We can't magically apply any of the previous techniques. Have to understand what goes on underneath.
      • Batching: How disk access works.
      • Caching: What is the access pattern?
      • Scheduling/concurrency: How disk access works, how system is being used (the workload).
      • Parallelism: What is the workload?
    • Techniques apply to multiple types of hardware.
      • E.g., caching is useful regardless of whether you have HDD or SSD.
  12. Useful numbers for your day-to-day-lives:
    • Latency:
      • 0.00000001ms: Instruction time (1 ns)
      • 0.0001ms: DRAM load (100 ns)
      • 0.1ms: LAN network
      • 10ms: Random disk I/O
      • 25–50ms: Internet east -> west coast
    • Throughput:
      • 10,000 MB/s: DRAM
      • 1,000 MB/s: LAN (or100 MB/s)
      • 100 MB/s: Sequential disk (or 500 MB/s)
      • 1 MB/s: Random disk I/O