(CS 286a) Performance Modeling

Announcements

• I have a few trips coming up, so we will need to adjust the lectures... Wed Oct. 8 will be a shorter lecture than normal. There won't be a lecture Mon Oct 13. And, I'm hoping to move the lecture on Wed Oct 15 to Fri Oct 17 if possible.
• HW 2 is out, and it's due Oct 17. Start early, it is longer than the 1st HW.
• HW 1 is due Oct 6.

Time/Location

NEW TIME: Monday and Wednesday, 10:30-11:50am, 306 Thomas

Instructor

Adam Wierman, 258 Jorg
Office Hours: It's a small class so feel free to stop by whenever, send me an email first if you want to make sure I'm around.

Course Description

When designing a new computer architecture, network protocol, distributed system, etc. it is essential to be able to quantify the performance impacts of design choices along the way. For example, should we invest in more buffer space or a faster processor? One fast disk or multiple slower disks? How should requests be scheduled? What migration policy will work best? Ideally, one would like to make these choices before investing the time and money to build a system. This class will teach students how to answer this type of "what if" question by introducing students to analytic performance modeling, the tools necessary for rigorous system design.

The course will focus on the mathematical tools, so it should be interesting and accessing both for Computer Science students and for students in Electrical Engineering, Applied Mathematics, Economics, and Operations Research.

The course will be packed with open research questions that are both interesting mathematically and important to computer system design today.

The topics covered will include:

• Probability Theory: stochastic ordering, modes of convergence, renewal theory
• Markov Chains: discrete & continuous time Markov chains, time reversibility, matrix geometric techniques, stochastic decomposition
• Scheduling Theory: FCFS, PS, SRPT, FB, and many more
• Queueing Theory: product form networks, conservation laws, heavy-traffic analysis, large deviations analysis
• Heavy-tailed distributions: where they come from and why they matter
• Operational Laws: Little's Law, response time law, modification analysis
• Applications and case studies: Web servers, call centers, server farms, TCP modeling, Google PageRank, Social network modeling

This course is targeted for graduate students and advanced undergraduates. Students should have a strong background in probability.

Textbook

There is no required textbook. I will pass out course notes and supplementary handouts at the end of each class. Some good reference texts are listed here.

Grading

This breakdown will likely change during the course.
In particular, we may have only 1 exam.

• Class participation and meeting with me outside of class -- 15%
• Homeworks -- 55%
• Midterm -- 15%
• Final -- 15%

Lectures

Since this is the first year of the course and I'm making all the lectures from scratch, I likely won't be able to latex up the notes in time and will instead be putting up my handwritten notes. I'll try to keep them legible!

• Lecture 1: The Power of Performance Modeling
  
• Lecture 2a: A first stochastic process
  
• Lecture 2b: DTMCs, our first hammer
  
• Lecture 3: DTMCs, the details
  
• Lecture 4: The celebrated (and demonized) Poisson Process
  
• Lecture 5: From DTMCs to CTMCs
  
• Lecture 6: Baby Queueing
  
• Lecture 7: TBA
  
• Lecture 8: TBA
  
• Lecture 9: TBA
  
• Lecture 10: TBA
  
• Lecture 11: TBA
  
• Lecture 12: TBA
  
• Lecture 13: TBA
  
• Lecture 14: TBA
  
• Lecture 15: TBA
  
• Lecture 16: TBA
  
• Lecture 17: TBA
  

Assignments

Homeworks will be assigned every 1-2 weeks. Some of the problems will be challenging, so please start early. Most assignments will be primarily proofs, however there will be an occasional problem that requires you to build a simulator and/or do numeric experiments. I assume that you can code and use Matlab/Mathematica.

• Homework 1: A probability refresher (out 9/29, due 10/6)
  
• Homework 2: Practice with DTMCs (out 10/6, due 10/17)
  
• Homework 3: TBA
• Homework 4: TBA
• Homework 5: TBA
• Homework 6: TBA
• Homework 7: TBA
• Homework 8: TBA

Collaboration vs. cheating

You will receive one homework every week or so. These will typically require a significant amount of work, and you should start immediately and come to office hours to discuss questions. Please do not search the web for help on the homework problems. It is difficult to develop good homework problems, and thus you may come across similar problems if you search the web for help.

You are strongly encouraged to collaborate with your classmates on these problems, but each person must write up the final solutions individually. You should note on your homework specifically which problems were a collaborative effort and with whom.

Course topics

This outline is overly ambitious and will likely guide us through 2 terms not just one!

• Introduction, overview, and background
  • Motivating examples of the power of stochastic modeling and queueing theory
  • Probability refresher
  • Discrete time Markov Chains
  • Continuous time Markov Chains & Time reversibility
• Baby Queueing Theory
  • Little's Law and other operational laws
  • M/M/1, PASTA, M/M/m, M/M/m/m, M/M/infinity, and applications
  • Queueing networks, Jackson networks, BCMP, etc.
• Beyond the Exponential distribution
  • Workload characterization overview
  • Matrix geometric techniques
  • Heavy-tailed distributions
  • Lindley's equation
  • M/GI/1 analysis
• Beyond First Come First Served -- both worst case and stochastic results
  • Preemptive Last Come First Served
  • Processor Sharing
  • Conservation Laws
  • Priority queues
  • Foreground Background scheduling
  • Scheduling beyond the single server
• Advanced Analysis techniques
  • Large deviations analysis
  • Heavy-traffic analysis
  • Rate conservation laws
  • Supplementary random variables
  • Stochastic complements
  • Stochastic stability

Relevant reference books and online notes

Online lecture notes

• CMU, Performance Modeling, Mor Harchol-Balter
• CUHK, Computer System Performance Evaluation, John C.S. Lui
• Columbia, Stochastic Models I, Ward Whitt
• TuE, Queueing Theory, Ivo Adan
• Columbia, Performance Modeling and Evaluation, Ed Coffman

Books on stochastic processes and queueing

• S. M. Ross, Introduction to Probability Models
• S. M. Ross, Stochastic Processes
• R. W. Wolff, Stochastic Modeling and the Theory of Queues
• L. Kleinrock, Queueing Systems, Vol. I: Theory
• L. Kleinrock, Queueing Systems, Vol. II: Computer Applications
• S. Karlin and H. M. Taylor, A First Course in Stochastic Processes
• S. I. Resnick, Adventures in Stochastic Processes
• W. Whitt, Stochastic Process Limits

Books on Scheduling

• R. Conway, W. Maxwell, and L. Miller, Theory of Scheduling
• Michael Pinedo, Scheduling Theory, Algorithms, and Systems

Books on Performance Modeling

• H. Kobayashi and B. L. Mark, System Modeling and Analysis
• E. Lazowska, J. Zahorjan, S. Graham, K. Sevcik, Quantitative System Performance
• D. Menasce, V. Almeida, and L. Dowdy, Capacity Planning and Performance Modeling
• R. Jain, The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling

Books on Probability and Statistics

• D. P. Bertsekas and J. N. Tsitsiklis, Introduction to Probability
• M. Mitzenmacher and Upfal, Probability and Computing
• K.L. Chung, A Course in Probability Theory
• W. Feller, An Introduction to Probability Theory and Its Applications
• P. Billingsley, Probability and Measure