CS 5214 (Fall 2022)

Modeling and Evaluation of Computer Systems


- Class Instructor
- Catalog Description
- Topics to be Covered
- Reference Textbook
- Grading Policies
- Class Schedule
- Important Dates
- FAQ
- Homeworks and Tests
- Lecture Slides
- Software

Class Instructor

Dr. Ing-Ray Chen, Professor of Computer Science Department

Catalog Description

5214: An overview of modeling, simulation and performance evaluation of computer systems, i.e., operating systems, database management systems, office automation systems, etc. Fundamentals of modeling, the life cycle of a simulation study, workload characterization, random number and variate generation, procurment, measurement principles, software and hardware monitors, capacity planning, system and program tuning, and analytic modeling.

Why do you need to take this course?

This course will cover two state-of-the-art performance and reliability modeling and analysis tools, namely, SHARPE (Symbolic Hierarchical Automated Reliability & Performance Evaluator) and SPNP (Stochastic Petri Nets Package), both developed by Duke University. SHARPE has its own language, while SPNP language is a subset of C. Users can use these tools to define a system and ask these tools to calculate system performance/reliability measures. This course will also cover discrete-event simulation. Students can use the tools and techniques learned in this course to evaluate computer system designs, e.g., control and scheduling algorithms, network communication protocols, distributed algorithms, reliability and dependability of cyber physical systems, network and service management of mobile wireless systems, trust and security management of IoT systems, admission and quality of service control algorithms in multimedia systems, replicated data control algorithms in databases, etc. This course is ideal for graduate students to equip themselves with mathematical modeling and analysis techniques for their research projects. Case studies will be drawn from distributed databases, mobile systems, and networked multimedia systems.

Topics to be Covered (Tentative)

Discrete event simulation (1 week)
Combinatorial reliability/availability modeling (1/2 week)
Fault trees, reliability graphs, network reliability models (1 week)
Reliability and availability modeling using SHARPE (1/2 week)
Markov models for performance/reliability/availability evaluation (2 weeks)
Single queueing systems (1 week)
Product-form queueing networks (1 week)
Stochastic Petri nets modeling using SPNP (1 week)
Hierarchical modeling (1/2 week)
Performability modeling (1/2 week)
Case studies and paper discussion (5 weeks)

Reference Textbook

There is no textbook but I will use the following reference text to cover SHARPE: R.A. Sahner, K.S. Trivedi and A. Puliafito, Performance and Reliability Analysis of Computer Systems: An Example-Based Approach Using the SHARPE Software Package, Springer, 1996, ISBN: 9781461523673, Price: $79.95.

Grading

  • 40% homework (3)
  • 25% exam 1
  • 25% exam 2
  • 10% participation and paper presentation


    Class Schedule

    Date HW/Test Source Subject
    8/25 hw1 S* Discrete event simulation
    9/1   2 Reliability and availability models
    9/8   9 Fault trees, reliability graphs, network reliability models using SHARPE
    9/15 hw1 due; hw2 S* Queueing theory: single queueing systems
    9/22   4,9 Markov models
    9/22   4,6,9,12 Markov reward models and performability analysis using SHARPE
    9/29   S*,5,10 Case study 1, queueing network models using SHARPE
    10/6 hw2 due 11 Hierarchical modeling
    10/13 exam 1   4:00 - 6:45 p.m. in class
    10/20     Conference travel (no class)
    10/27 hw3 7 Stochastic Petri net modeling using SPNP
    11/3   7,S* Stochastic Petri net modeling using SPNP, case study 2, case study 3
    11/10     Conference travel (no class)
    11/17 hw3 due [P1] [P2] Paper presentation
    11/17   [P3] [P4] Paper presentation
    11/24     Thanksgiving break (no class)
    12/1 exam 2   4:00 - 6:45 p.m. in class

    S* - supplement materials (slides)
    SHARPE - Symbolic Hierarchical Automated Reliability & Performance Evaluator.
    SPNP - Stochastic Petri Net Package
    [Pn] - paper presentation material, to be determined in class.

    Presentation Schedule and List of Papers

  • [P1] (11/11 - Presented by Lamba and Lin) (presentation slides)
    R. Mitchell, and I.R. Chen,
    "On Survivability of Mobile Cyber Physical Systems with Intrusion Detection,"
    Wireless Personal Communications,
    vol. 68, no. 4, 2013, pp. 1377-1391.

  • [P2] (11/11 - Presented by Zhang, Yu, and Shahriar) (presentation slides)
    D.C. Wang, I.R. Chen, and H. Al-Hamadi,
    "Reliability of Autonomous IoT Systems with Intrusion Detection Attack-Defense Game Design,"
    IEEE Transactions on Reliability,
    vol. 70, no. 1, 2021, pp. 188-199.

  • [P3] (11/18 - Presented by Chen and Wang) (presentation slides)
    R. Mitchell, and I.R. Chen,
    "Modeling and Analysis of Attacks and Counter Defense Mechanisms for Cyber Physical Systems,"
    IEEE Transactions on Reliability,
    vol. 65, no. 1, March 2016, pp. 350-358.

  • [P4] (11/18 - Presented by Wan and Liu) (presentation slides)
    G. Choudhary, P.V. Astillo, I. You, K. Yim, I.R. Chen, and J.H. Cho,
    "Lightweight Misbehavior Detection Management of Embedded IoT Devices in Medical Cyber Physical Systems,"
    IEEE Transactions on Network and Service Management,
    vol. 17, no. 4, Dec 2020, pp. 2496-2510.

    Presentation Policy

  • Each presentation should last for about 1 hour and 15 minutes, including 15 minutes for questioning/answering.
  • Each group of presenters should prepare a powerpoint file for the paper to be presented and send it to the instructor for posting on the class web site one day prior to the presentation date. (Click to see example powerpoint slides.)
  • Every student is required to attend the presentation session. Class participation will be counted toward the presentation grade.
  • The presentation will be graded based on the following criteria: organization of presentation (20%), material understanding (30%), clearness of presentation (10%), and whether the presentation covers the main idea (40%).


    Important Dates

    HW #1 due 9/15
    HW #2 due 10/6
    HW #3 due 11/17
    Exam #1 10/13
    Exam #2 12/1

    Frequently Asked Questions and Answers

  • HW #1 (ascii format)
  • HW #2 (ascii format)