CS 5214 (Spring 2009)

Modeling and Evaluation of Computer Systems


- Class Instructor
- Catalog Description
- Topics to be Covered
- Textbook
- Grading Policies
- Class Schedule
- Important Dates
- FAQ
- Homeworks and Tests
- Lecture Slides
- Software

Class Instructor


Dr. Ing-Ray Chen , Professor of Computer Science Department

Catalog Description


5214: An overview of modeling, simulation and performance evaluation of computer systems, i.e., operating systems, database management systems, office automation systems, etc. Fundamentals of modeling, the life cycle of a simulation study, workload characterization, random number and variate generation, procurment, measurement principles, software and hardware monitors, capacity planning, system and program tuning, and analytic modeling.

Why do you need to take this course?

This course will cover two state-of-the-art performance and reliability modeling and analysis tools, namely, SHARPE (Symbolic Hierarchical Automated Reliability & Performance Evaluator) and SPNP (Stochastic Petri Nets Package), both developed by Duke University. SHARPE has its own language, while SPNP language is a subset of C. Users can use these tools to define a system and ask these tools to calculate system performance/reliability measures. This course will also cover discrete-event simulation. Students can use the tools and techniques learned in this course to evaluate computer system designs, e.g., control and scheduling algorithms, network communication protocols, distributed algorithms, location management algorithms in mobile wireless systems, admission and quality of service control algorithms in multimedia systems, fault-tolerant designs, replicated data control algorithms in databases, etc. This course is ideal for graduate students to equip themselves with mathematical modeling and analysis techniques for their research projects. Case studies will be drawn from distributed databases, mobile systems, and networked multimedia systems.

Topics to be Covered (Tentative)


Discrete event simulation (1 week)
Combinatorial reliability/availability modeling (1 week)
Fault trees, reliability graphs, network reliability model (1 week)
Reliability and availability modeling using SHARPE (1 week)
Dependency modeling (1 week)
Markov models for performance/reliability/availability evaluation (2 weeks)
Single queueing systems (1 week)
Product-form queueing networks (1 week)
Stochastic Petri nets modeling using SPNP (2 weeks)
Hierarchical modeling methods (1 week)
Performability modeling (1 week)
Case studies and paper discussion (2 weeks)

Textbook


R.A. Sahner, K.S. Trivedi and A. Puliafito, Performance and Reliability Analysis of Computer Systems: An Example-Based Approach Using the Sharpe Software Package, Springer, 1996, ISBN: 0-7923-9650-2, Price: $79.95.

Grading


  • 40% homework (4)
  • 25% exam 1
  • 25% exam 2
  • 10% participation and paper presentation



    Class Schedule


    Date HW/Test Source Subject
    1/27   S*,2,9 Simulation modeling, reliability and availability modeling using SHARPE
    2/3 hw1 2,9 Fault trees, reliability graphs, network reliability models
    2/10   S* Queueing theory: single queueing systems
    2/17 hw1 due, hw2 4,9,6,12 Markov reward models, performability
    2/24   S*,5,10 Case study 1, queueing network models
    3/3 hw2 due, hw3 5,10,11 Queueing network models, hierarchical modeling
    3/13 hw3 due by email   Spring Break (no class)
    3/17 exam 1   4:00 - 6:45 p.m. in class
    3/24 hw4 7 Stochastic Petri net modeling using SPNP
    3/31   S* Case studies 2 & 3
    4/7 hw4 due [P1] [P2] Paper presentation
    4/14   [P3] [P4] Paper presentation
    4/21   [P5] [P6] Paper presentation
    5/5 exam 2 4:00 - 6:45 p.m. in class

    S* - supplement materials (slides)
    SHARPE - Symbolic Hierarchical Automated Reliability & Performance Evaluator.
    SPNP - Stochastic Petri Net Package
    P* - paper presentation material, to be determined in class.

    Presentation Schedule and List of Papers

  • [P1]
    (4/7/08 - presented by Bilodeau and Reed)
    (Powerpoint Slide)
    W. He and I.R. Chen,
    "DMAP-FR: Integrated Mobility and Service Management with Failure Recovery Support for Mobile IPv6 Systems,"
    6th IEEE International Conference on Information Technology: New Generation,
    Las Vegas, April 2009.

  • [P2]
    (4/7/08 - presented by Feng)
    (Powerpoint Slide)
    J.H. Cho and I.R. Chen,
    "Performance Analysis of Distributed Intrusion Detection Protocols for Mobile Group Communication Systems,"
    The 5th International Workshop on Security in Systems and Networks,
    Rome, May 2009.

  • [P3]
    (4/14/08 - presented by Li and Miao)
    (Powerpoint Slide)
    S. George and I.R. Chen,
    ``Movement-based checkpointing and logging for failure recovery of database applications in mobile environments,''
    Distributed and Parallel Databases,
    2008.

  • [P4]
    (4/14/08 - presented by Gujarati and Motupalli)
    (Powerpoint Slide)
    J.H. Cho, I.R. Chen, and M. Eltoweissy,
    ``On optimal batch rekeying for secure group communications in wireless networks,''
    ACM/Springer Wireless Networks,
    2008.

  • [P5]
    (4/21/08 - presented by Elgammal and Sunkara)
    (Powerpoint Slide)
    I.R. Chen and D.C. Wang,
    ``Modeling and Analysis of Regional Registration Based Mobile Multicast Service Management,''
    3rd IEEE International Workshop on Performance Modelling and Analysis of Communication in Wired and Wireless Networks,
    Hsinchu, Taiwan, Dec. 2007.

  • [P6]
    (4/21/08 - presented by Chang and Peng)
    (Powerpoint Slide)
    O. Yilmaz and I.R. Chen,
    "On QoS guarantees with reward optimization for servicing multiple priority classes in wireless networks,"
    IEEE 17th International Conference on Computer Communications and Networks,
    St. Thomas, US Virgin Islands, August 2008.

    Presentation Policy

  • Each presentation should last for about 1 hour and 15 minutes, including 15 minutes for questioning/answering.
  • Each group of presenters should prepare a powerpoint file for the paper to be presented and send it to the instructor for posting on the class web site one day prior to the presentation date.
  • Every student is required to attend the presentation session. Class participation will be counted toward the presentation grade.
  • The presentation will be graded based on the following criteria: organization of presentation (20%), material understanding (30%), clearness of presentation (10%), and whether the presentation covers the main idea(40%).


    Important Dates


  • HW #1 due 2/17
  • HW #2 due 3/3
  • HW #3 due 3/13 (midnight by email)
  • Exam #1 3/17
  • HW #4 due 4/7
  • Exam #2 5/5


    Frequently Asked Questions and Answers


  • Lecture
  • Homework #1
  • Homework #2
  • Homework #3
  • Test #1
  • Homework #4
  • Test #2


    Homeworks and Tests


  • Homework #1
  • Solution to Homework #1
  • Homework #2
  • Solution to Homework #2
  • Sample Questions for Test #1
  • Solution to Sample Questions for Test #1
  • Homework #3
  • Solution to Homework #3
  • Solution to Test #1
  • Homework #4
  • Solution to Homework #4
  • Sample Questions for Test #2

    Late policy: a late homework turned in will be penalized 10% off for each day late. After a week late, the late homework will be 100% off.

    Lecture Slides



  • Lecture 1 (pages 1-23: Simulation) (.pdf format)
  • Lecture 2 (pages 24-43: Reliability and Availability Models) (.pdf format)
  • Lecture 3 (pages 44-63: Fault Trees, Reliability Graphs, and Network Reliability Models) (.pdf format)
  • Lecture 4 (pages 64-87: Single Queueing Systems) (.pdf format)
  • Lecture 5 (pages 88-106: Markov Models) (.pdf format)
  • Lecture 6 (pages 107-118: Markov Reward Models and Performability) (.pdf format)
  • Lecture 7 (pages 119-128: Markov Reward Models and Case Study #1) (.pdf format)
  • Lecture 8 (pages 129-149: Queueing Network Models) (.pdf format)
  • Lecture 9 (pages 150-160: Hierarchical Models) (.pdf format)
  • Lecture 10 (pages 161-185: Stochastic Petri Net Models) (.pdf format)
  • Lecture 11 (pages 186-205: Case Study #2) (.pdf format)
  • Lecture 12 (pages 206-237: Case Study #3) (.pdf format)

    Software


    smpl
    Source: M.H. MacDougall, Simulating Computer Systems, Computer Systems Series, The MIT Press, 1987.

    How to compile and execute an smpl program in UNIX?
    First set up your .cshrc file: copy .cshrc to your home directory and issue the command 'source .cshrc' to activate it. Then issue the following command to compile your program 'my-program.c':
            gcc -g3 -I. my-program.c smpl.c rand.c bmeans.c stat.c -lm -o my-program
    For example:
            gcc -g3 -I. mm1.c smpl.c rand.c bmeans.c stat.c -lm -o mm1
    You can also make a shell script (executable) called "ccc" with the content of:
            gcc -g3 -I. $* smpl.c rand.c bmeans.c stat.c -lm -o `basename $1 .c`
    Then you can compile your program, say, mm1.c, as follows:
            ccc mm1.c
    Then you will find an executable file 'mm1' in your default directory. Execute it by simply typing:
            mm1
    Below is a set of smpl source code to upload to your default directory:
  • smpl.h (plain ascii)
  • smpl.c (plain ascii)
  • rand.c (plain ascii)
  • bmeans.c (plain ascii)
  • stat.c (plain ascii)
  • mm1.c (plain ascii)
  • Output of mm1.c (plain ascii)
  • mm1_bmeans.c (plain ascii)
  • Output of mm1_bmeans.c (plain ascii)

    Java Meets Simulation (Developed by MITRE)

    Java Meets Simulation API is developed by MITRE; it provides both process-orient and discrete-event simulation capabilities using Java. Copy all files listed below to your Java source directory and set the CLASSPATH environment variable to find sim.jar and aspectjrt.jar, i.e., in UNIX put a "setenv CLASSPATH .:sim.jar:aspectjrt.jar" command in your login shell script. Then use javac to compile source programs and java to execute the class file in the same directory. View the API documentation file for its usage.

  • sim.jar (JAR file)
  • aspectjrt.jar (JAR file)
  • javadoc.jar (Document JAR file)
  • MyEntity.java (Example Java Module)
  • MySimulation.java (Example Java Module)
  • AdHocSim-July2004.zip (Example Ad Hoc Simulation Code)


    TimeNET

    TimeNET is installed at /pkg/TimeNET. Its user documentation is available under /pkg/TimeNET/doc. TimeNET 3.0 can be downloaded free from ttp://pdv.cs.tu-berlin.de/~timenet/.



    SHARPE

    Sharpe is installed at /usr/local/sharpe. The textbook is the best source for its documentation. A user manual is available under /usr/local/sharpe/doc. A PC version is available at /usr/local/sharpe/pc.



    SPNP

  • SPNP Reference Manual (.pdf format)
  • SPNP User Manual (.pdf format)
  • SPNP Program Example #1 (.ascii text)
  • SPNP Program Example #2 (.ascii text)
  • SPNP Program Example #3 (.ascii text)
  • SPNP Program Example #4 (.ascii text)
  • SPNP Program Example #6 (.ascii text)
  • SPNP Program Example #7 (.ascii text)
  • SPNP Program Example #8 (.ascii text)
  • SPNP Program Example #9 (.ascii text)
  • SPNP Program Example #10 (.ascii text)
  • SPNP Program Example #11 (.ascii text)


    How to run SPNP on your UNIX account?

  • insert a line as follows in your .cshrc file:
            alias spnp "make -f /usr/local/spnp/v4obj/Makerun DIR=/usr/local/spnp PLATFORM=v4obj SPN=\!^"
  • compile and run an SPNP program, say, "hw.c" (with .c extension), by typing in:
            spnp hw