CS 6704/5914 - Special Topics in Software Engineering
Software Testing and Debugging in Emerging Domains
Course Information
Instructor: Muhammad Ali Gulzar
Office: 4106 Gilbert Place (220 Gilbert Street)
Lecture : McBryde Hall 210, 11:00 AM - 12:15 PM TR
Office Hours: TBD
Optional Textbook: Introduction to Software Testing (2nd Edition) — Paul Ammann and Jeff Offutt
Course Description
The type of software we write today has dramatically changed in the last decade, from sequential Java applications to data-intensive applications integrated with ML/AI components. The prevalence of these emerging applications was mainly because of platforms such as Apache Spark, TensorFlow, and SparkML, which have almost removed the development and deployment barrier. However, frameworks once used by domain experts are now being leveraged by data scientists, business analysts, and researchers. This shift in user demographics calls for immediate advancements in developing, debugging, and testing practices for emerging applications.
This class will discuss several aspects of these emerging applications and the corresponding advances made by the software testing and debugging research community. We will start by building a foundation in classical and fundamental software program analysis, testing, and debugging, and then transition to learning methodologies for applying these techniques to emerging applications. Software engineering and program analysis is rapidly evolving; therefore, this course is designed around seminal research papers that introduced state-of-the-art tooling to the community. By the end of this course, students will be able to:
- Understand the theory behind fundamental program analysis
- Apply and build static and dynamic analysis tools
- Use classical and advanced testing and debugging methods
- Match testing and debugging techniques to specific software contexts
- Understand emerging application domains and their unique analysis needs
- Be familiar with current research in the above areas
Course Prerequisites
This course requires familiarity with core computer science concepts, including programming languages, data structures, algorithms, operating systems, and undergraduate-level software engineering. Students are expected to have completed:
- Programming Languages: CS 3304, CS 3314, or CS 4303
- Software Engineering: CS 3704, CS 3724, or CS 4704
Students should be able to review and critically analyze research papers, complete in-class presentations and demos, and work on a semester-long project. Late submissions will not be accepted, and missed presentations/demos will receive a zero.
Course Schedule
| Date | Topic | Description | Reading |
|---|---|---|---|
| Aug 26 | Introduction + Testing Basics | Terminology and Program Analysis | |
| Aug 28 | Testing Fundamentals | Test Adequacy, Coverage, Efficacy | |
| Sept 2 | Traditional Testing | Unit Test Generation | Pacheco et al |
| Sept 4 | Automated Testing | Symbolic Execution | JavaPath Finder |
| Sept 9 | Automated Testing | Mutation-based Testing | FairFuzz |
| Sept 11 | Automated Testing | Grammar-based Fuzzing | Zest |
| Sept 16 | New Testing Techs | Metamorphic | Segura et al |
| Sept 18 | New Testing Techs | Differential | Yang et al |
| Sept 23 | Test Evolution | Regression | Rothermal et al |
| Sept 25 | Testing in Emerging Domains | Big Data | BigTest |
| Sept 30 | Testing in Emerging Domains | Database Testing | CERT |
| Oct 2 | Testing in Emerging Domains | ML/AI | DeepXplore |
| Oct 7 | Testing in Emerging Domains | LLM-Based Testing | Fuzz4All |
| Oct 9 | Midterm Exam | ||
| Oct 14 | Traditional Debugging | Delta Debugging | Delta Debugging |
| Oct 16 | Traditional Debugging | Spectra-based Fault Localization | Tarantula |
| Oct 21 | Traditional Debugging | Program Slicing | Head et al |
| Oct 23 | Traditional Debugging | Tracing | Penumbra |
| Oct 28 | Interactive Debugging | Whyline | Ko et al |
| Oct 30 | Interactive Debugging | Feedback Debugging | Lin et al |
| Nov 4 | Automated Debugging | Advanced DD | HDD |
| Nov 6 | Automated Debugging | Slicing/Tracing/Tainting | Carbin et al |
| Nov 11 | Automated Debugging | Debugging via ML | DeepFL |
| Nov 13 | Debugging in Emerging Domains | Data | Titian |
| Nov 18 | Debugging in Emerging Domains | BigData | BigSift |
| Nov 20 | Debugging in Emerging Domains | LLM-Based Fault Localization | FixAgent |
| Nov 25 | Thanksgiving Break | ||
| Nov 27 | Thanksgiving Break | ||
| Dec 2 | Debugging in Emerging Domains | AI | Lamp |
| Dec 4 | Project Presentations | ||
| Dec 9 | Project Presentations | ||
Grading Policy
- 50% — Course Project: Semester-long with a full conference-style paper report.
- 15% — Midterm Exam.
- 15% — In-class Presentation.
- 15% — Tool Demo.
- 05% — Quizzes.
This is a project-intensive course. The final project grade will be assessed according to standard reviewing practices in tier-1 SE workshops, e.g., MSR, ICPC, and CAIN.
Academic Integrity
The Virginia Tech Honor Code applies to all graded work. For more information: honorsystem.vt.edu
Mental Health and Accommodations
Virginia Tech welcomes students with disabilities and supports mental health and well-being. Contact the Services for Students with Disabilities (SSD) at 540-231-3788 or ssd@vt.edu.
For well-being resources, visit: well-being.vt.edu