Course Title: CS 6814: Knowledge-guided Machine Learning (3 credits, CRN: 91145)
Instructor: Anuj Karpatne
Class Type: Face-to-face Instruction
Class Timings: TR: 11:00 am - 12:15 pm Eastern
Classroom Location: Data and Decision Sciences (D&DS) Building 240
Instructor Office Hours and Location: TR: 12:30 pm - 1:30 pm; D&DS 438

Course Overview

While the impact of machine learning (ML) in commercial disciplines involving vision, speech, and text related problems is well understood, the promise of ML is yet to be fully realized for accelerating discoveries in scientific and engineering disciplines. This is because mainstream “black-box” ML models, that only rely on data, are susceptible to learning spurious relationships that do not generalize well outside the data they are trained upon. Moreover, black-box ML models do not provide any mechanistic insights about the scientific processes being studied, thus making them unfit to be used as building blocks in scientific discovery. What is fundamentally lacking in black-box ML is their inability to ingest the rich background of scientific knowledge driving real-world phenomena along with the information contained in data. To address this, there is a growing research trend to deeply integrate scientific knowledge in the ML process, referred to as the paradigm of scientific Knowledge-guided ML (KGML). This course will introduce the foundations of KGML and provide a coherent perspective of research themes in KGML. These research themes will be illustrated using recent examples of cutting-edge research from diverse scientific disciplines. The course will also impart hands-on experience in conducting KGML research through a semester-long project.

Learning Aims

By the end of the course, students will:

  • Be well-versed with the foundations and theme areas of KGML, as well as recent developments in every theme area
  • Be able to compare and contrast different KGML research themes and identify their strengths, limitations, and opportunities for future research
  • Be equipped to cross-pollinate KGML ideas from one application domain to another
  • Develop essential research skills including reading, discussing, and critiquing research papers, identifying research gaps and brainstorming solutions, and communicating research ideas through technical writing and oral presentations
  • Gain practical experience in pursuing KGML reseach through a course project

Course Topics

This course will tentatively cover the following list of research themes in KGML:

  1. Knowledge-guided Learning: Techniques for modifying the learning algorithms of ML, e.g., with the help of priors, constraints, and loss functions to ensure that the learned ML solutions are scientifically consistent.
  2. Knowledge-guided Architecture: Techniques for hard-coding (or "baking in") scientific knowledge in the design of ML models, e.g., using neural network architectures that capture the physics of the scientific process.
  3. Knowledge-guided Refinement: Techniques for refining the outputs of ML models using scientific knowledge, e.g., by pruning or post-processing.
  4. Discovery of Scientific Laws from Data: Techniques for automatically discovering the governing equations of a scientific problem from simulated or real-world data using ML.
  5. Inferring parameters in science-based models: Techniques for inferring parameters or state-variables in science-based forward models using ML-based inversion techniques.
  6. Hybrid-science-ML Modeling: Techniques for integrating ML models with science-based models to augment systematic biases or replace sub-components of science-based models that are currently lacking.

Learning Activities

This is a project-based course that will use a mix of learning activities. Introductory lectures will cover the basics of ML, foundations of KGML, and research themes in KGML. The course will also include occassional guest lectures by leading researchers in KGML covering special topics of interest. The lectures will be interspersed with paper discussions led by students from a reading list of relevant literature in KGML. Students will also submit reviews of the paper they read after every discussion session. A major component of the course will be a semester-long project where students will get to work on a research problem in KGML of their interest at the intersection of ML and science from scratch to finish. Students will get to work in groups to identify and formulate a research problem, apply, explore, and design KGML algorithms to solve the problem, and demonstrate the real-world effectiveness of KGML procedures using rigorous evaluation setups, potentially leading to publications. All project activities starting from idea generation to report preparation will be facilitated through peer discussions coordinated by the instructor.

Background Required

This advanced topics course does not require any formal pre-requisite courses and is broadly open to students with the interest and ability to learn topics in KGML. Specifically, this course is meant for two categories of graduate students: (a) students familiar in ML who are eager and willing to learn about scientific problems and pursue KGML research, and (b) students from scientific disciplines with little familiarity in ML who are eager to learn and apply KGML in an area they are familiar with. Students can assess their preparedneess for the course by discussing with the instructor and attending the first class.