CS 6804: World Models

Fall 2026

Department of Computer Science, Virginia Tech

Location: 1084 Derring Hall, 926 West Campus Drive, Blacksburg, VA.
Meeting time: Tuesdays and Thursdays, 12:30 PM - 1:45 PM
Instructor: Chris Thomas
E-mail: Please message me through the messaging system on Canvas rather than sending e-mail.
Office: 378 Data and Decision Sciences Building
Office hours: 2:00 - 3:00 PM Tuesdays. My Zoom is linked here.
Exam section: 12T. December 11, 2026, 10:05 AM - 12:05 PM. There are no exams in this course and the final project presentation and report serve as the comparable evaluation in place of a final examination. We may use this time for final presentations (virtually) if necessary.

Course overview

Course description: World models are AI systems that learn to simulate environments, predicting how the world evolves over time and in response to actions. Long studied in model-based reinforcement learning, world models have become a central focus of machine learning research as video generation models begin to serve as approximate simulators of the physical world and embodied agents learn to play within them. This graduate seminar examines the foundations of world modeling alongside the current research frontier. Topics include latent dynamics models and model-based reinforcement learning; video generation as world simulation; interactive and controllable world generation; world models for robotics and autonomous driving; memory, spatial consistency, and long-horizon prediction; physical reasoning and evaluation; and the robustness, security, and safety of world models. Students will read and critique current research papers, lead presentations and discussions, and complete a semester-long research project.

Prerequisites: A background in deep learning is strongly recommended. Prior machine learning coursework would also be highly beneficial. Concerned students should speak to the instructor as soon as possible to ensure they have sufficient background to successfully complete this course. You aren’t expected to be familiar with every technique in every paper covered in this course, but you should be able to understand the main ideas of each paper discussed and propose (and possibly implement for your class project) extensions or improvements to these papers. If you are uncomfortable with your ability to do this, speak to the instructor as soon as possible.

Format: This is a seminar-style graduate course covering recent research on world models. The majority of the class time will be spent on listening to paper presentations by other students, followed by group discussions. Before each class, you will read and write a review of the primary paper presented. You will also work in a group on a final project which will be presented at the end of the semester to the class.

Canvas: Grades, class schedule, announcements, and other materials for this course will be released and updated on Canvas. You are strongly encouraged to sign up for Canvas e-mail and/or push notifications so that you are notified of new discussions and course updates posted to Canvas.

Course learning objectives:

  • Learning about state of the art methods in world models

  • Learning to think critically about research, which applies beyond this class or subject area. This involves developing the ability to critically assess research papers you encounter and to understand how different works are connected.

  • Learning how to conceive novel ideas and extensions of existing research methods and implement your ideas

  • Learning how to clearly write up and present research

  • Learning how to work and collaborate with others in a research group

Topical outline: Example topics to be covered in this course include latent dynamics models, model-based reinforcement learning, video generation as world simulation, interactive and controllable world generation, world models for robotics and autonomous driving, memory and spatial consistency, long-horizon prediction, physical reasoning, benchmarks and evaluation, and the robustness, security, and safety of world models. Students will be asked to vote on these topics (and to possibly contribute new ones that interest them) on the first day of the course. The schedule for the remainder of the course will be determined to ensure adequate topic coverage and to reflect student interest.

Requirements

Your final grade in the course will be based on your in-class participation and discussion, paper reviews, paper presentation, and final project, which will be weighted as follows:

Component Weight
Final project 45%
In-class participation and discussion 15%
Paper presentation 20%
Paper reviews 20%


The grade scale for the term will be:

Percentage 100 90 89 85 80 79 75 70 69 65 60 <60
Letter A A- B+ B B- C+ C C- D+ D D- F

Virginia Tech does not award A+ grades. Any component of the course may be curved at instructor discretion. No grades will be lowered as a result of a curve.

Attendance: Because this is a seminar-style class attendance is required. Attendance is taken through the in-class peer review of each presentation, which is only open during the class session. Missing more than two classes without a valid excuse will negatively affect your final grade. After your second absence, any further absence will only be excused with documentation from the Dean of Students Office. If you are sick, please do not come to class. Please think of others in the class before coming to class sick, and contact me so that your absence can be handled properly.

Participation: You are expected to regularly and meaningfully participate in this seminar class. Participation could include asking interesting questions or offering comments on a paper, as well as answering and responding to comments from other class members. Questions asked within the class about course topics are assumed to be addressed to the class to answer, rather than the instructor. Note that thoroughly reading the papers ahead of the class and doing a good job on your paper review will help prepare you to participate in class. Each presenter has spent a considerable amount of time preparing their presentation and studying the paper, so you should do your part to engage.

Paper reviews

The purpose of paper reviews is to prepare students to critically assess and review world models research. Students are required to write one paper review for each class session in which papers are presented. Students who are presenting that class session are not required to submit a paper review for that class. The paper review must be of one of the primary paper(s) presented each class, though students might refer to other papers in their review to compare and contrast the paper. On days when there are two presenters, you should still read the other primary paper in order to be able to participate in the discussion. To ensure both papers are roughly equally reviewed, students will be randomly assigned to review one or the other, and reviewing assignments will be posted on Canvas once the presentation schedule is set. On those days you must also submit two discussion questions about the paper you were not assigned to review, along with your review. Paper reviews should be 1 page single-spaced using 11-point Times New Roman and 1 inch margins. Paper reviews should follow the following guidelines (modified from the CVPR Reviewer Guidelines).

Tips for writing good reviews

  • In your review, look for what is good or stimulating in the paper, and what knowledge advancement it has made. Your review should highlight both the novelty and potential impact of the work. Above all, you should be specific and detailed in your reviews.

  • Take the time to write good reviews. Ideally, you should read a paper and then think about it over the course of several days before you write your review.

  • You should take enough time to write a thoughtful and detailed review. Bullet lists with one short sentence per bullet are NOT a detailed review.

  • Be specific about novelty. Claims in a review that the submitted work “has been done before” MUST be backed up with specific references and an explanation of how closely they are related. At the same time, for a positive review, be sure to summarize what novel aspects are most interesting in the Strengths section.

  • It is best to avoid the term “the authors” in your review because you are reviewing their work and not the person. Instead, try to refer to “the paper” or “the method”. Referring to the authors can be perceived as being confrontational when you write real reviews, even though you may not mean it this way.

  • Be generous about suggesting new ideas for how the paper could be improved. You might suggest a new technical tool that could help, a dataset that could be tried, an application area that might benefit from the method, or a way to generalize the idea to increase its impact.

Contents of paper reviews

Each paper review should consist of the following parts and should clearly indicate which is being addressed at that point in the review (e.g. Summary, Relation to prior work, Strengths, Weaknesses, Future work).

  • Summary: The paper review should first summarize the entire paper. This means first explaining what the paper is trying to do and how the paper proposes to do it. Your summary should focus on the primary novelty and contributions of the paper, rather than unimportant details. Your summary of the paper typically will include a summary of a new model architecture or loss function but might also involve describing key mathematical insights which undergird the paper. If the primary contribution of the paper is a dataset, the paper should describe details about what makes the dataset significant. Your summary should also summarize how the method is experimentally evaluated and any significant findings or results, whether quantitative or qualitative.

  • Relation to prior work: The review should next summarize the paper’s relation to prior work and why its contributions are (or are not) significant in your opinion. Reading the paper’s “Related Work” section and understanding how the paper differs from prior work will help you write this section.

  • Strengths: The review should mention at least three strengths of the approach. For example, you might explain how a particular technique or design is expected to solve problems with existing work. Simply rephrasing the strengths of the paper from its contribution section does not adequately address this point. Instead, rely on your own impression of the work and your own judgments about its novelty.

  • Weaknesses: What do you feel detracts from the paper’s contributions? Your review should mention at least three weaknesses. Some example weaknesses include cases where the method is likely to not perform well because of its design, computational costs, non-standard inference or training requirements, shortcomings in the proposed loss function or formulation, or a weak experimental evaluation.

  • Future work: Propose at least one possible extension of the paper. This might be a fix to a weakness you identified (e.g. a modified model or loss function) or you might propose how the techniques developed in the paper could be applied in some novel way for a different task. You should not, however, simply rephrase or repeat the future work suggested by the paper itself. Instead, think critically about how you might extend the paper as a researcher.

Grading policy for paper reviews

Paper reviews will be submitted through Canvas in PDF format and are due at 11:59 PM the day before the class in which the paper will be presented. The submission time reported by Canvas will be used to determine the time of submission. A submission that is one minute late is still late. Timely reviews start at 10 points. You will lose points for making false statements, vague or irrelevant claims which don’t indicate a deeper understanding of the paper, reviews which simply rephrase the paper’s own summary, claimed strengths, or weaknesses without providing your own insights (we want to hear what you think - not what the authors say!), or reviews which otherwise fail to address the above criteria. If your review is submitted by 10:00 AM the day of the paper presentation, your review will start at half credit. After 10:00 AM on the day of the class, you will not receive credit for your review. You will get three late days, meaning you can submit three reviews late (i.e. by 10:00 AM on the day of the class) without a penalty. These late days apply to paper reviews only.

Paper presentations

Each student will give one paper presentation during the course. Each class will be focused on a particular topic of interest and will contain one primary paper and possibly several background papers. Most days will have a single presenter and some days will have two, depending on final enrollment. Students will express their topic preferences after the first class via Canvas, and you will also be able to say whether you would rather present on your own or share a day with a second presenter, though due to the class size, there is no guarantee students will be matched to their desired topic or to their preferred kind of day.

Guidelines for paper presentations

Students should thoroughly read the assigned papers and other relevant background papers. This means fully understanding key equations, model design choices, etc. Your presentation, at a minimum, should:

  • Clearly define what problem the paper is addressing.

  • Provide motivation for why the problem is important, interesting, and/or challenging.

  • Address prior related work that has attempted to address this problem (or a related problem).

  • Describe, in detail, the proposed approach for the problem. For example, this may involve describing details of the model design and key loss functions used to train it. You should understand all equations that you present in class.

  • Explain how the paper is evaluated. You should fully describe the experimental set-up and present any quantitative and qualitative results. If there are any unusual metrics that students may not know, you should explain what those are and how they are computed.

  • Discuss key strengths and weaknesses of the paper.

  • Propose ideas for future work and identify any open research questions.

The non-primary paper(s) often provide background context to the primary paper. For example, the primary paper may build upon methods or results developed in these papers. In this case, you should clearly present this background work in such a way that students will be able to understand how the primary paper builds upon that prior work.

If you are the only presenter that day, your presentation should be 40 minutes long. If you are sharing the day with a second presenter, your presentation should be 25 minutes long. Good paper presentations are the result of extensive preparation and practice. You should practice your presentation many times before presenting it to the class to ensure you know what to say and to time yourself. Your presentation should be at most 2-3 minutes shorter than the time specified and certainly no longer than the time specified. In sum, your presentation is expected to be highly polished. This means you end on time, your presentation is well organized, and you explain the paper(s) presented clearly. After your presentation you will moderate a discussion session on your own paper. If you are presenting alone that discussion is about 25 minutes. If you are sharing the day, it is about 10 minutes and runs immediately after your own presentation, before the second presenter begins. You moderate your own discussion by yourself in either case. Part of what this class is meant to teach you is how to lead a research discussion and how to handle questions on your feet, which is why the discussion is never shared between presenters. The presenter is responsible for preparing possible topics for discussion and driving the discussion. The discussion could involve potential weaknesses in the approach, ideas for future work, thoughts on terminology used by the authors, the relationship of the paper to other literature, claims that the paper made that weren’t adequately justified, choices that you didn’t understand, etc.

You are strongly encouraged to use illustrations and graphics to explain concepts. Using animations, images, and videos (if applicable) is highly encouraged to make your presentation more engaging. Avoid slides which are just walls of text. Instead, you are encouraged to use short bullets (which you animate) and explain the rest verbally. You are also encouraged to animate equations and explain them piece by piece, which can help students better understand complex concepts. You are highly encouraged to search online for relevant materials which you may use in your presentation. However, make sure to clearly cite all your sources. You are free to use slides made by others, but be aware that your presentation requirements are different from the authors or others who may be describing the paper. Unlike authors presenting their work at a conference, your presentation should view the paper critically. This means identifying weaknesses and thinking about the significance of the work in its broader context. Finally, make sure to use your own words on slides and during your presentation. You should not be memorizing someone else’s words and presenting them as your own or copying text verbatim from the paper (or elsewhere) to slides.

Paper presentation slides should be uploaded to Canvas as a PDF by 11:59 PM on the day before the intended class presentation.

Peer review of presentations: After each presentation you will complete a short peer review of the presenters through Canvas. This is where you give your opinion on how well the presenter explained the paper, how well they answered questions, and how well they guided the discussion. Learning to present research and to answer questions on your feet is a large part of what this class is for, and honest feedback from the class is how presenters find out how they are doing. The peer review opens at the start of class and closes at the end of the class session, so it can only be completed while class is in session. Completing it is how attendance is recorded.

Presenting in the classroom: It is your responsibility to make sure you have any connectors you need, such as an HDMI cable or a display adapter for your laptop, so that you can present in the room on the day of your paper presentation or project presentation.

Grading policy for paper presentations

Your grade for your paper presentation will be based on: 1) clarity and presentation quality; 2) whether you covered the key points of the paper; 3) correctness of your statements made during the presentation; 4) whether you addressed all the guidelines above; 5) peer reviews by others in the class; 6) how well you facilitated the discussion of the paper; and 7) how well you delivered your presentation (i.e. was it clearly practiced, met time constraints, etc.).

Final project

This course will conclude with a student-driven group project, with a report due at the end of the course. In order to make meaningful progress on a project, groups must be 3-5 students. Note that larger groups will have higher expectations. Given that a significant portion of your final grade depends on the final project, each student is expected to contribute significantly to the final project. All projects must involve implementation of a world model, or a substantive component of one, along with a thorough evaluation. The goal is for your final project report to resemble a conference paper like those you have read throughout the class. Ideally, your group project will become a subsequent conference publication. The topic of your final project is open-ended and groups are free to choose a topic of their choosing. However, final projects should at least fall into one of the following broad categories:

  • Extend one of the papers we covered in class in a significant way, complete with a thorough experimental evaluation;

  • Propose a novel method or approach for solving a world modeling problem we discussed in class or that is already known in the literature and thoroughly evaluate it;

  • Propose a completely new world modeling problem and explain why it is significant and needs solving, implement an approach to solve the problem, and evaluate the approach

In summary, your final project can address any world modeling problem, either existing or new, as long as you propose a new method or significant extension or modification of existing methods. Applications of existing methods or techniques to new datasets or problems without at least some technical novelty are not sufficient for the final project. Taking existing methods, running them on something new, and writing up the results is not a semester-long project for a PhD level seminar. There has to be a technical contribution of your own. All projects must be thoroughly experimentally evaluated. This may involve benchmarking existing relevant work in the case of a new problem or applying your method on standard benchmarks and computing standard metrics. Projects that overlap in some way with your existing research are OK, but design, conception, implementation, evaluation, and delivery of the project should be the result of the students, not other faculty members, and should be specific to this course. However, your project can build upon or extend your prior research efforts without an issue.

Each student in the group should document everything they contributed to the project and how work was divided among group members. Students are required to provide a review of each other students’ contributions in their group as a form of peer review at the end of the course. This is required rather than optional, and students who do not submit it will not receive a grade for the final project. Please take this seriously and recognize that free-riders will be significantly penalized. Groups experiencing free-riding members should speak to the instructor as soon as possible if unable to resolve the issue internally. Failure of the group to reach out will result in the group being judged as if the free-riding member(s) were fully participating and may result in a lower grade.

Project proposal

The project proposal should be 3-5 pages long (excluding references) and must use the CVPR latex template. Your project proposal should include the following:

  • Project title

  • Group members

  • Group logistics (how will you communicate, how will the group regularly meet to discuss the project, who will handle each part, etc.)

  • A clear problem statement which describes the goal of the project.

  • A thorough literature review. Make sure you thoroughly search the literature before you start writing. You might find that your idea has already been taken. The literature review should resemble that in a CVPR conference paper and should cite existing work. It should clearly show how the proposed project does something the prior work you cite does not.

  • A detailed description of the proposed approach. The authors should describe new loss functions they plan to use, changes to existing models, etc.

  • Identify the computational resources that you plan to use. Do not propose a project that you are not confident you will have sufficient resources to execute.

  • The proposed experimental evaluation protocol and expected results. You should describe what experiments you plan to run and how you will run this. You should describe which datasets you plan to evaluate on, any existing code bases you will use, and what needs to be implemented by the group. You should also describe what your group is aiming for with the project (i.e. what do you consider a success). You should explain what you hope each experiment will show and discuss any uncertainty you have about the project. If you already have preliminary results, feel free to include them.

Project status report

The purpose of the project status report is to update me on your progress while also moving your project proposal document closer to the final report. The project status report should be about 3-5 pages (excluding references) and should describe the group’s progress on the project and any unforeseen blockers or challenges you are facing. The project status report should also use the CVPR latex template and should include Introduction, Related Work, Approach, and Results sections following the standard CVPR paper layout. You are free to reuse text from the proposal in the project status report (e.g. literature review). Please include any preliminary results, even if they aren’t good.

Project presentation

Each group will present their project to the class during the final several sessions of the class. The final project presentations should address the points listed in the guidelines for paper presentations above, but should be more descriptive of your project (since other students haven’t read your paper). The same guidelines for paper presentations apply to the project presentation, i.e. the presentation should be engaging, clear, and well-rehearsed. You don’t need to be as critical of your project as you were in the paper presentation, but you should point out any strengths or weaknesses of your project and think critically about how it could be improved. The length of the presentations will depend on the number of groups and will be announced once groups are finalized. Each member of the group should present a part or parts of the presentation that they are individually responsible for. Like your paper presentation, you should carefully rehearse your presentation both individually and as a group to ensure it flows well and that it is on time. Your presentation should cover the same points as the paper presentation, but should also mention any thinking behind design choices, motivations, etc. You should thoroughly present related work and your method since class members will not be familiar with the background of your project. The grading criteria for the project presentation is the same as for the paper presentation, though your project presentation should be especially polished. Since you have now received one round of feedback on presentation skills, you should use it to improve your presentation style. Mistakes from your prior presentation that are repeated in your project presentation will be graded more harshly, since you have received prior feedback. Please carefully review the peer reviews and grading notes from your paper presentation. Following your presentation, we will discuss your group project and provide feedback you can incorporate in your final report.

Final report

The project final report should resemble a CVPR conference paper and should be eight pages (excluding references). This means having a polished concept figure, method figure(s), tables with results, qualitative results, etc. To be clear, your final report should be of the same quality of presentation as the other conference papers you have read in this class (even though your results may not be as compelling from a one-semester class project). Your final report must include an Abstract as well as Introduction, Related work, Approach, Results, and Conclusion sections. The final report should be self-contained and thoroughly described in sufficient detail that someone else in this class could implement your approach given the chance. Each student in the group will be required to privately submit a brief writeup on a separate Canvas page documenting what every other group member in the group contributed to the project and how work was divided among group members.

Final project grading criteria

Your final project grade will primarily be based on the thought process and effort put into your project as demonstrated through your presentation and final report. While you should work to get the best results possible, it is understood that given the limited time available your method might not outperform other state-of-the-art approaches. The best projects are those that have new, clever ideas, not necessarily those that perform best on a given benchmark. Your project report (and presentation) will be evaluated on the following factors: 1) how well you related it to prior research; 2) the clarity and format of the presentation and report; 3) and completeness. For completeness, you will be evaluated on whether you complied with all the requirements of the project (e.g. does your report have all the required sections) and the degree to which you put in the thought and effort required to deliver an interesting and compelling class project. The paper will also be evaluated on the degree to which all experimental evaluations necessary for evaluating it have been performed (e.g. main results, ablations, qualitative results).

Project deliverables

All project deliverables will be uploaded on Canvas.

  • Project proposal (5% of final grade) - due October 2nd, 11:59 PM

  • Project status report (5% of final grade) - due November 6, 11:59 PM

  • Project presentations (15% of final grade) - due the day before your scheduled presentation at 11:59 PM

  • Project final report (20% of final grade) - due December 11, 11:59 PM

Late policy for project deliverables: Each group gets three late days that apply to the project deliverables. Late days are measured in days, not hours or minutes, so a submission that is one minute late is one day late and uses one late day. The three late days are to be used across all three deliverables, not per deliverable, and they are subtracted from the group as a whole. You do not need to request an extension and you will not receive a penalty as long as you stay within your three day budget. It is strongly recommended that you preserve these for the final report. Once your late days are used up, every day that a deliverable is late will result in your score for that deliverable being multiplied by 0.5. These late days apply to the project only and are separate from the three late days you get for paper reviews.

Final grades must be submitted shortly after the end of the exam period, so your three late days are the only extension available on the final report. A final report submitted more than three days late will not be graded.

A note on submission: You must submit all components of the final project on time. It is your responsibility to make sure all submissions in this class are complete. Once you submit, please download the file again and verify it opens successfully. Corrupted files will receive no credit. In the event of an outage on Canvas that affects submission, you may e-mail the instructor your files as a fallback (before the deadline).

Regrades: If you have a question about a grade you received or want to request a regrade, you must submit the request to me by Canvas message within one week of receiving your score. I will not consider regrade requests after a week. In general, I have discretion to assign grades and partial credit which fairly reflect your performance, so regrade requests should be reserved for cases where you believe the grade is clearly erroneous or where I may have misunderstood or overlooked part of your work. Please do not file regrade requests over small judgment calls. At the end of the semester this window is necessarily shorter. Any question about the grade on your final report or on your final course grade must be raised as soon as you become aware of it, and in no case later than 12:00 PM on the day final grades are due. If you do not submit a regrade request within the time allotted, your grade is final.

GPU Access: Your final project will likely require use of a GPU. Some GPU resources you can utilize include Google Colab, the Advanced Research Computing center at Virginia Tech, and the GLogin cluster run by the department (SSH to glogin.cs.vt.edu). Please take note of GPU limitations when designing your final project.

Additional information

Academic accommodations

Virginia Tech welcomes students with disabilities into the University’s educational programs. The University promotes efforts to provide equal access and a culture of inclusion without altering the essential elements of coursework. If you anticipate or experience academic barriers that may be due to disability, including but not limited to ADHD, chronic or temporary medical conditions, deaf or hard of hearing, learning disability, mental health, or vision impairment, please contact the Services for Students with Disabilities (SSD) office at (540)-231-3788, ssd@vt.edu, or visit https://ssd.vt.edu. If you have an SSD accommodation letter, please e-mail me as early in the semester as possible to deliver your letter and discuss your accommodations. You must give me reasonable notice to implement your accommodations, which is generally 5 business days.

Academic integrity

The tenets of the Virginia Tech Graduate Honor Code will be strictly enforced in this course, and all assignments shall be subject to the stipulations of the Graduate Honor Code. For more information on the Graduate Honor Code, please refer to the GHS Constitution. Specifically, you are encouraged to discuss the content covered in this course with others. However, you are responsible for doing your paper reviews on your own. This means you should work on your paper reviews yourself and should not share your paper reviews with others. You are allowed to use code and materials from other papers and sites, but you must cite your sources and clearly describe your contributions. You may also use generative AI tools (e.g. ChatGPT) to polish your writing or to help with coding, but you must not use such tools to complete your paper reviews for you. For example, uploading a paper to ChatGPT and requesting it list strengths and weaknesses that you then re-write on your own is an academic integrity violation. Such activity will be reported to the graduate school and may result in you failing the class and receiving a disciplinary penalty. The normal sanction recommended for a violation of the Honor Code is an F* sanction as your final course grade. If you have any questions as to whether something runs afoul of this policy, please contact the instructor before using the resource or submitting the assignment.

To be clear about what is and is not allowed with generative AI in this course: you may use these tools to help you understand a paper, to answer background questions you have while you are reading, to help you write code for your final project, and to polish the language of something you have already written yourself. What you may not do is use them to produce the substance of your work. You are responsible for reading each paper yourself and for understanding what is in it. The strengths, the weaknesses, and the ideas for future work in your review have to be your own, arrived at by thinking about the paper. Writing down what ChatGPT or Claude told you the strengths and weaknesses were defeats the entire point of this class, which is to teach you to think critically about research. Students who do this are mostly short changing themselves, because reading a paper and forming your own judgment about it is the skill this class exists to build, and it is a skill you will need for the rest of your research career.

If you use a generative AI tool for any part of an assignment in this course, you must disclose it and submit the complete chat(s) or interaction(s) you had along with your submission. This means the entire interaction, not a summary of it and not the portions you think are relevant. Failure to disclose, or to attach the complete interaction, is an Honor Code violation and will be treated as one.

Subject to the above, you may use any tools you find productive in preparing your written work, but you must be aware that you are responsible for any misrepresentation, factual inaccuracy, or plagiarism submitted. Submissions containing citations of non-existent, misrepresented, or fabricated material or obvious factual inaccuracies will be treated as academic misconduct and may result in a significant penalty, including receiving a zero for the entire assignment or referral for violation of the Honor Code. It is not a defense to a charge of plagiarism or of inaccuracy to argue that “an LLM did it”. You are taking a graduate course and are responsible for what you submit.

Copyrighted material

All materials provided in this course are subject to copyright. This applies to slides, recordings, notes, project descriptions, etc. You may use all the provided materials for your personal use. However, you may not share any course material with others, including posting the material on the web or distributing through other file sharing services.

Emergencies and medical conditions

If you have an emergency or medical condition, you must inform the instructor before the deadline of the assignment, or if it was genuinely impossible to do so (this is extraordinarily rare), as soon as possible afterwards. If you fail to inform the instructor before the deadline, you may receive a zero on the assignment even if your absence is later approved or verified by the Dean of Students. No exceptions will be made unless imminent and extraordinary circumstances made notifying me impossible. You may be required to submit documentation of the emergency or condition to the Dean of Students Office.

Extension requests: To ensure fairness, extensions will not be considered absent documented extraordinary circumstances. You should also inform the instructor before the deadline of the assignment or exam. You will also likely be required to submit documentation to the Dean of Students Office for verification.

Incomplete requests: Should you be unable to complete the requirements of this course during the semester because of extraordinary circumstances, you may request an incomplete through the last day of class. You will also likely be required to submit documentation to the Dean of Students Office for verification.

Food access

If you have difficulty affording groceries or accessing sufficient food to eat every day, or if you lack a safe and stable place to live, and you believe this may affect your performance in this course, you are urged to contact the Dean of Students Office for support at (540)-231-3086, or to complete an interest form to participate in The Market at Virginia Tech.

The Dean of Students, through The Market at Virginia Tech, offers food options and other resources. There is also a Student Emergency Fund program. If you are comfortable doing so, please notify me or your departmental advisor of your situation. This will allow us to direct you to any resources we have access to.

Other Virginia Tech food access programs: Food share cabinets are distributed throughout campus. These mini-pantries are intended to support short-term food access needs and primarily provide non-perishable easy-prep meals, snacks, and nutritious food options. They are placed in various locations across campus and are available to all students. Locations are shown on the VT Food Share Cabinet Map.

Community food access programs: 209 Marketplace is a student-run food pantry located just off campus. It is open to any Virginia Tech student, though first-time customers must complete a form. Their website has up-to-date hours and location information. The NRV Food Assistance Directory lists food assistance resources available in the New River Valley.

Schedule

Date Topic Papers Slides
08/25 Introduction, what world models are, and a brief history   Slides
08/27 Recurrent networks, attention, transformers   Slides
09/01 Transformers, self-supervised learning, reinforcement learning   Slides
09/03 Markov decision processes, the Bellman equation, deep Q-learning   Slides
09/08 Policy gradient methods, actor-critic, model-based RL and world models   Slides
09/10 Latent dynamics models and learning in imagination Primary: Dream to control: Learning behaviors by latent imagination
Secondary: Recurrent world models facilitate policy evolution
Primary: Mastering diverse control tasks through world models
Secondary: Learning latent dynamics for planning from pixels
Extra Background: Mastering Atari with discrete world models
 
09/15 Transformer world models Primary: STORM: Efficient stochastic transformer based world models for reinforcement learning
Secondary: Transformers are sample-efficient world models
Extra Background: Transformer-based world models are happy with 100k interactions
 
09/17 Planning and policy learning with a learned model Primary: When to trust your model: Model-based policy optimization
Secondary: Deep reinforcement learning in a handful of trials using probabilistic dynamics models
Extra Background: Mastering Atari, Go, chess and shogi by planning with a learned model
 
09/22 Prediction in representation space Primary: V-JEPA 2: Self-supervised video models enable understanding, prediction and planning
Secondary: Revisiting feature prediction for learning visual representations from video
Extra Background: Intuitive physics understanding emerges from self-supervised pretraining on natural videos
 
09/24 Diffusion world models Primary: Diffusion for world modeling: Visual details matter in Atari
Secondary: Model-based reinforcement learning for Atari
Extra Background: Denoising diffusion probabilistic models
 
09/29 Video generation as world simulation Primary: Learning interactive real-world simulators
Secondary: Learning universal policies via text-guided video generation
Extra Background: Video models are zero-shot learners and reasoners
 
10/01 Neural game engines Primary: Diffusion models are real-time game engines
Secondary: Genie: Generative interactive environments
Extra Background: Playable video generation
 

Acknowledgements

This course was inspired by and/or uses resources from the following courses: