About the Course
See the syllabus for details. Deep neural networks (DNNs) have been widely used for tackling numerous machine learning problems that were once believed to be challenging. With their remarkable ability of fitting training data, DNNs have achieved revolutionary successes in many fields such as computer vision, natural language progressing, and robotics. This is an introduction course to deep learning. Topics covered by this course include but are not limited to:
- Machine Learning Overview
- Optimization
- MLP, CNN, RNN, GNN
- Generative models
- Advanced topics
Instructor: Yao Li
Teaching Assistant: Kyung Rok Kim
Grader: Chengze Xie
Class: TTH 3:30PM-4:45PM, Hanes 120
Office Hours:
- Instructor Office Hours: W 4:00PM-5:00PM, Hanes 334
- TA Office Hours: M 10:00AM-12:00PM, Zoom
Course Material
| Date | Lecture | Slides | Tutorial |
|---|---|---|---|
| Aug. 20 | Welcome | Slides | |
| Aug. 22 | Overview of Machine Learning | Slides | Colab |
| Aug. 27 | Linear Regression and Classification | Slides | |
| Aug. 29 | Optimization | Slides | |
| Sep. 03 | Well-Being Day | ||
| Sep. 05 | Kernel Methods | Slides | |
| Sep. 10 | Neural Networks | Slides | |
| Sep. 12 | Convolutional Neural Networks I | Slides | |
| Sep. 17 | Convolutional Neural Networks II | Slides | HW1 Review |
| Sep. 19 | Convolutional Neural Networks III | Tutorial 1 | ResNet |
| Sep. 24 | Recurrent Neural Networks | Slides | |
| Sep. 26 | GRU and LSTM | Slides | |
| Oct. 01 | Project Proposal Discussion | ||
| Oct. 03 | Project Proposal Discussion | ||
| Oct. 08 | NLP Pre-training | Slides | Word2Vec |
| Oct. 10 | Generative Models | Slides | GAN |
| Oct. 15 | Online Tutorial | Zoom | Tutorial 2 |
| Oct. 17 | Fall Break | ||
| Oct. 22 | Adversarial Attack | Slides | |
| Oct. 24 | Networks Opinions (Guest Presentation) | Slides | |
| Oct. 29 | Adversarial Defense | Slides | |
| Oct. 31 | Transformer | Slides | BERT |
| Nov. 05 | Vision Transformer | Slides | HW3 Review |
| Nov. 07 | Federated Learning | Slides | pFL |
| Nov. 12 | Poisoning Attack | Slides | |
| Nov. 14 | Poisoning Defense | Slides | HW4&5 Review |
| Nov. 19 | Large Language Models | Slides | |
| Nov. 21 | Final Project Work Day | ||
| Nov. 26 | Final Presentation | ||
| Nov. 28 | Thanksgiving | ||
| Dec. 03 | Final Presentation |
Homework Tracker
All homework assignments are to be submitted via Canvas. Late homework will receive a grade of 0.
| Date assigned | Instructions | Due Date (Time) |
|---|---|---|
| Aug. 29 | HW1(.ipynb) | Sep. 15 (11:59 PM) |
| Sep. 12 | HW2(.ipynb) | Oct. 02 (11:59 PM) |
| Oct. 01 | HW3(.ipynb) | Oct. 13 (11:59 PM) |
| Oct. 13 | HW4(.ipynb) | Oct. 30 (11:59 PM) |
| Oct. 29 | HW5(.ipynb) | Nov. 10 (11:59 PM) |
Final Project Details
This course includes a final project in lieu of a final exam. Projects will be completed in groups of four and consist of:
- Project Proposal (10%)
- Project Presentation (30%)
- Project Paper (50%)
- Peer Review Score (10%)
Group List
Please form the final project group before August 30th, and sign up using the shared spreadsheet. Please don’t modify the information of other groups.
Four Parts Including Point Values
I will meet with each group to discuss the final project topic. Project topics can be:
- Solve an interesting or new problem with existing method
- Develop a new algorithm/model
- Compare state-of-the-art algorithms on some problems
- …
P1: Project Proposal (10 Points): The project proposal is limited to 2-page (excluding reference) and contains:
- Problem to solve
- Review of existing studies in this field
- Your proposed method or Methods you would like to compare
- Evaluation metric
- Reference
See latex template at link.
P2: Project Presentation (30 Points): All groups will present their final projects during the last week or two weeks of classes. Every group member is expected to join the presentation. The length of the presentation depends on the number of groups (10–20min) and will be announced later.
P3: Project Paper (50 Points): Each team must submit a written project report. It is recommended to include a discussion of how your research work can be further extended. It is required to use the NeurIPS Latex style files and submit the report in PDF format. The report should be less than 8 pages without references (no minimum requirement).
P4: Peer Score (10 Points): Ten points of the final project is based on an average score measuring overall contribution as seen by you and the other members of your group. Each group member should score every person in their group on a continuous scale from 0 (Bad) to 10 (Good). Before the due date of the final paper, every member is required to submit the group scoring through the google survey link below. Your name and this information will remain private between me and you. If you fail to submit this group scoring before the deadline, 2 points penalty will be applied and I will give the other members a score of 10.
Due Dates of Individual Parts
| Part | Description | Method (Location) of Submission | Due Date (Time) |
|---|---|---|---|
| P1 | Project Proposal | Canvas | Sep. 29 (11:59PM) |
| Proposal Meeting | Hanes 334 | Oct. 01 / Oct. 03 (3:30PM-4:45PM) | |
| P2 | Presentation Slides | Canvas | Nov. 20 / Nov. 25 (11:59PM) |
| Final Presentation | Class | Nov. 21 / Nov. 26 (3:30PM-4:45PM) | |
| P3 | Final Report | Canvas | Dec. 04 (11:59PM) |
| P4 | Peer Scoring | Google Survey | Dec. 04 (11:59PM) |
Paper Presentation
Check the paper list for paper presentation opportunity.
Reading
- Deep Learning, by Ian Goodfellow, Yoshua Bengio, Aaron Courville.
- Deep learning theory lecture note, by Matus Telgarsky.
- Foundations of machine learning, by Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar.
- Convex optimization, by Stephen Boyd, and Lieven Vandenberghe.
This page was last updated on 2024-11-19 15:24:45.788786 Eastern Time.