Fall, 1999

CIS 798 (Topics in Computer Science)

Topics in Intelligent Systems and Machine Learning




Hours: 3

Prerequisite: CIS 300 (Algorithms and Data Structures) or equivalent; basic course in probability and statistics recommended

Textbook: Machine Learning, T. M. Mitchell. McGraw-Hill, 1997. ISBN: 0070428077

Instructor: William H. Hsu, Department of Computing and Information Sciences

Course Description

An introductory course in machine learning for development of intelligent knowledge based systems. The first half of the course will focus on basic taxonomies and theories of learning, algorithms for concept learning, statistical learning, knowledge representation, pattern recognition, and reasoning under uncertainty. The second half of the course will survey fundamental topics in combining multiple models, learning for plan generation, decision support, knowledge discovery and data mining, control and optimization, and learning to reason.

The course will include several written and programming assignments and a term project option for graduate students. Ancillary readings will be assigned; students will write a brief synopsis and review for one of these papers every other lecture.

Reserve texts

Other references Syllabus
 
Lecture Date Topic Source
0 August 24 Administrivia; overview of learning TMM Chapter 1
1 August 26 Concept learning, version spaces TMM 2
2 August 31 Inductive bias, PAC learning TMM 2, 7.1-3; handouts
3 September 2 PAC, VC dimension, mistake bounds TMM 7.4.1-3, 7.5.1-3
4 September 7 Decision trees; using MLC++ TMM 3; RN 18
5 September 9 Decision trees, overfitting, Occam TMM 3
6 September 14 Perceptrons, Winnow TMM 4
7 September 16 Multi-layer perceptrons, backprop TMM 4; CB; handouts
8 September 21 Estimation and confidence intervals TMM 5
9 September 23 Bayesian learning: MAP, max likelihood TMM 6
10 September 28 Bayesian learning: MDL, BOC, Gibbs TMM 6
11 September 30 Naïve Bayes; prob. learning over text TMM 6; handouts
12 October 5 Bayesian networks TMM 6; RN 14-15
13 October 7 Bayesian networks TMM 6; paper
14 October 12 Bayesian networks; midterm review TMM 1-7; RN 14-15, 18
15 October 14 Midterm Exam (Paper)
16 October 19 EM, unsupervised learning TMM 6
17 October 21 Time series and stochastic processes Handouts
18 October 26 Policy learning; MDPs RN 16-17
19 October 28 Reinforcement learning I TMM 13; RN 20; papers
20 November 2 Reinforcement learning II TMM 13
21 November 4 Neural computation Papers; RN 19
22 November 9 Combining classifiers (WM, bagging) TMM 7
23 November 11 Boosting TMM 9; papers
24 November 16 Introduction to genetic algorithms TMM 9; DEG
25 November 18 Genetic programming TMM 9; JK; papers
26 November 23 IBL, k-nearest neighbor, RBFs TMM 8.1-4
27 November 30 Rule learning and extraction TMM 10; paper
28 December 2 Inductive logic programming TMM 10; RN 21
29 December 7 Data mining/KDD: application survey Handouts (No paper)
30 December 9 Final review TMM 1-10, 13; RN 14-21
31 December 14 FINAL EXAM  

TMM: Machine Learning, T. M. Mitchell
RN: Artificial Intelligence: A Modern Approach, S. J. Russell and P. Norvig
DEG: Genetic Algorithms in Search, Optimization, and Machine Learning, D. E. Goldberg
CB: Neural Networks for Pattern Recognition, C. M. Bishop
JK: Genetic Programming: On The Programming of Computers by Means of Natural Selection, J. Koza