CIS 732: Machine Learning and Pattern Recognition
Fall 2001
Hours: 3 hours (extended CIS732 course project option
available)
Prerequisite: CIS 501 or equivalent coursework
in data structures and algorithms; CIS 301 (set theory/logic) and 575
(algorithm design), Math 510 (discrete math), Stat 410 (intro probability)
recommended
Textbook: Machine Learning,
T. M. Mitchell. McGraw-Hill, 1997.
ISBN: 0070428077
Time and Venue: Tuesday, Thursday 14:05 – 15:20, Room 236 Nichols Hall
(N236)
Instructor: William H. Hsu, Department of Computing and
Information Sciences
Office: N213 Office phone: (785) 532-6350 x29 Home phone: (785) 539-7180
URL: http://www.cis.ksu.edu/~bhsu E-mail: cis732ta@www.kddresearch.org
Office hours: before CIS730 (12-12:30);
14:00 – 15:30 Wednesday; by appointment
Class web page: http://www.kddresearch.org
/Courses/Fall-2001/CIS732/
An introductory course in machine
learning for development of intelligent knowledge based systems. The first half of the course will focus on
basic taxonomies and theories of learning, algorithms for concept learning,
statistical learning, knowledge representation, pattern recognition, and
reasoning under uncertainty. The second
half of the course will survey fundamental topics in combining multiple models,
learning for plan generation, decision support, knowledge discovery and data
mining, control and optimization, and learning to reason.
The
course will include several written and programming assignments and a term
project option for graduate students.
Ancillary readings will be assigned; students will write a brief
synopsis and review for one of these papers every other lecture.
·
Artificial Intelligence: A
Modern Approach,
S. J. Russell and P. Norvig. Prentice
Hall, 1995. ISBN: 0131038052
·
Readings in Machine Learning, J. W. Shavlik and T. G.
Dietterich, eds. Morgan Kaufmann, 1990.
ISBN: 1558601430
·
Readings in Computer Inference
and Knowledge Acquisition, B. G. Buchanan and D. C. Wilkins, eds.
Morgan Kaufmann, 1993. ISBN:
1558601635
·
Readings in Uncertain Reasoning, G. Shafer and J. Pearl. Morgan Kaufmann, 1990. ISBN: 1558601252
·
Genetic Algorithms in Search,
Optimization, and Machine Learning, D. E. Goldberg.
Addison-Wesley, 1989. ISBN:
0201157675
·
Neural Networks for Pattern Recognition, C. M. Bishop.
Oxford University Press, 1995.
ISBN: 0198538499
·
Genetic Programming: On The Programming of Computers by Means of Natural
Selection, J.
Koza. MIT Press, 1992. ISBN: 0262111705
|
Lecture |
Date |
Topic |
Source |
|
0 |
2001
Aug 21 |
Administrivia;
overview of learning |
TMM
Chapter 1 |
|
1 |
2001
Aug 23 |
Concept
learning, version spaces |
TMM
2 |
|
2 |
2001
Aug 28 |
Inductive
bias, PAC learning |
TMM
2, 7.1-3; handouts |
|
3 |
2001
Aug 30 |
PAC,
VC dimension, error bounds |
TMM
7.4.1-3, 7.5.1-3 |
|
4 |
2001
Sep 04 |
Decision
trees; using MLC++ |
TMM
3; RN 18 |
|
5 |
2001
Sep 06 |
Decision
trees, overfitting, Occam |
TMM
3 |
|
6 |
2001
Sep 11 |
Perceptrons,
Winnow |
TMM
4 |
|
7 |
2001
Sep 13 |
Multi-layer
perceptrons, backprop |
TMM
4; CB; handouts |
|
8 |
2001
Sep 18 |
Estimation
and confidence intervals |
TMM
5 |
|
9 |
2001
Sep 20 |
Bayesian
learning: MAP, ML |
TMM
6 |
|
10 |
2001
Sep 24 |
Bayesian
learning: MDL, BOC, Gibbs |
TMM
6 |
|
11 |
2001
Sep 27 |
Naïve
Bayes; prob. learning over text |
TMM
6; handouts |
|
12 |
2001
Oct 02 |
Bayesian
networks |
TMM
6; RN 14-15 |
|
13 |
2001
Oct 04 |
Bayesian
networks |
TMM
6; paper |
|
14 |
2001 Oct 09 |
BNs concluded; midterm review |
TMM 1-7; RN 14-15, 18 |
|
15 |
2001 Oct 11 |
Midterm Exam
|
(Paper) |
|
16 |
2001
Oct 16 |
EM,
unsupervised learning |
TMM
7 |
|
17 |
2001
Oct 18 |
Time
series and stochastic processes |
Handouts |
|
18 |
2001
Oct 23 |
Policy
learning; MDPs |
RN
16-17 |
|
19 |
2001
Oct 25 |
Reinforcement
learning I |
TMM
13; RN 20; papers |
|
20 |
2001
Oct 30 |
Reinforcement
learning II |
TMM
13 |
|
21 |
2001
Nov 01 |
Neural
computation |
Papers;
RN 19 |
|
22 |
2001
Nov 06 |
Combining
classifiers (WM, bagging) |
TMM
7 |
|
23 |
2001
Nov 08 |
Boosting |
TMM
9; papers |
|
24 |
2001
Nov 13 |
Introduction
to genetic algorithms |
TMM
9; DEG |
|
25 |
2001
Nov 15 |
Genetic
programming |
TMM
9; JK; papers |
|
26 |
2001
Nov 20 |
IBL,
k-nearest neighbor, RBFs |
TMM
8.1-4 |
|
27 |
2001
Nov 27 |
Rule
learning and extraction |
TMM
10; paper |
|
28 |
2001
Nov 29 |
Inductive
logic programming |
TMM
10; RN 21 |
|
29 |
2001
Dec 04 |
Data
mining/KDD: application survey |
Handouts
(No paper) |
|
30 |
2001 Dec 06 |
Final review |
TMM 1-10, 13; RN 14-21 |
|
31 |
TBD |
FINAL EXAM
|
TMM 1-10, 13 |
TMM: Machine Learning, T. M. Mitchell
RN: Artificial Intelligence: A Modern Approach, S. J. Russell and P.
Norvig
DEG: Genetic Algorithms in Search, Optimization, and Machine Learning,
D. E. Goldberg
CB: Neural Networks for Pattern Recognition, C. M. Bishop
JK: Genetic Programming: On The Programming of Computers by Means of
Natural Selection, J. Koza
Lightly-shaded entries denote
the (Thursday) due date of a written problem set.
Heavily-shaded entries
denote the (Thursday) due date of a machine problem (programming
assignment).
Green-shaded entries
denote the (Tuesday) due date of a paper review.