CIS 732: Machine Learning and Pattern Recognition

Fall 2002

 

Hours: 3 hours (extended CIS732 course project option available)

Prerequisite: CIS 501 or equivalent coursework in data structures and algorithms; CIS 301 (set theory/logic) and 575 (algorithm design), Math 510 (discrete math), Stat 410 (intro probability) recommended

Textbook: Machine Learning, T. M. Mitchell.  McGraw-Hill, 1997. ISBN: 0070428077

Time and Venue: Tuesday, Thursday 11:05 - 12:20, Room 127 Nichols Hall (N127)

Instructor: William H. Hsu, Department of Computing and Information Sciences

Office: N213            Office phone: (785) 532-6350 x29            Home phone: (785) 539-7180

URL: http://www.cis.ksu.edu/~bhsu              

E-mail: cis732ta@www.kddresearch.org

Office hours:    10:00 – 11:00, 13:50 – 14:50 Tuesday/Thursday;

by appointment, Friday after 16:00

Class web page: http://www.kddresearch.org /Courses/Fall-2002/CIS732/

 

Course Description

 

            An introductory course in machine learning for development of intelligent knowledge based systems.  The first half of the course will focus on basic taxonomies and theories of learning, algorithms for concept learning, statistical learning, knowledge representation, pattern recognition, and reasoning under uncertainty.  The second half of the course will survey fundamental topics in combining multiple models, learning for plan generation, decision support, knowledge discovery and data mining, control and optimization, and learning to reason.

 

            The course will include several written and programming assignments and a term project option for graduate students.  Ancillary readings will be assigned; students will write a brief synopsis and review for one of these papers every other lecture.

 

Selected reading (on reserve in K-State CIS Library)

 

·         Artificial Intelligence: A Modern Approach, S. J. Russell and P. Norvig.  Prentice Hall, 1995.      ISBN: 0131038052

·         Readings in Machine Learning, J. W. Shavlik and T. G. Dietterich, eds.  Morgan Kaufmann, 1990. ISBN: 1558601430

 

Additional bibliography (excerpted in course notes and handouts)

 

·         Readings in Computer Inference and Knowledge Acquisition, B. G. Buchanan and D. C. Wilkins, eds.  Morgan Kaufmann, 1993.  ISBN: 1558601635

·         Readings in Uncertain Reasoning, G. Shafer and J. Pearl.  Morgan Kaufmann, 1990.  ISBN: 1558601252

·         Genetic Algorithms in Search, Optimization, and Machine Learning, D. E. Goldberg.  Addison-Wesley, 1989.  ISBN: 0201157675

·         Neural Networks for Pattern Recognition, C. M. Bishop.  Oxford University Press, 1995.  ISBN: 0198538499

·         Genetic Programming: On The Programming of Computers by Means of Natural Selection, J. Koza.  MIT Press, 1992.  ISBN: 0262111705

Syllabus

 

Lecture

Date

Topic

Source

0

2002 Aug 26

Administrivia; overview of learning

TMM Chapter 1

1

2002 Aug 28

Concept learning, version spaces

TMM 2

2

2002 Sep 03

Inductive bias, PAC learning

TMM 2, 7.1-3; handouts

3

2002 Sep 05

PAC, VC dimension, error bounds

TMM 7.4.1-3, 7.5.1-3

4

2002 Sep 10

Decision trees; using MLC++

TMM 3; RN 18

5

2002 Sep 12

Decision trees, overfitting, Occam

TMM 3

6

2002 Sep 17

Perceptrons, Winnow

TMM 4

7

2002 Sep 19

Multi-layer perceptrons, backprop

TMM 4; CB; handouts

8

2002 Sep 24

Estimation and confidence intervals

TMM 5

9

2002 Sep 26

Bayesian learning: MAP, ML

TMM 6

10

2002 Oct 01

Bayesian learning: MDL, BOC, Gibbs

TMM 6

11

2002 Oct 03

Naïve Bayes; prob. learning over text

TMM 6; handouts

12

2002 Oct 08

Bayesian networks

TMM 6; RN 14-15

13

2002 Oct 10

Bayesian networks

TMM 6; paper

14

2002 Oct 15

BNs concluded; midterm review

TMM 1-7; RN 14-15, 18

15

2002 Oct 17
Midterm Exam

(Paper)

16

2002 Oct 22

EM, unsupervised learning

TMM 7

17

2002 Oct 24

Time series and stochastic processes

Handouts

18

2002 Oct 29

Policy learning; MDPs

RN 16-17

19

2002 Oct 31

Reinforcement learning I

TMM 13; RN 20; papers

20

2002 Nov 05

Reinforcement learning II

TMM 13

21

2002 Nov 07

Neural computation

Papers; RN 19

22

2002 Nov 12

Combining classifiers (WM, bagging)

TMM 7

23

2002 Nov 14

Boosting

TMM 9; papers

24

2002 Nov 19

Introduction to genetic algorithms

TMM 9; DEG

25

2002 Nov 21

Genetic programming

TMM 9; JK; papers

26

2002 Nov 26

IBL, k-nearest neighbor, RBFs

TMM 8.1-4

27

2002 Dec 03

KDD / Rule learning and extraction

TMM 10; paper

28

2002 Dec 05

KDD / Inductive logic programming

TMM 10; RN 21

29

2002 Dec 10

KDD / Support vector machines

New

30

2002 Dec 12

Final review

TMM 1-10, 13; RN 14-21

31

TBD
FINAL EXAM

TMM 1-10, 13

 

TMM: Machine Learning, T. M. Mitchell

RN: Artificial Intelligence: A Modern Approach, S. J. Russell and P. Norvig

DEG: Genetic Algorithms in Search, Optimization, and Machine Learning, D. E. Goldberg

CB: Neural Networks for Pattern Recognition, C. M. Bishop

JK: Genetic Programming: On The Programming of Computers by Means of Natural Selection, J. Koza

 

Lightly-shaded entries denote the (Thursday) due date of a written problem set.

Heavily-shaded entries denote the (Thursday) due date of a machine problem (programming assignment).

Green-shaded entries denote the (Tuesday) due date of a paper review.