Spring 2001

Hours: 3 hours (extended course project option for 4 credit hours: 3 of CIS 830/864, 1 of CIS 798)

Prerequisite: First course in artificial intelligence (CIS 730 or equivalent coursework in theory and design of intelligent systems) and basic database systems background, or instructor permission

Textbook: Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. I. H. Witten and E. Frank. Morgan Kaufmann. ISBN: 1558605525

Venue: 10:30am-11:20am Monday/Wednesday/Friday, Room 127 Nichols Hall

Instructor: William H. Hsu, Department of Computing and Information Sciences

Office: 213 Nichols Hall URL: http://www.cis.ksu.edu/~bhsu E-mail: bhsu@cis.ksu.edu

Office phone: (785) 532-6350 Home phone: (785) 539-7180 ICQ: 28651394

Office hours:

In classroom: 9:45am – 10:15am Monday/Wednesday

At office:12:45pm-1:45pm, Wednesday; by appointment 1pm-2pm, Friday

Class web page: http://www.kddresearch.org/Courses/Spring-2001/CIS830/

Course Description

This course provides intermediate background in intelligent systems for graduate and advanced undergraduate students. Four topics will be discussed: fundamentals of data mining (DM) and knowledge discovery in databases (KDD), artificial neural networks for regression and clustering in KDD, reasoning under uncertainty for decision support, and genetic algorithms and evolutionary systems for KDD. Applications to practical design and development of intelligent systems will be emphasized, leading to class projects on current research topics.

Course Requirements

Homework: 5 of 6 programming and written assignments (15%)

Papers: 8 (out of 12) written (1-page) reviews of research papers (16%); class presentations (10%)

Class participation: in-class discussion, quizzes (4%)

Examinations: 1 in-class midterm (15%), 1 take-home final (20%)

Computer language(s): C/C++, Java, or student choice (upon instructor approval)

Project: term programming project for all students (20%); additional term paper or project extension (4 credit hour) option for graduate students and advanced undergraduates

Selected reading (on reserve in K-State CIS Library):

· Artificial Intelligence: A Modern Approach, S. J. Russell and P. Norvig. Prentice-Hall, 1995. ISBN: 013108052

· Machine Learning, T. M. Mitchell. McGraw-Hill, 1997. ISBN: 0070428077

Additional bibliography (excerpted in course notes and handouts):

· Pattern Classification, 2^nd Edition, R. O. Duda, P. E. Hart, and D. G. Stork. John Wiley and Sons, 2000. ISBN: 0471056693

· Neural Networks for Pattern Recognition, C. M. Bishop. Oxford University Press, 1995. ISBN: 0198538499

· Genetic Algorithms in Search, Optimization, and Machine Learning, D. E. Goldberg. Addison-Wesley, 1989. ISBN: 0201157675

· Advances in Knowledge Discovery and Data Mining, U. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy, eds. MIT Press, 1996. ISBN: 0262560976

· Readings in Machine Learning, J. W. Shavlik and T. G. Dietterich, eds. Morgan-Kaufmann, 1990. ISBN: 1558601430

· Probabilistic Reasoning in Intelligent Systems, J. Pearl. Morgan-Kaufmann, 1988. ISBN: 0934613737

· Genetic Programming, J. Koza. MIT Press, 1992. ISBN: 0262111705

Syllabus

Lecture	Date	Topic	(Primary) Source
0	January 12	Administrivia; overview of topics	RN Chapters 1-9, 14, 18
1	January 17	AI topics review	RN Chapters 1-9, 14, 18
2	January 19	Introduction to machine learning	TMM 2-5, 7; RN 18
3	January 22	Data mining basics	WF 1-2
4	January 24	DM/KDD/decision support: overview	WF 3, 8; MLC++ tutorial
5	January 26	From DM to KDD: discussion	FSS96; MLC++ tutorial
6	January 29	From DM to KDD: presentation	Fayyad, Shapiro, Smyth
7	January 31	DM/KDD feature selection: discussion	LM98; KJ97; WF 4.3
8	February 2	DM/KDD feature selection: presentation	Liu and Motoda
9	February 5	DM/KDD rule learning: discussion	HK96; WF 4.1-4.2, 4.4-4.5
10	February 7	DM/KDD rule learning: presentation	Hsu and Knoblock
11	February 9	DM/KDD/decision support: conclusions	WF 7-8
12	February 12	Introduction to ANNs	TMM 4, RN 19
13	February 14	ANNs for KDD	TMM 4
14	February 16	Linear models, regression: discussion	Mo94, WF 4.6
15	February 19	ANNs and time series: presentation	Mozer
16	February 21	Combining ANNs: discussion	Sh99
17	February 23	Combining ANNs: presentation	Sharkey
18	February 26	Clustering: discussion	WF 3.9; DHS 10.1-10.9; BL 5.3
19	February 28	ANNs, unsupervised ML: presentation	Vesanto and Alhoniemi
20	March 2	Midterm review	WF 1-4
21	March 5	ANNs: conclusions	RN 19
22	March 7	Midterm exam	WF 1-4
23	March 9	Introduction to uncertain reasoning	RN 14-15, 19; TMM 6; WF 4.2
24	March 12	Bayesian network learning	Friedman and Goldszmidt
25	March 14	Bayesian network inference	Charniak; Cheeseman
26	March 16	Probabilistic clustering: discussion	CS96; WF 6.6; DHS 3.1-3.3, 3.9
27	March 26	Probabilistic clustering: presentation	Cheeseman and Stutz
28	March 28	BN learning: discussion	He96
29	March 30	BN learning: presentation	Heckerman
30	April 2	BN inference: discussion	Co98; RH 15; TMM 6.9-6.11
31	April 4	BN inference: presentation	Cowell
32	April 6	Uncertainty: conclusions	TMM 6
33	April 9	Genetic algorithms	M. Mitchell; TMM 9
34	April 11	Genetic programming	Koza: book, GP3 video
35	April 13	GAs for KDD: discussion	Go98; TMM 9; Goldberg 1
36	April 16	GAs: presentation	Goldberg
37	April 18	Classifier systems: discussion	BGH89
38	April 20	Classifier systems: presentation	Booker, Goldberg, Holland
39	April 23	GP: discussion	Lu97; Gustafson and Hsu
40	April 25	GP: presentation	Luke
41	April 27	GAs/GP/EC: conclusion; FINAL EXAM	Goldberg 6
42	April 30	Project presentations I; projects due	N/A
43	May 2	Project presentations II	N/A
44	May 4	Project presentations III	N/A
45	May 8	FINAL EXAM DUE (TENTATIVE)	RN Sections V-VII

WF: Data Mining, I. H. Witten and E. Frank

RN: Artificial Intelligence: A Modern Approach, S. J. Russell and P. Norvig

TMM: Machine Learning, T. M. Mitchell

Lightly-shaded entries denote the due date of a paper review.

Heavily-shaded entries denote the due date of a written or programming assignment (always on a Friday). Assignments are assessed a late penalty beginning the day of the next class (a Monday). Note: to be permitted to turn in an assignment past the Friday due date, a student must notify the instructional staff (cis830ta@ringil.cis.ksu.edu) in writing between the shaded date and the due date