Courses:

You should apply one or more of the models covered in class for your final project.

If you implement the algorithms yourself, then you do not need to compare more than one model, but you have to document your implementation and append it with the final report (e.g., you implemented a genetic algorithm that selects variables to be used in a neural network model).

If you are using existing software packages (See5.0, Genie, Rosetta, etc.), then I would like you to apply your data in at least two different types of models, preferably more (e.g. classification tree and rough sets), justify why you think the data are better suited for these methods (e.g., categorical data are better suited for classification trees than neural networks).

Use medically related data. I will provide those with little medical connections with published data sets (e.g., trauma patients, extracted from Christensen's textbook on logistic regression). Use this set or another of your interest to compare performance of several models or to investigate selection of variables (e.g., comparison of different selection methods), selection of cases (e.g., detection of outliers). You can use medical data from:

• UCI depository and UCI KDD archive
• Statlib
• Delve - Data for Evaluating Learning in Valid Experiments
• R Project
• NCBI - National Center for Biotechnology Information

Remember to use the evaluation methods covered in class (ROC curves, calibration curves and indices, etc.) to report performance of your models.

The report should be between 4-6 single spaced pages, 10-point font. Report can be longer if you really need, but please do not add fluff. Include the usual:

• Abstract
• Background: (why the problem being addressed is important.)
• Material and Methods: (include here as much detail as possible here, such as the exact parameters you used -- learning rate 0.05, etc. -- software name and version, and anything that would make your work reproducible by another person.) Include pointers to the data sets you used, as well as any detail of pre- or post processing that was used (e.g., constructed randomly 10 pairs of training and test sets for cross-validation, etc.).
• Results
• Discussion
• Conclusion
• References
• Appendices (e.g., code implemented, pointers to data sets)

Delivery: Please create a web-page for your report. This web-page should contain:

• Title of the report
• An abstract/summary of the report
• A link to the report proper
• Links to source code/application files and data needed to repeat what you did for the report

If you want to install stuff on your own machine, R is free. Feel free to implement algorithms if you want. Other packages you might consider are:

• Neural networks: NevProp4, by Prof. Phil Goodman
• Logistic regression: SAS
• Classification trees: See5.0
• Rough sets: Rosetta
• Belief networks: SamIam

Please note that some of these are freely available as demo versions only, so there may be limits on number of cases, variables, days available for trial, etc.

• Compare performances of 4 different machine learning models in a certain data set, and fully report results using evaluation methods described in class.
• Compare performance of 1 machine learning model (explain why you chose this one) and one or more medical experts (the expert cannot be yourself).
• Investigate variable selection methods in different types of models (use of artificial sets to make a point is OK, but please also try in some real data).
• Investigate case selection methods in different types of models (use of artificial sets to make a point is OK, but please also try in some real data).

• Build a framework for application of some method without actually using or implementing the method and showing results in artificial or real data.
• Build a decision tree on some medical topic using you or your colleague's probabilities and utilities without testing in the real world.
• Apply one machine learning model in a data set and not compare results to an existing reported model or other models.
• Report results of machine learning models using accuracy alone.
• Recycle projects from other classes.

If you don't know whether your project would be OK or not, please discuss it with the instructors.

Example final projects are included courtesy of the students listed below. Used with their permission.

Application of Sequence Alignment Algorithms in Location Tracking Data to Determine Patient State in a Clinical Process

Final Report (PDF)

Oral Presentation (PDF - 1.70 MB)

Behavior of Various Machine Learning Models in the Face of Noisy Data

Final Report (PDF)

Computerized Pulmonary Artery Catheter Waveform Interpretation

Final Report (PDF)

Oral Presentation (PDF)

C# .NET Algorithm for Variable Selection Based on the Mallow's Cp Criterion

Final Report (PDF)