Project: Decision Tree Classification with Logistic Regression-Based Splitting
Objective: To build decision tree classifiers using logistic regression-based splitting on three datasets, evaluate their performance metrics, interpret the output rules, and compare the results with those obtained from regular decision trees.
Approach:
1: Train decision tree classifiers on all three datasets using logistic regression as the function to split the node, and calculate precision, recall, accuracy, and AUC-ROC curve. Choose one attribute and pairs of attributes at each node for the split, and report the findings. Read the Logit Tree paper for relevant information on this topic.
2: Interpret the rules output from the decision tree by visualizing the tree and split criteria. Compare the output rules with those from regular decision trees, and make a list of observations from visualizing the splits. Perform this for both single-attribute and multi-attribute split models.
3: Perform 5-fold cross-validation on the models and report the performances in terms of precision, recall, F1-score, and accuracy.