Quick Question
Compute the AUC of the CART model from the previous video, using the following command in your R console:
as.numeric(performance(pred, "auc")@y.values)
What is the AUC?
Explanation
If you run the command given above after going through the commands in Video 4, you get an AUC of 0.6927.
Now, recall that in Video 4, our tree had 7 splits. Let's see how this changes if we change the value of minbucket.
First build a CART model that is similar to the one we built in Video 4, except change the minbucket parameter to 5. Plot the tree.
How many splits does the tree have?
Explanation
You can build a CART model with minbucket=5 by using the following command:
StevensTree = rpart(Reverse ~ Circuit + Issue + Petitioner + Respondent + LowerCourt + Unconst, method="class", data = Train, minbucket=5)
If you plot the tree with prp(StevensTree), you can see that the tree has 16 splits! This tree is probably overfit to the training data, and is not as interpretable.
Now build a CART model that is similar to the one we built in Video 4, except change the minbucket parameter to 100. Plot the tree.
How many splits does the tree have?
Explanation
You can build a CART model with minbucket=100 by using the following command:
StevensTree = rpart(Reverse ~ Circuit + Issue + Petitioner + Respondent + LowerCourt + Unconst, method="class", data = Train, minbucket=100)
If you plot the tree with prp(StevensTree), you can see that the tree only has one split! This tree is probably not fit well enough to the training data.