Entropy: controls how a DT decides where to split the data
definition: measure of impurity in a bunch of examples
entropy = Σi -Pi log2 (Pi)
Pi is fraction of examples in class i
all examples are same class -> entropy = 〇
examples are evenly split between classes -> entropy = 1.0
grade, bumpiness, speed limit, speed
ssff
Pi = 2 / 4 = 0.5
entropy
>>> import math >>> -0.5*math.log(0.5, 2) - 0.5*math.log(0.5, 2) 1.0
