Oct 18, 2006
-------------

- Midterm return
	- discussion of solutions

- Back to learning
	- PlayTennis handout
	- learning decision trees from this data

- Learning decision trees
	- what is a good tree?

- Entropy 
	- # bits necessary to encode something
	- \Sum_i p(i) log_2 p(i)
		- log_2 so you can think in bits

- Examples
	- Entropy of a coin that always falls heads
	- Entropy of a coin that always falls tails
	- Entropy of a fair coin
	- Entropy of a loaded die

- What does this have to do with decision trees?
	- think of entropy of "classifications"
		- encode PlayTennis="Yes" versus
		         PlayTennis="No"

- Example decision tree for PlayTennis dataset
	- why did we pick the attributes we did?
		- Answer: because they cause the
		  most reduction in entropy!

- Details, details
	- picking the first node
		- calculate entropy improvement 
	- divide up the dataset
		- recurse the calculations!
	- a greedy approach

- Worked out example
	- PlayTennis dataset