bss +ica -pca
directional +ica -pca
faces
natural sene ->ica <-edge
documents->topics
information theory
x1, x2, x3 -> learner -> y
010101
A 0, B 110, C 111, D 10
Joint Entropy
H(x,y)= -ΣP(x,y)logP(x,y)
Conditional Entropy
Hcy(x) = -Σp(x,y)logP(y|x)
H(y|x)
I(x,y)=H(y)-H(x|y)
D(p||g) = sp(x)log p(x)/q(x)
