Pointwise Mutual Information using Collocation

Pointwise Mutual Information defined:


P(x,y) = frequency of bigram of x,y
P(x) = frequency of term x
P(y) = frequency of term y
log is base 2

Given a corpus of clinical text, in our case guideline recommendations, if two terms (x,y) occur sequentially, PMI gives us a way to decide whether the pair is closely connected or whether one or the other of the two terms appears so often in the corpus on its own that the fact they appear together is more structure than meaning. Higher PMI values indicate lower individual counts relative to paired counts, so the term is more likely to be important in identifying the contextual meaning of the recommendation being considered.