Pointwise Mutual Information using Collocation

Pointwise Mutual Information defined:


P(x,y) = frequency of bigram of x,y
P(x) = frequency of term x
P(y) = frequency of term y
log is base 2

Given a corpus of clinical text, in our case guideline recommendations, if two terms (x,y) occur sequentially, PMI gives us a way to decide whether the pair is closely connected or whether one or the other of the two terms appears so often in the corpus on its own that the fact they appear together is more structure than meaning. Higher PMI values indicate lower individual counts relative to paired counts, so the term is more likely to be important in identifying the contextual meaning of the recommendation being considered. For example, if we look at the lowest PMI values, -1.66, -1.22, -1.08 and their respective pairs (patient, should) , (risk, patients), (treatment, patients), we can see that they are not indicative of a particular guideline. On the other hand, the highest PMI values 10.9, give us (bundle, branch), (infantile, spasms), (microscopic, hematuria). (neuromuscular, blocking) which are much more indicative of a particular guideline.