Information Theory & Natural Language Processing

[ In my spare time, I look into some of these topics as part of my excercises in my studies of optimization models and iterative algorithms. It's so interesting to see those different topics have similar mathematics background.

[This page was not changed during the past year. ]

Often, the behavior of Internet can be rooted back to the user behavior. I wonder if we can look at the web content in a natural language view and understand the usage pattern better...

A popular model in NLP is Maximum Entropy Model. The concept of entropy came from Shannon’s information theory in the middle of last century…

Some of the tutorials:

Adam Berger’s MaxEnt Tutorial at CMU

Adwait Ratnaparkhi’s Introduction to MaxEnt in NLP

Projects:

SourceForge Project: OpenNLP (MaxEnt is an interesting part of it.)