The Power and Perils of MDL

Prof. Pieter Adriaans (UvA), February 27th, 2007 - 15:00H

In this lecture I will present some recent work I did with Paul Vitanyi and Ceriel Jacobs on the application of the MDL (Minimum Description Length) principle to grammar induction. We have studied MDL in terms of two-part
code optimization and randomness deficiency.

1. These notions will be explained in the lecture. In this framework we showed that:
   Shorter code not necessarily leads to better theories, e.g. the randomness deficiency does not decrease monotonically with the MDL code,
2. contrary to what is suggested by the results of Gold:1967 there is no fundamental difference between positive and negative data from an MDL perspective,
3. MDL is extremely sensitive to the correct calculation of code length.

Using these ideas we have implemented a MDL variant of the EDSM algorithm. The results show that although MDL works well as a global optimization criterion, it falls short of the performance of algorithms that evaluate local features of the problem space. MDL can be described as a global strategy for featureless learning.

Back ground material:
http://homepages.cwi.nl/~paulv/papers/perils.pdf
http://staff.science.uva.nl/~pietera/ALS/background/lncs_icgi-mdl.pdf

Location

Nieuwe Achtergracht 127
Building C 
room C.210

Click here for a description on how to get to building C