Summary

What is information? What is its structure? How does it manifest itself? And how does information emerge and change while being used?
These themes have been studied since ancient times as information is key to all forms of communication and knowledge: it gives us a handle on a complex world.

Information is not just categorical, linguistic or visual. Sound—and silence—may generate strong signals. Interpretation and the recipient of the message are essential in this process: without the context that they provide, information remains a vacuous signal.

Information is both a foundational unifying scientific theme as well as a social phenomenon. As a consequence, theoretical understanding continuously interacts with experimental validation and applied practice.

Semantic relevance, for instance, is the core notion that underlies textual and visual search engines. While a search algorithm may be deterministic, the relevance of the search engine’s results is not, but the notion of relevant and meaningful information does have a counterpart in modern logic.

Semantic search engines learn from examples and the kind of inferencing that they perform is implicit and approximate. They gather semantic patterns from data about people, measurement data, images of objects or of the atmosphere, the meaning and sentiment of a paragraph of text.

Machine learning—the algorithmic recognition of patterns—plays an essential role here. There is a direct connection with the type of layered inferential architectures that have been found in cognitive science.

Finally, there is the user’s perspective on next generation search engines. Here, interaction, interpretation, the creation of contexts and profiles all play a central role, a role that is closely related to modern logic and its emphasis on information exchange.

This research priority is aimed at four questions on the structure and use of information.

• What are the fundamental characteristics and invariant patterns of information, whether textual, visual or encoded in other carriers?
   
• How does the representation of external information align with internal—neural, intuitive, social—representations as these are being observed through fMRI scans, Internet-wide sentiment analysis or cognitive studies?
• How do people search and use information, which algorithmic and mathematical patterns emerge through such acts and how does information change by being used, both by individuals and through strategic interactions?
• Which effective and efficient algorithms facilitate semantic, cognitive and interactive access to information?

This research priority has a single core objective: the development of theories concerning textual, visual and sensory information in which we arrive at an algorithmic and model-based understanding of content, interaction, context, use and experience.

On the one hand, our theories will lead to the design of fundamentally new search engines at the level of human cognition. To this end, the research priority will develop new methods and algorithms for aligning the interpretation image and text with human and social intuitions as well as experimental methodologies to achieve this.

On the other hand, the research priority will lead to the development of social software: procedures that govern our information behavior.
These design ambitions are in keeping with the intellectual tradition that goes back to antiquity. For instance, the rise of large scale ontologies within the semantic web can be seen as a practical and algorithmic realization of the ambition of the encyclopédistes.

Researchers at the Informatics Institute (IvI) and the Institute for Logic, Language and Computation (ILLC) cooperate in the theme “Information and Meaning in Image, Text and Inference.” The research priority is acknowledged by the Faculty of Science.

Scientific case

Understanding the meaning of the linguistic, visual and sensory world around us and thereby uncovering the structure of information in non-deterministic models is one of today’s great scientific challenges.

Within UvA significant strength has been built up at IvI (in De Rijke’s and Smeulders’ groups) in next generation search engines, especially concerning image search engines for the objective reality and concerning semantic textual search engines that are context-sensitive.

In annual academic benchmarking competitions in these areas (TREC, TRECvid, CLEF and PASCAL), in which universities such as Oxford, Carnegie Mellon, Berkeley, INRIA and Tsinghua take part, next to IBM, Microsoft and Yahoo! Research, UvA has occupied leading positions for many years.

In addition, through IvI UvA plays a coordinating role in European Science Foundation’s ELIAS program on evaluation methodology for information access systems, which brings together leading partners from 12+ European countries. Through ILLC, UvA plays a leading role in the European Strategic Research Program LOGICC (Modeling Intelligent Interaction: Logic in the Humanities, Computational and Social Sciences), funded by the European Science Foundation, which brings together strong research groups in logic and computer science from 10+ European countries.

Seven publications with large impact since 2009 are mentioned below.

Publications

1. Beninca E, J Huisman, R Heerkloss, KD Johnk, P Branco, EH van Nes, M Scheffer & SP Ellner (2008) Chaos in a long-term experiment with a plankton community. Nature 451: 822-825.
2. De Roos AM, T Schellekens, T van Kooten & L Persson (2008) Stage-specific predator species help each other to persist while competing for a single prey. Proc Natl Acad Sci USA 105: 13930-13935.
3. Mateos-Langerak J, M Bohn, W de Leeuw, O Giromus, EM Manders, PJ Verschure, MH Indemans, HJ Gierman, DW Heermann, R van Driel & S Goetze (2009) Spatially confined folding of chromatin in the interphase nucleus. Proc Natl Acad Sci USA 106: 3812-3817.
4. Vermeer JEM, JM Thole, J Goedhart, E Nielsen, T Munnik &TWJ Gadella (2009) Visualisation of PtdIns4P dynamics in living plant cells. Plant J. 57: 356-372.
5. Takken FLW & WIL Tameling (2009) To nibble at plant resistance proteins. Science 324: 744-746.
6. Young, BP, Shin, JJH, Orij, R, Chao, JT, Li, SC,  Guan XL, Khong, A,  Jan, E, Wenk, MR, Prinz, WA, Smits, GJ & CJR Loewen (2010) Phosphatidic acid is a pH biosensor that links membrane biogene­sis to metabolism. Science 329: 1085-1088.
7. TerBeek, A. and Brul, S (2010) To kill or not to kill Bacilli; opportunities for food biotechnology. Curr Opin Biotechnol. 21: 168-174.
8. Kolodkin A.N., Bruggeman FJ., Plant N, Moné MJ, Bakker BM, Campbell MJ, Van Leeuwen JPTM, Carlberg C, Snoep JL and Westerhoff HV (2010) Design principles of nuclear receptor signaling: how complex networking improves signal transduction. Mol. Systems Biology 6; doi 0.1038/msb.2010.102
9. Kramer G., Sprenger R.S., Nessen M.A., Roseboom W., Speijer D., de Jong L., Teixeira de Mattos M.J., Back J.W., de Koster C.G. (2010) Proteome-wide alterations in Escherichia coli translation rates upon anaerobiosis Molecular & Cellular Proteomics 9:2508–2516, 2010.
10. Duynhoven J. van, Vaughan E.E., Jacobs D., Kemperman R., Velzen E.J.J. van, Gross G., Roger L., Possemiers S., Smilde A.K., Dore J., Westerhuis J.A. and Wiele T. van der (2011) The metabolic fate of polyphenols in the human superorganism. Proc Natl Acad Sci USA 108: 4531-4538.