DIRAC - Detection and Identification of Rare Audio-visual Cues

An EC 6th Framework Programme supported IST Integrated Project (01/2006-12/2010)

Unexpected rare events are potentially information rich but still poorly processed by today's computing systems. DIRAC project has addressed this crucial machine weakness and developed methods for environment-adaptive autonomous artificial cognitive systems that will detect, identify and classify possibly threatening rare events from the information derived by multiple active information-seeking audio-visual sensors. Biological organisms rely for their survival on detecting and identifying new events. DIRAC therefore has combined its expertise in physiology of mammalian auditory and visual cortex and in audio/visual recognition engineering with the aim to move the art of audiovisual machine recognition from the classical signal processing/pattern classification paradigm to human-like information extraction.

The core research question of DIRAC concerns how to identify samples of data that do not fit into any of the classes seen during training, while at the same time constituting a valid object in the sense that the new sample is not merely a noisy measurement. The notion of a hierarchy of processing and classification levels has been identified as essential. Such a hierarchy is defined in terms of two views, parts-whole and class-memberhship partial orders. A "rare" event is detected when classifiers at different levels of the hierarchy give different -incongruous- results.

The book, papers and presentation below provide a good introduction to the theory, algorithms and application domains (audio, visual, engineering, biology) considered in DIRAC:

Weinshall D., Zweig A., Hermansky H., Kombrink S., Ohl F.W., Anemüller J., Bach J.-H., Van Gool L., Nater F., Pajdla T., Havlena M., Pavel M. "Beyond Novelty Detection: Incongruent Events, when General and Specific Classifiers Disagree", IEEE Transaction on Pattern Recognition and Machine Intelligence (PAMI), accepted for publication.
Weinshall D., Anemüller J., van Gool L. (Eds.) "Detection and Identification of Rare Audiovisual Cues", Springer, Studies in Computational Intelligence Vol. 384, 2012.
Weinshall D., Hermansky H., Zweig A., Luo J., Jimison H., Ohl F., Pavel M. "Beyond Novelty Detection: Incongruent Events, when General and Specific Classifiers Disagree", in 'Advances in Neural Information Processing Systems (NIPS)', Vancouver, December 2008. paper presentation
DIRAC Project Final Activity Report

Training

DIRAC conducted an extensive training and education program. In total, 48 PhD students have conducted their thesis work within the project. A highly successful series of yearly workshops with international guest speakers and hands-on project work addressed graduate students and researchers from within and outside of the project.

• DIRAC workshop 2010, a satellite event to the ECML/PKDD 2010 conference in Barcelona, Spain 
• Proceedings included in the book volume: Weinshall D., Anemüller J., van Gool L. (Eds.) "Detection and Identification of Rare Audiovisual Cues", Springer, Studies in Computational Intelligence Vol. 384, 2011.
• DIRAC workshop 2009, Leuven, Belgium
• DIRAC Cognitive Science summerschool 2008, Volterra, Italy
• DIRAC/CoSy workshop on Multi-Sensory Modalities in Cognitive Science 2007, Gerzensee, Switzerland

Awards and Prices

DIRAC researchers have won a number of prestigious awards and prices

• 2011 Most Cited Paper award by the J. on Computer Vision and Image Understanding was awarded to the ETHZ paper on SURF, a feature investigated at the start of DIRAC (WP1).
• Best paper award was awarded to: D. Hollosi, J. Schröder, S. Götze, J.-E. Appell, "Voice Activity Detection Driven Acoustic Event Classification for Monitoring in Smart Homes", 3rd International Symposium on Applied Sciences in Biomedical and Communication Technologies, Rome, Italy, November 2010.
  
• ITG Innovation award for DIRAC researchers, December 18, 2009: Innovation Prize ITG 2009 handed out by Electrosuisse for the submission `quo vadis? - from video to motion pattern', by M. Breitenstein, E. Koller-Meier, F. Reichlin, L. Van Gool, work on pedestrian tracking with a single camera
• Best Paper Award at 2009 IEEE International Conference on Robotics and Automation, May12 to 17, 2009: Best vision paper award, Int. Conf. on Robotics and Automation, for the paper entitled `moving obstacle detection in highly dynamic scenes', by A. Ess, B. Leibe, K. Schindler, L. van Gool, an application of the Awear setup on a mobile platform.
• Best Video Award: KULeuven and ETHZurich received the Best Video Award at the International Conference on Computer Vision and Pattern Recognition 2006 in New York, USA.

Publications

DIRAC has resulted in approximately 300 scientific publications, including one book volume, almost 10 percent of which are joint partner publications on collaborative work on the DIRAC theme.

Additional joint partner manuscripts are currently in the review process.