Public Deliverables

Workpackage 1: Sensors

 D1.1 Omni-directional Sensor, and Features for Tracking and 3D Reconstruction
 D1.2 Feature Detectors for Body Parts, Tracking, and Fast Object Detection
 D1.3 Acoustic Features that Employ Limited Frequency Ranges and Longer Temporal Spans
 D1.4 Omnidirectional Image Acquisition and Processing Using High-Level Information
Head-Geometry Beamformerwith Binaural Output and its Perceptual Quality Assessment
 D1.7 Low level processing modules for AWEAR demonstrator 
 D1.8 Incongruence detection for detecting, removing and repairing incorrect functionality in low level processing

Workpackage 2: Audio Representations

 D2.1 Data Recorded from Different Acoustic Environments
 D2.2 Recordings of Spectro-Temporal Receptive Fields (STRFs) from Gerbil Auditory Cortex
 D2.3 Features of Audio Signals, Obtained with Different Modeling Techniques
 D2.4 Results of Processing with Advanced Hierarchical Model Including Results on an Accepted ASR Task
Acoustic Classification Method
 D2.6 Detection of Unexpected Words In Machine Recognition of Speech


Workpackage 3: Visual Representations

 D3.1 Framework for Bottom-Up 3D Reconstruction
 D3.2 Setup and Stimuli for Neurophysiological Experiments
 D3.3 Conclusions from the First Neurophysiological STP Single-Cell Experiments
 D3.4 Biological Motion Detection Based on the Learned, Statistical Representations
Dynamic 3D Scene Analysis Using Cognitive Loops
 D3.6 Results on Biological and Computational Action Recognition
Dynamic 3D Scene Analysis on Omni-Directional Video Data
Camera tracking and autocalibration for detecting and correcting camera de-calibration
Incongruences detected between trackers working with weaker and stronger expectations about the world
Report on single STS neurons to stimuli showing walking and running, as well as rare events, and qualitative comparison with conclusions from computational models for the same stimuli
Qualitative comparison of neurophysiological findings and computation models for the same stimuli
Fully functional implementation of a tracker tree pre-learned for a specific location and tested there, i.c. the OHSU Living Lab
Concepts for self-learning tracker trees available as a report


Workpackage 4: Learning and Categorization

 D4.1 Evaluation of Localization Algorithm and Retrieval
 D4.2 Prototype of the VR Environment
 D4.3 Learning-induced plasticity in cortical receptive fields
 D4.4 Test & evaluation of motion boundary detection algorithm
Conclude Categorization Study with PEC/NEC, Design Developmental Study, Feature Category Relationship
 D4.6 Categorization of Level and/or Temporal Dynamics
 D4.7 Object Detection from Small Sample, Optimization
Adaptive Distance Learning and Classification
Neuronal Correlates of Categorization
Learning-induced plasticity in cortical dynamics
Information Transfer between Categories & Hierarchy Learning
Incremental Learning with Bounded Memory Growth
Modeling cortical cells with distance functions
Transfer of knowledge for learning of rare events
Adaptive learning of hierarchies
Current source density analysis of auditory cortex


Workpackage 5: Multi-Sensory Information Fusion

 D5.1 Setup of experimental paradigm for studying audio-visual fusion in rodent cortex
 D5.2 Combination of Nonlinear Classsification
 D5.3 Framework for Cognition-Based Fusion
 D5.4 Report on High Level (Categorical) Audio-Visual Integration Results
Example/Demo of Hierarchical Multimodal Fusion
 D5.6 Report on electrophysiological correlates of audio-visual temporal order judgments
Report on Knowledge Transfer in Artificial and Living Cognitive Systems
Discriminative methods for multi-cue classification
Measure of rareness of incongruent events


Workpackage 6: Integration

 D6.1 Application Scenario
 D6.2 Omnidirectional Camera Tracking
 D6.3 Annotated Bibliography Webpage
 D6.4 Website for Audio-Visual Data
Technical Report on Detection of Audio-Visual Rare Events
 D6.6 First audio-visual data collection with AWEAR and OHSU system
 D6.7 Database of low level incongruencies
Database of incongruent human locomotion
Database of recordings of scenario 1
Testing and validation plan
Catalogue of basic scenes containing incongruent events
Evaluation Results
The DIRAC databases


Workpackage 7: Training and Education

 M7.1 Bundled notes of the Ice Breaker Workshop
 M7.2 Establishment of Internship Programs (undergraduate & graduate)
Establishment of Liaison with other (European) Consortia - list of contact person
 D7.1 First DIRAC Summer School on Cognitive Engineering