2008 IAPR Workshop on
Cognitive Information Processing

June 9-10, 2008, Santorini, Greece

CIP 2008 logo

Program for First IAPR Workshop on Cognitive Information Processing
Day Time Session
Mon 08:00 AM-08:50 AM Plenary talk: The Cubature Kalman Filter: An Essential Tool for Cognitive Dynamic Systems
  09:00 AM-10:40 AM M1: Cognitive Radio
  11:00 AM-11:50 AM Plenary talk: Adaptive Networks
  12:00 PM-01:00 PM M2: Adaptive Algorithms
  02:30 PM-03:20 PM Plenary talk: Kernel Tricks, Means and Ends
  03:30 PM-04:10 PM M3: Data Representation and Analysis
  04:30 PM-06:00 PM P: POSTER
Tue 08:00 AM-08:50 AM Plenary talk: From Quantification of Information to Quantification of Meaning Using Socio-cognitive Computing
  09:00 AM-10:40 AM T1: Kernel Methods and Applications
  11:00 AM-11:50 AM Plenary talk: The Central Role of PDF Moments in Advanced Adaptive Filtering and Information Theories
  12:00 PM-01:00 PM T2: Cognitive Component Analysis - ICA
  02:30 PM-03:20 PM Plenary talk: Predictive Information and the Optimal Perception-Action Cycle
  03:30 PM-04:10 PM T3: Speech Processing and Recognition

Monday, Jun 9

8:00 AM - 8:50 AM

Plenary talk: The Cubature Kalman Filter: An Essential Tool for Cognitive Dynamic Systems

Simon Haykin (McMaster University, Canada)

9:00 AM - 10:40 AM

M1: Cognitive Radio

Ana I. Pérez-Neira (Polytechnic University of Catalonia, Spain)
9:00 Spectrum Sensing in IEEE 802.22
Stephen Shellhammer (Qualcomm, USA)
The IEEE is developing a standard for cognitive wireless regional area networks operating in unused television channels. One of the cognitive features of this standard is spectrum sensing, which is used to identify unused television channels. This paper describes the requirements for spectrum sensing and the spectrum sensing framework. A description is given of the method used for evaluating specific spectrum sensing techniques. Finally, the paper provides a survey of the spectrum sensing techniques that are included in the draft standard.
9:20 Spectrum Labeling for cognitive radio systems: candidate spectral estimation
Ana I. Pérez-Neira (Polytechnic University of Catalonia, Spain); Miguel Angel Lagunas (Telecommunications Technological Center of Catalonia, Spain)
A key challenge of the physical architecture of the cognitive radio is an accurate detection of weak signals of licensed users over a wide spectrum range. This paper describes a method for detection and frequency location of a given primary user, even when a non-candidate interferer is located at the same frequency. The range of SNR covered proves that the estimate is efficient for realistic scenarios providing accurate detection even in locations where proper reception of the candidate information is not possible (below 10 dB). All together, the performance is evidenced for very short data records (50 symbols of the candidate signal). The proposed technique shows much better performance than energy detectors and less complexity than ciclo-stationary based ones.
9:40 How much learning is sufficient in interference games?
Yi Su (University of California, Los Angeles, USA); Mihaela van der Schaar (University of California, Los Angeles (UCLA), USA)
This paper studies the learning behavior of self-interested users interacting in a two-user OR-channel interference game. We discuss how a strategic user should learn the behavior of its opponent, adapt its actions, and improve its own performance. Specifically, we investigate the trade-off that can be made by a user between learning duration and performance, if the opponent plays a mixed strategy. First, we assume a stationary opponent and we apply optimization theory and large deviations theory to analytically derive an upper bound of the minimum training required by the user given the tolerable performance loss and outage probability. Next, we extend the results to the cases, where an adaptive opponent plays a conditional strategy based on its bounded memory. By solving linear programs, we design optimized learning strategies which minimize an upper bound of the duration of learning against the adaptive opponent.
10:00 Distributed Mode Classification in Embodied Cognitive Radio Terminals
Andrea Cattoni (University of Genova, Italy); Irene Minetti (University of Genoa, Italy); Carlo Regazzoni (University Of Genova, Italy)
In the last decade optimal radio resources allocation has become a problem requiring more complex solutions, including the possibility to think to re-configurable, adaptive use of spectrum. A cognitive approach to radio spectrum management has been proposed as a suitable and potentially efficient solution. In this paper, a Distributed Mode Classification problem is here considered as a reference approach on which to propose new results and a framework of development of future research. Such approach strictly relies on the definition of a Cognitive (Radio) system as a system capable of environmental interactions. The Embodied Cognition approach, for the design of new CR terminals, here followed is based on inspiration from works in AI looking at intelligence as to an emergent behavior of a set of (computational) entities provided of possibility of active interaction with the surrounding environment. In this domain, distributed spectrum sensing is seen as a perception capability of a CR terminal strictly related to the motion action to optimize its capability to optimally perform mode classification.

11:00 AM - 11:50 AM

Plenary talk: Adaptive Networks

Ali Sayed (UCLA)

12:00 PM - 1:00 PM

M2: Adaptive Algorithms

João Romano (DSPCom-Unicamp: Digital Signal Processing for Comm. Lab., State University of Campinas, Campinas, Br, Brazil)
12:00 Extended recursive least squares in RKHS
Weifeng Liu (University of Florida, USA); Jose Príncipe (University of Florida, USA)
In this paper, a kernelized version of the extended recursive least squares (Ex-RLS) algorithm, along with its Kalman filter interpretation will be presented. The center piece of this development is a reformulation of the Ex-RLS algorithm which only requires inner product operations between input vectors. Thus, the kernel trick can be readily applied to obtain nonlinear versions in reproducing kernel Hilbert spaces (RKHS). In so doing, we arrive at extended RLS algorithms with kernels that are better suited for tracking the state-vector of general linear state-space models in the feature space, when compared with a fixed state model in the standard recursive least squares. The proposed kernel Ex-RLS is applied to a nonlinear Rayleigh multipath channel tracking problem. We show that the proposed algorithm is able to outperform the standard kernel RLS in a fading environment.
12:20 Optimal Sliding Window Sparsification for Online Kernel-Based Classification by Projections
Konstantinos Slavakis (University of Peloponnese, Greece); Sergios Theodoridis (University of Athens, Greece)
This paper presents a new sparsification method for a very recently introduced projection-based algorithm for the online classification task in Reproducing Kernel Hilbert Spaces (RKHS). To accommodate limited computational resources, sparsification is achieved by a sequence of finite dimensional subspaces, with dimensions upper bounded by a predefined buffer length. In the case of a buffer overflow, the term that contributes the least to the kernel series expansion is removed. Such a sparsification scheme shows strong similarities with the classical sliding window adaptive schemes. We validate the proposed design by considering the adaptive equalization problem of a nonlinear communication channel. Since the fundamental tool of metric projections is used, and although a classification problem is considered here, the method can be readily extended to regression tasks, and to cost functions that are in general non-differentiable.
12:40 Diffusion Strategies for Distributed Kalman Filtering: Formulation and Performance Analysis
Federico Cattivelli (University of California, Los Angeles, USA); Cassio Lopes (University of California, Los Angeles, USA); Ali Sayed (University of California, Los Angeles, USA)
We consider the problem of distributed Kalman filtering, where a set of nodes are required to collectively estimate the state of a linear dynamic system from their individual measurements. Our focus is on diffusion strategies, where nodes communicate with their direct neighbors only, and the information is diffused across the network. We derive and analyze the mean and mean-square performance of the proposed algorithms and show by simulation that they outperform previous solutions.

2:30 PM - 3:20 PM

Plenary talk: Kernel Tricks, Means and Ends

Bernhard Schölkopf (Max Planck Institute, Germany)

3:30 PM - 4:10 PM

M3: Data Representation and Analysis

Steve McLaughlin (University of Edinburgh, United Kingdom)
15:30 Empirical Mode Decomposition based denoising techniques
Yannis Kopsinis (University of Edinburgh, School of Engineering and Electronics, United Kingdom); Steve McLaughlin (University of Edinburgh, United Kingdom)
One of the most challenging tasks for which EMD could be useful is that of non-parametric signal denoising, an area in which wavelet thresholding has been the dominant technique for many years. In this paper, the major wavelet thresholding principle is used in the decomposition modes resulting from applying EMD to a signal. We show, that although a direct application of this principle in the EMD case is not feasible, it can appropriately adapted by exploiting the special characteristics of the EMD decomposition modes. In the same manner, inspired by the translation invariant wavelet thresholding, a similar technique adapted to EMD is developed leading to enhanced denoising performance.
15:50 Linear Dimensionality Reduction with Gaussian Mixture Models
Jose Leiva Murillo (Universidad Carlos III de Madrid, Spain); Antonio Artés Rodríguez (Universidad Carlos III de Madrid, Spain)
We explore the application of several information-theoretic criteria to the problem of reducing the dimension in pattern recognition. We consider the use of Gaussian mixture models for estimating the distribution of the data. Three algorithms are proposed for linear feature extraction by the maximization of the mutual information, the likelihood or the hypotheses test, respectively. The experiments show that the proposed methods outperform the classical methods based on parametric Gaussian models, and avoid the intense computational complexity of nonparametric kernel density estimators.

4:30 PM - 6:00 PM

P: POSTER

Danilo Mandic (Imperial College, London, UK, Algeria)
The Augmented Complex Least Mean Square Algorithm with Application to Adaptive Prediction Problems
Soroush Javidi (Imperial College, United Kingdom); Maciej Pedzisz (Ensieta, France); Su Lee Goh (Shell Exploration and Production, The Netherlands); Danilo Mandic (Imperial College, London, UK, Algeria)
An augmented complex least mean square (ACLMS) algorithm for complex domain adaptive filtering which utilises the full second order statistical information is derived for adaptive prediction problems. This is achieved based on some recent advances in complex statistics and by using widely linear modelling in the complex domain. This way, both circular and non--circular complex signals can be processed optimally, using the same model. Simulations on complex--valued wind field and on a complex autoregressive process show the effectiveness of this approach as compared to the standard Complex LMS algorithm.
Signal Modality Characterisation Using Collaborative Adaptive Filters
Beth Jelfs (Imperial College London, United Kingdom); Danilo Mandic (Imperial College, London, UK, Algeria)
A method for extracting information (or knowledge) about the nature of a signal is presented, this is achieved by tracking the dynamics of the mixing parameter within a hybrid filter rather than the actual filter performance. Implementations of the hybrid filter for tracking the nonlinearity and the sparsity of a signal are illustrated and simulations on benchmark synthetic data in a prediction configuration support the analysis. It is then shown that by combining the information obtained from both hybrid filters it is possible to use this method to gain a more complete understanding of the nature of signals and changes in signal modality.
Adaptive Bayesian Equalizer with Superimposed Training for MIMO Channels
Rafael Krummenauer (University of Campinas, Brazil); Fabiano Chaves (University of Campinas (UNICAMP), Brazil); Rafael Ferrari (University of Campinas (UNICAMP), Brazil); Mario Uliani (CPqD, Brazil); João Romano (DSPCom-Unicamp: Digital Signal Processing for Comm. Lab., State University of Campinas, Campinas, Br, Brazil); Amauri Lopes (Unicamp, Brazil)
The Bayesian equalizer is implementable by a proper employment of a radial basis function (RBF) neural network, with the inverse filtering problem posed as a classification problem. The proposed approach allows that the transmission of information and the RBF training be accomplished in a simultaneous and uninterrupted way. Moreover, the channel estimation procedure remains an unimodal optimization problem. Simulation results confirm the effectiveness of the proposed MIMO equalizer.
Learning for Cross-layer Optimization
Fangwen Fu (University of California Los Angeles, USA); Mihaela van der Schaar (University of California, Los Angeles (UCLA), USA)
Cross-layer optimization solutions have been proposed in recent years to improve the performance of network users operating in a time-varying, error-prone wireless environment. However, these solutions often rely on ad-hoc optimization approaches with known environmental dynamics experienced at various layers by a user and violate the layered network architecture of the protocol stack. This paper presents a new theoretic foundation for cross-layer optimization, which allows each layer autonomously to learn the environmental dynamics, while maximizing the utility of the wireless user by optimally determining what information needs to be exchanged among layers. Hence, this cross-layer framework does not change the current layered architecture. The experimental results demonstrate that the proposed layered learning framework achieves near-optimal performance.
On the Performance of In-Band Sensing without Quiet Period in OFDM system
Dong Chen (Xidian University, P.R. China); Jiandong Li (Xidian University, P.R. China); Jing Ma (Xidian University, P.R. China)
In-Band sensing is an effective method of avoiding harmful interference with primary users in cognitive radio. However, the traditional quiet period during In-Band sensing will interrupt on-going transmission and import overheads. In this paper, we have the In-Band sensing method without quiet period(NoQP Sensing) in OFDM system discussed. Towards a general NoQP Sensing problem, we proved that the weighted energy detector is the optimal detector under Neyman-Pearson criterion. The performance of the general NoQP Sensing is researched as well as the performance of two NoQP Sensing approaches, Complementary Symbol Couple(CSC) and Self-Signal Suppression(SSS). Evaluations indicate that they have similar performance in IEEE802.22 environment.
Network Tomography, Delay Estimation & Bottleneck Link Discovery
Nick Johnson (University of Edinburgh, United Kingdom); John Thompson (University of Edinburgh, United Kingdom); Steve McLaughlin (University of Edinburgh, United Kingdom); Francisco Garcia (Agilent, United Kingdom)
An important issue in the measurement of networks is the ability to infer characteristics of internal network links from measurements made on end-to-end paths. It may be impractical in terms of equipment, time or cost to monitor each individual link but it is often feasible to monitor a number of existing paths. Provided there is enough data traffic through the different paths then it may be possible to infer some characteristics of each link. In this paper we compare two methods based on the method of moments and a Gaussian approximation for inferring the end-to-end delay distributions. This information can then be used to compute packet delay on any link in a network and then which link has the highest latency. This procedure is often termed bottleneck link discovery
An Optimal MMSE Fuzzy Predictor for SISO and MIMO Blind Equalization
Rafael Ferrari (University of Campinas (UNICAMP), Brazil); Ricardo Suyama (University of Campinas, Brazil); Renato Lopes (University of Campinas, Brazil); Romis Attux (State University of Campinas - Unicamp, Brazil); João Romano (DSPCom-Unicamp: Digital Signal Processing for Comm. Lab., State University of Campinas, Campinas, Br, Brazil)
The present work deals with the research of optimal solutions in unsupervised and nonlinear signal processing. The proposed framework is based on nonlinear prediction, to be implemented by a fuzzy filter structure. Our main result consists in establishing the optimality of the approach by showing the equivalence between the minimum mean square error estimator and the fuzzy predictor. The result is then applied in the contexts of SISO equalization and convolutive source separation (MIMO equalization). We also propose a strategy for the updating of the unsupervised nonlinear equalizer. Simulation results confirm the effectiveness of the proposal in SISO and MIMO scenarios.
Higher-Order Blind Estimation of Generalized Eigenfilters Using Independent Component Analysis
Toni Huovinen (Tampere University of Technology, Finland); Ali Shahed hagh ghadam (Tampere University of Technology, Finland); Mikko Valkama (Tampere University of Technology, Finland)
Assuming a noisy linear mixing model of source random variables or signals, maximizing the output signal-to-interference-and-noise-ratio (SINR) among linear transformations of observed data leads to solving the generalized eigenvalue problem. The explicit solution of the problem assumes the knowledge of the mixing coefficients and noise variance and, for this reason, is not a blind method as such. However, we show in this paper that the solution can be estimated blindly and directly using basic independent component analysis (ICA) designed for noise-free linear models. In addition, the theoretical and numerical results of the paper show that one of the most widely applied ICA algorithms, the equivariant adaptive source identification (EASI) algorithm, is, in practice, identical with SINR-maximizing generalized eigenfiltering, even though it does not use explicit knowledge of the mixing coefficients nor source and noise statistics.
Contour Level Estimation from Gaussian Mixture Models Applied to Nonlinear BSS
Jugurta Montalvao (Universidade Federal de Sergipe, Brazil); Jânio Canuto (UFS, Brazil)
Probability density function estimation from limited data sets is a classical problem in pattern recognition. In this paper, we propose a reformulation of the well-known nonparametric Parzen method as a parametrically regularized Gaussian Mixture Model, from which we can easily estimate density contour level. As an application illustration to the proposed contour level estimator, we also address the Blind Source Separation problem through the analysis of contour level distortions in joint probability density functions. Finally, we use the proposed estimator to undo a nonlinear mixture of two images.
Multiple-Channel Signal Detection using the Generalized Coherence spectrum
David Ramírez García (University of Cantabria, Spain); Javier Vía (University of Cantabria, Spain); Ignacio Santamaria (University of Cantabria, Spain)
Recently, a generalization of the magnitude squared coherence (MSC) spectrum for more than two random processes has been proposed. The generalized MSC (GMSC) spectrum definition, which is based on the largest eigenvalue of a matrix containing all the pairwise complex coherence spectra, provides a frequency-dependent measure of the linear relationship among several stationary random processes. Moreover, it can be easily estimated by solving a generalized eigenvalue problem. In this paper we apply the GMSC spectrum for detecting the presence of a common signal from a set of linearly distorted and noisy observations. Specifically, the new statistic for the multiple-channel detection problem is the integral of the square root of the GMSC, which can be estimated as the sum of the P largest generalized canonical correlations (typically P = 1 is enough in practice). Unlike previous approaches, the new statistic implicitly takes into account the spectral characteristics of the signal to be detected (e.g., its bandwidth). Finally, the performance of the proposed detector is compared in terms of its receiver operating characteristic (ROC) with the generalized coherence (GC) showing a clear improvement in most scenarios.
Automatic Recognition of Urban Environmental Sound Events
Stavros Ntalampiras (University of Patras, Greece); Ilyas Potamitis (Techological Education Institute of Crete, Greece); Nikos Fakotakis (University of Patras, Greece)
Computer audition is an evolving and relatively new researching field with a lot of new applications. It would be of great convenience to live in an environment that can change automatically based on its “auditory sense”. In this work we propose a novel framework for automatic recognition of urban soundscenes. Our system facilitates an hierarchical classification schema while the performance of two well known feature sets is compared. A new post- processing algorithm to enhance the discrimination quality of MPEG-7 features is proposed and shown to provide improved results. Our approach is examined utilizing a compact testing procedure while MPEG-7 LLDs reach higher recognition rates than MFCCs.
Recognition of human activities using Layered Hidden Markov Models
Serafeim Perdikis (Aristotle University of Thessaloniki, Greece)
Human activity recognition has been a major goal of research in the field of human - computer interaction. This paper proposes a method which employs a hierarchical structure of Hidden Markov Models (Layered HMMs) in an attempt to exploit inherent characteristics of human action for more efficient recognition. The case study concerns actions of the arms of a seated subject and depends on the assumption of a static office environment. The first layer of HMMs detects short, primitive motions with direct targets, while every upper layer processes the previous layer inference to recognize abstract actions of longer time granularities. The problem of unsupervised learning within the LHMM framework is also addressed, through automatic segmentation of raw data and hierarchical clustering of motion samples. Finally, the idea of context - aware HMM modeling is also introduced and future directions for its application are proposed. The results demonstrate the efficiency, the tolerance on noise interpolation and the high degree of person - invariance of the method.
Robust nonlinear adaptive network classification of anaesthesia
Rudolf Baumgart-Schmitt (University of Applied Sciences Schmalkalden, Germany)
To control the administration of hypnotics during operations it is necessary to classify the depth of anaesthesia in a robust and efficient way. The frontal EEG was selected as a feature source. Different populations of topologically optimized trained neural networks solved the problem of robust classification. Robustness in this context means that the performance of the classifier is independent of the agent administration strategies used to induce different depth of anaesthesia. Training and optimization of the neural networks were supported by genetic programming and simulated evolution. The results are compared to the performance of the BIS XP monitor. For this purpose we applied this monitor to all patients in the cooperating hospital and measured the frontal EEG in a parallel way. The anaesthetist used several autonomic parameters like heart rate and blood pressure to recognize the depth of anaesthesia. The performance of both approaches using the frontal EEG only has been measured by confusion matrices which represent the concordances and deviants between the scores of the anaesthetist and the results of the automatic procedure. Our approach led to higher degrees of concordances for all stages especially if the anaesthetic agent ketamine is included. The extension of the evaluated EEG frequency range improved the results for the difficult recognition of transitional stages.
Generalized Statistical Methods For Unsupervised Minority Class Detection in Mixed Data Sets
Cecile Levasseur (University of California, San Diego, USA); Uwe Mayer (Fair Isaac Corporation, USA); Brandon Burdge (University of California, San Diego, USA); Ken Kreutz-Delgado (University of California, San Diego, USA)
Minority class detection is the problem of detecting the occurrence of rare key events differing from the majority of a data set. This paper considers the problem of unsupervised minority class detection for multidimensional data that are highly nongaussian, mixed (continuous and/or discrete), noisy, and nonlinearly related, such as occurs, for example, in fraud detection in typical financial data. A statistical modeling approach is proposed which is a subclass of graphical model techniques. It exploits the properties of exponential family distributions and generalizes techniques from classical linear statistics into a framework referred to as Generalized Linear Statistics (GLS). The methodology exploits the split between the data space and the parameter space for exponential family distributions and solves a nonlinear problem by using classical linear statistical tools applied to data that has been mapped into the parameter space. A fraud detection technique utilizing low-dimensional information learned by using an Iteratively Reweighted Least Squares (IRLS) based approach to GLS is proposed in the parameter space for data of mixed type. ROC curves for an initial simulation on synthetic data are presented, which gives predictions for results on actual financial data sets.
A Novel Data Fusion Approach using Two-Layer Conflict Solving
Rui Li (Ostwestfalen-Lippe University of Applied Sciences, Germany); Volker Lohweg (Ostwestfalen-Lippe University of Applied Sciences, Germany)
A Two-Layer Conflict Solving data fusion approach is pro-posed in this work, with an aim to provide another approach to data fusion community. Since the evidence of Dempster-Shafer Theory, algorithms for combining pieces of evidence have drawn a considerable attention from data fusion researchers, along with many alternatives invented. However, none of these approaches receives an agreement for being able to perform very successfully in all scenarios and hence this topic is still in hot discussion. Therefore, the suggested approach in this work will contribute as a novel method and present its own merits.
Theory of Genetic Algorithms with $\alpha$-Selection
Andre Neubauer (Muenster University of Applied Sciences, Germany)
Genetic algorithms are random heuristic search (RHS) algorithms for adaptive systems with a wide range of applications in search, optimisation, pattern recognition and machine learning as well as signal processing. Despite their widespread use a general theory is still lacking. A promising approach is offered by the dynamic system model which describes the stochastic trajectory of a population under the dynamics of a genetic algorithm with the help of an underlying deterministic heuristic function and its fixed points. However, even for the simple genetic algorithm (SGA) with fitness-proportional selection, crossover and mutation the determination of the population trajectory and the fixed points of the heuristic function is unfeasible for practical problem sizes. In order to simplify the mathematical analysis $\alpha$-selection is introduced in this paper. Based on this strong selection scheme it is possible to derive the dynamic system model and the respective fixed points in closed form. In addition to the theoretical analysis experimental results are presented.
Evolving Neural Networks Ensembles NNEs
Hany Sallam (Phd. Student, Genova University, Italy)
A new method to design and evolve neural network ensembles NNEs based on speciation is presented in this paper. The main advantage of this method is that, it completely evolves NNEs by combining the evolution of neural networks and the configuration of the ensemble in one evolutionary phase. In every generation, population is evolved toward the best set of structure and weights then, the ensemble is configured and its performance is evaluated. Evolution is stopped if the best performance is reached or the maximum number of generations is reached. The main idea of this method is to generate NNE based on fitness sharing and genotype diversity measure. The size of the ensemble is depending on the number of species. The output of the ensemble is calculated by the weighted sum of the output of each member. The members weights are changed dynamically from generation to generation depending on the characteristics of species exist in the current population. Experiments with Iris data and breast cancer data from the UCI machine learning repository showed that the proposed method can produce NNEs with comparable performance compared to other methods.
A Mutual Information-based Method for the Estimation of the Dimension of Chaotic Dynamical Systems Using Neural Networks
Chistos Chatzinakos (Aristotle University, Greece); Nikos Kofidis (University of Macedonia, Greece); Athanassios Margaris (ATEI of Thessaloniki, Greece); Konstantinos Tsouros (Aristotle University, Greece)
In this paper, a method of estimating the dimension of dynamical systems from a time series, using neural networks, is examined. It is based (a) on the hypothesis that a member of a time series can be optimally expressed as a deterministic function of the $d$ past series values (where d is the dimension of the system), and (b) on the observation that neural networks'learning ability is improved rapidly when the appropriate amount of information is provided to a neural structure which is as complex as needed. To estimate the dimension of a dynamical system, neural networks are trained to learn the component of the attractor expressed by a reconstructed vector in a suitable phase space whose embedding dimension m, has been estimated using the mutual information method. More specifically, the information supplied to the networks is represented by vectors consisting of the m past values of the time series, where m varies from 1 to D+2, D being a pre-estimation for the maximum value of the embedding dimension of the system. The current method proposes that when m meets the dimension d of the dynamical system, the neural model of the attractor remarkably improves its learning ability, minimizing locally the RMS error of the training set. The logistic and the Henon map as well as the Lorenz and the Rosler attractors expressed as systems of difference equations, were examined to test the validity of the method.
A Contributin to specification toward truly autonomous robots
Giovanna Morgavi (National Research Council, Italy)
A great deal of current research work in robotics and autonomous systems is still focused on getting an agent to learn to do some task such as recognizing an object or going to a specific place. The learning process may be supervised, unsupervised or a process of occasional reinforcement, but the whole aim in such work is to get the robot to achieve the task that was predefined by the researcher. The next logical step along the road towards truly autonomous robots that can dive in unpredictable environments is to investigate how one might design robots that are capable of `growing up' through experience. A living artifact grows up when its capabilities, abilities/knowledge, shift to a further level of complexity, i.e. the complexity rank of its internal capabilities performs a step forward. In this paper we studied the modalities through which pre-school children (from 4 to 5) tackle with a growing up process: he abstraction. Children of these ages are not supposed to be able to build an abstraction, but they have a knowledge of the natural language that allow the description of the processes they are using to reach the meaning. This experiment resulted in some very interesting suggestions for the architecture of an adaptive and evolving robot. The importance of multi-sensor perception, motivation and emotional drives are underlined and, above all, the growing up insights shows similarities to emergent self-organized behaviors.
Mobile Robot Localization in Indoor Environment using Scale-Invariant Visual Landmarks
Soon Young Park (Kyungpook National University, Korea); Suk Chan Jung (Kyungpook National University, Korea); Young Sub Song (Kyungpook National University, Korea); Hang Joon Kim (Kyungpook National University, Korea)
This paper presents a three-dimensional mobile robot localization system using visual landmarks. We use the scale-invariant feature points as visual landmarks. This feature is independent of camera view point, thus it is proper to use the landmark. The keypoints detected by SIFT are invariant to scale change and rotations, thus we use the keypoint as a landmark. Since the image coordinates for the landmarks are projected depending on the camera pose, the camera pose is determined using the relation between the two-dimensional image coordinates and three-dimensional world coordinates for the landmarks. The camera pose is considered the same as the robot pose, as the camera taking the images is fixed to the robot. The inclusion of falsely detected landmarks has an adverse effect on the accuracy of the robot localization. Therefore, the proposed method estimates the robot pose, while eliminating any falsely detected landmarks. To evaluate the proposed method, experiments are performed using a mobile blimp robot in an indoor environment. The results confirm that the proposed method can estimate the robot pose with a good accuracy.
Toward Agents That Can Learn Nonverbal Interactive Behavior
Yasser Mohammad (Kyoto University, Japan); Toyoaki Nishida (Kyoto University, Japan)
Humans are social agents and the social dimension is an important aspect of human cognition. One challenge facing the realization of artifacts and artificial agents that posses human-like cognition abilities is to implement human-like interactive capabilities into them. Natural Language Processing is one of the earliest applications of AI techniques because of the importance of language in shaping human cognitive and interactive capabilities. Nevertheless nonverbal communication is starting to gain more importance specially in the domains of HRI and ECA because natural human-human communications are known to utilize a variety of nonverbal interaction protocols. This paper proposes a new adaptation algorithm for interactive agents that aims to develop agents that can learn and adapt their theory of mind concerning nonverbal interaction in real-time during actual interactions. The proposed method utilizes elements of the theory of theory and the theory of simulation to guide the adaptation process. A proof of concept simulation experiment with the proposed system is also illustrated.

Tuesday, Jun 10

8:00 AM - 8:50 AM

Plenary talk: From Quantification of Information to Quantification of Meaning Using Socio-cognitive Computing

Timo Honkela (Helsinki University of Technology, Finland)

9:00 AM - 10:40 AM

T1: Kernel Methods and Applications

David Miller (Penn State University, USA)
9:00 Topranking : Predicting the Most Relevant Element of a Set
Kristiaan Pelckmans (Leuven University, Belgium); Johan Suykens (KULeuven, Belgium)
This short paper concerns the task of identifying the element of a set which is probably the most useful, based on previous {\em incomplete} experiments on similar tasks. It is shown that this problem can be solved effectively using a quadratic program, while a probabilistic guarantee is given that such a prediction will solve the problem on the average. We comment on the relation and difference of this setting with amongst others the structured output learning model, transductive inference and the multi-task learning setting. Finally, a number of immediate applications are described.
9:20 Margin-based feature selection techniques for support vector machine classification
Yaman Aksu (The Pennsylvania State University, USA); David Miller (Penn State University, USA); George Kesidis (Pennsylvania State University, USA)
Feature selection for classification in high-dimensional feature spaces can improve generalization accuracy, reduce classifier complexity, and is also useful for identifying the important feature "markers", e.g. biomarkers in a bioinformatics context. For SVM classification, a widely used technique is recursive feature elimination (RFE). In recent work, we demonstrated that the RFE objective is not consistent with the margin maximization objective central to SVM learning. We proposed explicit margin-based feature elimination (MFE) for SVMs. In this paper, after reviewing MFE, we first introduce an extension which achieves further gains in margin at small computational cost. This extension solves the SVM problem, albeit in a lightweight fashion by optimizing only two degrees of freedom -- the weight vector slope and intercept. We next consider the case of a nonlinear kernel. We show that RFE assumes that the weight vector length is strictly decreasing as features are eliminated. We demonstrate that this assumption is not valid for the Gaussian kernel and that, consequently, RFE may give poor results for this case. An extension of MFE for nonlinear kernels gives both better margin and generalization accuracy.
9:40 Error Control Coding based on Support Vector Machine
Johnny Kao (University of Auckland, New Zealand); Stevan Berber (University of Auckland, New Zealand)
A novel approach of decoding convolutional codes using a multi-class support vector machine is presented in this paper. Support vector machine is a recently developed and well recognized algorithm for constructing maximum margin classifiers. Unlike traditional adaptive learning approaches such as a multi-layer neural network, it is able to converge to a global optimum solution, hence achieving a superior performance. However, up to this date so far, no work has yet been done on applying support vector machine on error control coding. In this investigation, decoding is achieved by treating each codeword as a unique class. Hence the decoding procedure becomes a multi-class pattern classification problem. Simulation results show that the bit error rate performance of decoder based on such approach compare favorably with a conventional soft decision Viterbi Algorithm under a noisy channel with additive white Gaussian noise and achieve an extra 2 dB coding gain over the conventional method in a Rayleigh’s fading channel.
10:00 Robust boundary learning for multi-class classification problems
Yoshikazu Washizawa (RIKEN, Japan); Seiji Hotta (Tokyo University of Agriculture and Technology, Japan)
The objective of pattern classification is minimizing generalization errors for innumerable unknown samples. In the structural risk minimization (SRM) principle, both empirical errors and complexities of classifiers are minimized instead of minimizing generalization errors. We define a criterion about both of empirical errors and complexities for multi-class classifiers directly, and propose a perceptron-based linear classifier obtained as the minimum solution of the criterion. Due to this direct measurement, our classifier is robust against outliers and mislabeled training samples. We discuss the advantages of our classifier by comparing with conventional classifiers such as support vector machines and neural networks. We verify classification ability of our classifier by experiments on benchmark datasets.
10:20 Integrating Motion and Color for Content-Based Video Classification
Markos Zampoglou (University of Macedonia, Greece); Theophilos Papadimitriou (Democritus University of Thrace, Greece); Konstantinos Diamantaras (Technological Education Institute of Thessaloniki, Greece)
How to achieve the goal of automatically classifying video shots by their content is still an issue under debate. In this paper we present a novel set of low-level descriptors for the classification of TV video shots into meaningful semantic classes which can then be useful when browsing a TV stations archives. The motion features we propose consist of a modified Perceived Motion Energy Spectrum descriptor for local motion and a Normalized Dominant Motion Histogram for camera motion. Since exclusively motion-based classification has a very limited applicability, we also add three normalized local HSV histograms, extracted from particular key-frames we select with a simple yet efficient approach, as color descriptors. Our experimental implementation is tested on real-world TV video shots using a binary classifier based on Support Vector Machines and the results demonstrate that the proposed features can achieve high success rates not only on narrow and specialized classes, but also on more generic ones.

11:00 AM - 11:50 AM

Plenary talk: The Central Role of PDF Moments in Advanced Adaptive Filtering and Information Theories

Jose Principe (University of Florida, Gainesville, USA)

12:00 PM - 1:00 PM

T2: Cognitive Component Analysis - ICA

Jan Larsen (Technical University of Denmark, Denmark)
12:00 A multimodal approach for frequency domain blind source separation for moving sources in a room
Syed Mohsen Naqvi (Loughborough University UK, United Kingdom); Jonathon Chambers (Loughborough University, United Kingdom); Yonggang Zhang (Loughborough University, United Kingdom)
A novel multimodal approach for frequency domain blind source separation of moving sources is presented in this paper. A very simple and robust algorithm is proposed which incorporates geometrical information and exploits the permutation free unmixing matrix of the previous block together with the whitening matrix of the mixtures of the current block, to initialize FastICA for separation of moving sources. The advantages of this work are that no extra processing is required to solve the permutation problem separately in the frequency domain BSS nor is postprocessing required. Experimental results show the significant improvement in the performance of the resulting intelligently initialized FastICA approach over conventional FastICA, and also confirm that the proposed algorithm is robust and potentially suitable for real time implementation for sources moving in the teleconference scenario.
12:20 On Phonemes as Cognitive Components of Speech
Ling Feng (Technical University of Denmark, Denmark); Lars Kai Hansen (Technical University of Denmark, Denmark)
COgnitive Component Analysis (COCA) defined as the process of unsupervised grouping of data such that the ensuing group structure is well-aligned with that resulting from human cognitive activity, has been explored on phoneme data. Statistical regularities have been revealed at multiple time scales. The basic features are 25-dimensional short time (20ms) mel-frequency weighted cepstral coefficients. Features are integrated by means of stacking to obtain features at longer time scales. Energy based sparsification is carried out to achieve sparse representations. Our hypothesis is ecological: we assume that features that essentially independent in a context defined ensemble can be efficiently coded using a sparse independent component representation. This means that supervised and unsupervised learning should result in similar representations. We indeed find that supervised and unsupervised learning seem to identify similar representations, here, measured by the classification similarity.
12:40 Recovery of images from a mixture with multiplicative noise
David Blanco (University of Granada, Spain)
The use of independent component analysis (ICA) in coherent images needs to take into account the presence of the multiplicative noise that exits in this kind of images. In this paper, the recovery of original images from a mixture contaminated with this type of noise is studied using the ICA ideas. The mixing matrix is obtained using the fourth order multiplicative ICA method, which extracts the mixture before removing the noise. The result is a noisy version of the original images, where the effect of other images is reduced. The quality of the images is finally improved with the used of a multiplicative noise removal method. The proposed approach is compared with the direct use of ICA method over the noisy mixture or a denoise version of it, using simulated images.

2:30 PM - 3:20 PM

Plenary talk: Predictive Information and the Optimal Perception-Action Cycle

Naftali (Tali) Tishby (The Hebrew University, Israel)

3:30 PM - 4:10 PM

T3: Speech Processing and Recognition

Eleftherios Kofidis (University of Piraeus, Greece)
15:30 Vocal-Tract Modeling for Speaker Independent Single Channel Source Separation
Michael Stark (Technical University Graz, Austria); Franz Pernkopf (Technical University Graz, Austria); Van Pham (TU of Graz, Austria); Gernot Kubin (Graz University of Technology, Austria)
In this paper, we investigate two statistical models for the source- filter based single channel speech separation task. We incorporate source-driven aspects by pitch estimation in the model-driven method which models the vocal-tract part as a priori knowledge. This approach results in a speaker independent (SI) source separation method. For modeling the vocal tract filters Gaussian mixture models (GMM) and non-negative matrix factorization are considered. For both methods, the final fusion of the source and filter parameters results in a reformulation of the models which finally are used for separation. Furthermore, for the GMM method we propose a new gain compensation and pitch adjustment method. Performance is evaluated and compared to the speaker dependent (SD) factorial Hidden Markov Model [1]. Although the SD method delivers the best quality our SI methods show promising results and possess a lower complexity in terms of used parameters.
15:50 Bio-inspired Broad-Class Phonetic Labeling
Pedro Gómez-Vilda (Universidad Politécnica de Madrid, Spain); José Ferrández-Vicente (Universidad Politécnica de Cartagena, Spain); Victoria Rodellar-Biarge (Universidad Politécnica de Madrid, Spain); Rafael Martínez-Olalla (Universidad Politécnica de Madrid, Spain); Cristina Muñoz-Mulas (Universidad Politécnica de Madrid, Spain); Agustín Álvarez-Marquina (Universidad Politécnica de Madrid, Spain); Luis Mazaira-Fernández (Universidad Politécnica de Madrid, Spain)
Recent studies have shown that the correct labeling of phonetic classes may help current Automatic Speech Recognition (ASR) when combined with classical parsing automata based on Hidden Markov Models (HMM). Through the present paper a method for Phonetic Class Labeling (PCL) based on bio-inspired speech processing is described. The methodology is based in the automatic detection of formants and formant trajectories after a careful separation of the vocal and glottal components of speech and in the operation of CF (Characteristic Frequency) neurons in the cochlear nucleus and cortical complex of the human auditory apparatus. Examples of phonetic class labeling are given and the applicability of the method to Speech Processing is discussed.