1) Review of Bayes decision theory. Bayes theorem, optimality of Bayes decision rule, rewriting of the discriminant functions into equivalent forms, case studies.
2) Review of artificial neural networks (ANN). Definitions, MLPs and deep architectures, supervised learning, mixtures of experts, autoencoders, application to pattern recognition, universality, estimation of class-posterior probabilities, estimation of scaled-likelihoods, radial basis functions.
3) Parametric estimation techniques. Equivalence between supervised and unsupervised setups. Maximum likelihood (ML) approach. Mixture densities and GMMs. From GMMs to k-Means clustering to competitive neural nets.
4) Nonparametric estimation. General framework. Parzen window, kn-nearest neighbor. Examples. Pros and cons.
5) Density estimation via Parzen neural networks (PNN). Training algorithm. Practical matters, model selection via cross-validated likelihood, application to pattern classification. Overview of the theoretical properties of PNNs (complexity, modeling capabilities, asymptotic convergence in probability). Graphical demos. Application to sex determination from human crania.
6) Nonparametric pdf estimates via soft-constrained ANNs that satisfy Kolmogorov's axioms of probability. Markov Chain Monte Carlo solution to the computation of the numeric integral of the ANN; technique for sampling from the ANN. Results of simulations.
7) Neural Mixture Models (NMM) for the estimation of mixture densities. The relevance of mixture densities, difference between mixture densities and mixture density models, ML soft-constrained training of the NMM. Results of simulations.
8) Sequence processing: hard-constrained RBF-based ML density estimation over sequences of patterns encoded via the Echo State Network. Application to emotion recognition from speech signals.
9) Graph processing: hard-constrained RBF-based ML density estimation over graphs (i.e., structured patterns, or relations) encoded via the recursive/graph neural networks. Applications to density estimation over graphs, graph clustering, and graph classification.
