Support Vector Machines, Neural Networks and Fuzzy Logic Models

LEARNING AND SOFT COMPUTING
Support Vector Machines, Neural Networks and Fuzzy Logic Models

Table Of Contents :-

1. Learning and Soft Computing: Rationale, Motivations, Needs, Basics

2. Support Vector Machines

3. Single-Layer Networks

4. Multilayer Perceptrons

5. Radial Basis Function Networks

6. Fuzzy Logic Systems

7. Case Studies

8. Basic Nonlinear Optimization Methods

9. Mathematical Tools of Soft Computing

Selected Abbreviations

Notes

References

Index

1. Learning and Soft Computing: Rationale, Motivations, Needs, Basics
1.1 Examples of Applications in Diverse Fields
1.2 Basic Tools of Soft Computing: Neural Networks, Fuzzy Logic
   Systems, and Support Vector Machines
   1.2.1 Basics of Neural Networks
   1.2.2 Basics of Fuzzy Logic Modeling
1.3    Basic Mathematics of Soft Computing
   1.3.1 Approximation of Multivariate Functions
   1.3.2 Nonlinear Error Surface and Optimization
1.4    Learning and Statistical Approaches to Regression and Classification
   1.4.1 Regression
   1.4.2 Classification
Problems
Simulation Experiments

2.     Support Vector Machines
2.1    Risk Minimization Principles and the Concept of Uniform Convergence
2.2    The VC Dimension
2.3    Structural Risk Minimization
2.4    Support Vector Machine Algorithms
   2.4.1 Linear Maximal Margin Classifier for Linearly Separable Data
   2.4.2 Linear Soft Margin Classifier for Overlapping Classes
   2.4.3 The Nonlinear Classifier
   2.4.4 Regression by Support Vector Machines
Problems
Simulation Experiments

3.      Single-Layer Networks
3.1     The Perceptron
   3.1.1 The Geometry of Perceptron Mapping
   3.1.2 Convergence Theorem and Perceptron Learning Rule
3.2   The Adaptive Linear Neuron (Adaline) and the Least Mean Square Algorithm
   3.2.1 Representational Capabilities of the Adaline
   3.2.2 Weights Learning for a Linear Processing Unit
Problems
Simulation Experiments

4.      Multilayer Perceptrons
4.1     The Error Backpropagation Algorithm
4.2     The Generalized Delta Rule
4.3    Heuristics or Practical Aspects of the Error Backpropagation Algorithm
   4.3.1 One, Two, or More Hidden Layers?
   4.3.2 Number of Neurons in a Hidden Layer, or the Bias-Variance Dilemma
   4.3.3 Type of Activation Functions in a Hidden Layer and the Geometry of
   Approximation
   4.3.4 Weights Initialization
   4.3.5 Error Function for Stopping Criterion at Learning
   4.3.6 Learning Rate and the Momentum Term
Problems
Simulation Experiments

5.      Radial Basis Function Networks
5.1    Ill-Posed Problems and the Regularization Technique
5.2    Stabilizers and Basis Functions
5.3    Generalized Radial Basis Function Networks
   5.3.1 Moving Centers Learning
   5.3.2 Regularization with Nonradial Basis Functions
   5.3.3 Orthogonal Least Squares
   5.3.4 Optimal Subset Selection by Linear Programming
Problems
Simulation Experiments

6.      Fuzzy Logic Systems
6.1    Basics of Fuzzy Logic Theory
   6.1.1 Crisp (or Classic) and Fuzzy Sets
   6.1.2 Basic Set Operations
   6.1.3 Fuzzy Relations
   6.1.4 Composition of Fuzzy Relations
   6.1.5 Fuzzy Inference
   6.1.6 Zadeh's Compositional Rule of Inference
   6.1.7 Defuzzification
6.2    Mathematical Similarities between Neural Networks and Fuzzy Logic Models
6.3    Fuzzy Additive Models
Problems
Simulation Experiments

7.      Case Studies
7.1    Neural Networks-Based Adaptive Control
   7.1.1 General Learning Architecture, or Direct Inverse Modeling
   7.1.2 Indirect Learning Architecture
   7.1.3 Specialized Learning Architecture
   7.1.4 Adaptive Backthrough Control
7.2    Financial Time Series Analysis
7.3    Computer Graphics
   7.3.1 One-Dimensional Morphing
   7.3.2 Multidimensional Morphing
   7.3.3 Radial Basis Function Networks for Human Animation
   7.3.4 Radial Basis Function Networks for Engineering Drawings

8.      Basic Nonlinear Optimization Methods
8.1    Classical Methods
   8.1.1 Newton-Raphson Method
   8.1.2 Variable Metric or Quasi-Newton Methods
   8.1.3 Davidon-Fletcher-Powel Method
   8.1.4 Broyden-Fletcher-Go1dfarb-Shano Method
   8.1.5 Conjugate Gradient Methods
   8.1.6 Fletcher-Reeves Method
   8.1.7 Polak-Ribiere Method
   8.1.8 Two Specialized Algorithms for a Sum-of-Error-Squares Error Function
   Gauss-Newton Method
   Levenberg-Marquardt Method
8.2    Genetic Algorithms and Evolutionary Computing
   8.2.1 Basic Structure of Genetic Algorithms
   8.2.2 Mechanism of Genetic Algorithms

9.      Mathematical Tools of Soft Computing
9.1    Systems of Linear Equations
9.2    Vectors and Matrices
9.3    Linear Algebra and Analytic Geometry
9.4    Basics of Multivariable Analysis
9.5    Basics from Probability Theory