• Neural network types:

- MLP - feed-forward multi-layer perceptron;
- RMLP - recurrent multi-layer perceptron (back-propagation through time with teacher forcing);
- Cascade-Correlation based on S. Fahlman papers (+simplified version).

• Training algorithms:

- standard steepest-descend with momentum term (off-line and on-line training);
- conjugate gradients (+2 modifications);
- quick-prop;

- automated training stop;
- dynamic network size adjustment:
   - smart insertion of new hidden units;
   - pruning of twin, dead and constant hidden units;
- Optimal Brain Surgeon (OBS) for weights elimination (based on the original paper by B. Hassibi et al.);
all structure modifications are safe to the network state (no error increase should be observed).

• Activation functions:

- standard sigmoid (logistic);
- 0-centered sigmoid;
- hyperbolic tangent;
- Elliott function (+unipolar version);
- arcus tangent (scaled to unipolar and bipolar);
- linear;

• Error functions:

- standard MSE;
- Pow4 and integrated hyperbolic arcus tangent for improved sensitivity on network error distribution tails;
- integrated hyperbolic tangent for training data polluted with outliers and gross errors;
- various asymmetric functions for different costs of sig->bkg and bkg->sig misidentification;

• Preprocessing:

- normalization (scaling) to 0-mean and unitary standard deviation;
- QSVD / ICA transformations - elimination of redundant data dimensions and better data representation;
- FFT filtering;

Standard classification algorithms:

- kNN (k-nearest neighbors);
- SVM classification and regression based on LIBSVM v2.6 library;
- probability density estimation;

back to the main page