Training procedure which preserves symmetry next up previous
Next: A wider window in Up: Suggestions for further work Previous: Use of Fourier transform

Training procedure which preserves symmetry

We can use a training procedure which preserves symmetrical distribution of weights in the Hartley or Fourier transform stages. In addition to the improved shift invariance, this approach can lead to parameter reduction. The procedure starts by noting the equal weights at initialization. Then the forward and backward passes are performed as usual. But in updating we use the same weight update for all the equal weights, namely the average value of all the weight updates corresponding to the equal weights. In this way we can preserve any existing symmetry in the initial weight distributions. At the same time number of parameters is reduced because only one parameter is needed to represent the whole class of equal weights.

Narada Warakagoda
Fri May 10 20:35:10 MET DST 1996

Home Page