We can use a training procedure which preserves symmetrical distribution of weights in the Hartley or Fourier transform stages. In addition to the improved shift invariance, this approach can lead to parameter reduction. The procedure starts by noting the equal weights at initialization. Then the forward and backward passes are performed as usual. But in updating we use the same weight update for all the equal weights, namely the average value of all the weight updates corresponding to the equal weights. In this way we can preserve any existing symmetry in the initial weight distributions. At the same time number of parameters is reduced because only one parameter is needed to represent the whole class of equal weights.