#StackBounty: #javascript #neural-network #genetic-algorithm Avoid local optima training XOR

Bounty: 50

I’m using Gynaptic to train a neural network that solves an XOR with a genetic algorithm. The fitness is defined as follows:

// max score = 0
score -= Math.abs(0 - network.activate([0, 0])) * 5000;
score -= Math.abs(1 - network.activate([1, 0])) * 5000;
score -= Math.abs(1 - network.activate([0, 1])) * 5000;
score -= Math.abs(0 - network.activate([1, 1])) * 5000;

Sometimes it runs just fine. But i’m assuming it just lucked out then. But quite often, it won’t even reach -6000. It will hang around -8000 a lot.

These are my settings:

  GNN = new Evolution({
    size: 100,
    elitism: 10,
    mutationRate: 0.3,
    networkSize : [2,3,1],
    mutationMethod: [
      Methods.Mutate.MODIFY_RANDOM_BIAS,
      Methods.Mutate.MODIFY_RANDOM_WEIGHT,
      Methods.Mutate.SWAP_BIAS,
      Methods.Mutate.SWAP_WEIGHT,
      Methods.Mutate.MODIFY_SQUASH
    ],
    crossOverMethod: [
      Methods.Crossover.UNIFORM,
      Methods.Crossover.AVERAGE,
      Methods.Crossover.SINGLE_POINT,
      Methods.Crossover.TWO_POINT
    ],
    selectionMethod: [
      Methods.Selection.FITNESS_PROPORTIONATE
    ],
    generationMethod: [
      Methods.Generation.POINTS
    ],
    fitnessFunction: function(network){
      var score = 0;

      score -= Math.abs(0 - network.activate([0, 0])) * 5000;
      score -= Math.abs(1 - network.activate([1, 0])) * 5000;
      score -= Math.abs(1 - network.activate([0, 1])) * 5000;
      score -= Math.abs(0 - network.activate([1, 1])) * 5000;

      return Math.round(score);
    }
  });

(view the JSFiddle here, open console and press train)

What settings do you advise me to change?

P.S. I understand that training an XOR is much easier through backpropagation, but this is just for experimental purposes.


Get this bounty!!!