0

I'm trying implement simple Neural network on C++. But,while testing it with HOR it always give output near 0.5. I try to change learning rate,momentum,value of bies neuron, but nothing helps.

Network by itself it's two-dimensional vector of structures (Neuron)

struct Neuron
{
    vector<double> weights;
    vector<double> deltaWeights;
    vector<double> gradient;
    double value;
    double delta;
};

typedef vector<Neuron> Layer;

class Network
{
    public:
    Network(vector<unsigned>& Topology);
    double Sigmoid(double& x) { return 1 / (1 + exp(-x)); };
    double SigmoidDerivative(double& x) { return Sigmoid(x) * (1 - Sigmoid(x)); };
    double RandomWeights() { return ((double)rand() / RAND_MAX); };
    double ErrorFactor(vector<double>& TargetVals);
    void FeedForward(vector<double>& inputVals);
    void BackPropagation(vector<double>& targetVals);
private:
    vector<Layer> Net;
    const double alpha = 0.15; //learning rate
    const double epsilon = 0.3; //momentum
};

I tried to FeedForward with const weights, and i think error in backpropagation algorithm

BackPropagation algorithm:

void Network::BackPropagation(vector<double>& targetVals)
{
    for (unsigned i = 0; i < Net.back().size() - 1; i++) //delta for output neurons
    {
        Net.back()[i].delta = (targetVals[i] - Net.back()[i].value)  * SigmoidDerivative(Net.back()[i].value);
    }
    for (int i = Net.size()-2; i >= 0; i--) //delta for hiden neurons
    {
        for (unsigned j = 0; j < Net[i].size(); j++)
        {
            Net[i][j].delta = 0;

            for (unsigned l = 0; l < Net[i][j].weights.size(); l++)
            {
                Net[i][j].delta += (Net[i][j].weights[l] * Net[i + 1][l].delta);
            }

            Net[i][j].delta *= SigmoidDerivative(Net[i][j].value);
        }
    }

    for (unsigned i = 0; i < Net.size() - 1; i++) //gradient
    {
        for (unsigned j = 0; j < Net[i].size(); j++)
        {
            for (unsigned l = 0; l < Net[i][j].weights.size(); l++)
            {
                Net[i][j].gradient[l] = Net[i][j].value * Net[i + 1][l].delta;
            }
        }
    }

    for (unsigned i = 0; i < Net.size() - 1; i++) //changing weigths
    {
        for (unsigned j = 0; j < Net[i].size(); j++)
        {
            for (unsigned l = 0; l < Net[i][j].weights.size(); l++)
            {
                Net[i][j].deltaWeights[l] = (Net[i][j].gradient[l] * epsilon + Net[i][j].deltaWeights[l] * alpha);
                Net[i][j].weights[l] += Net[i][j].deltaWeights[l];
            }
        }
    }
}
  • 1
    And when you used your debugger to run your program, what did you see? This is what a debugger is for. If you don't know how to use a debugger this is a good opportunity to learn how to use it to run your program one line at a time, monitor all variables and their values as they change, and analyse your program's logical execution flow. Knowing how to use a debugger is a required skill for every C++ developer, no exceptions. With your debugger's help you should be able to quickly find all bugs in this and all future programs you write, without having to ask anyone for help. – Sam Varshavchik Feb 16 '20 at 15:41
  • [What is a debugger and how can it help me diagnose problems?](https://stackoverflow.com/q/25385173/5910058) – Jesper Juhl Feb 16 '20 at 15:51
  • When asking a question, posting a [mcve] rather than just code snippets is a really good idea. – Jesper Juhl Feb 16 '20 at 15:53

0 Answers0