0

I implemented ANN and my dataset have the first 100 data from class 1, and next 100 from class 2,..., and last 100 from class 10 (so I have 10 binary output units). I do back_propagation on my data to learn weights, but when I feed the new weights with training data again to the ANN, it predicts everything from class 10 which is the last 100 data's class. I am new to ANN, what am I missing?

user5808583
  • 89
  • If you're only doing one training pass and the inputs aren't randomized it's possible it is forgetting the older classes and relearning to only output 10 since it is the last portion of the training set. – Evan Weissburg Oct 22 '17 at 04:25
  • Thanks, I am trying to mix the data now. Does NN learn only one set of weights for network even when we have 10 units in the output layer? – user5808583 Oct 22 '17 at 04:41
  • I need more information about your network architecture. How many layers are there? What activation function? What loss function? Are you using softmax on the output layer? – Evan Weissburg Oct 22 '17 at 04:44
  • 1
    I mixed the data, and it still doesn't work! I am probably having a mistake in the design, I have 3 layers, inputs and then 3 units in hidden layer, and 10 units in the output layer. I have used sigmoid function for units, I think softmax would solve the problem as it converts the 10 output units to only 1? – user5808583 Oct 22 '17 at 04:51
  • I can get back to this later but I recommend both this Google doc reference (skip around as needed) and this GitHub to check your model: https://docs.google.com/document/d/1mpZ00nksriNA4KtT1Tu78mLOpAchOJXBxYYEBFrzIXQ/edit?usp=drivesdk. https://github.com/nave01314/NNClassifier – Evan Weissburg Oct 22 '17 at 04:55
  • @user5808583 I'd start with ReLU on the hidden layers and softmax+logloss (often fused as softmax-cross-entropy) for the final one. Also a bottleneck of 3 units in the hidden layer is very narrow. I'd experiment with choices between 10 to 100 (on the higher end you probably need some form of regularization). – CodesInChaos Oct 23 '17 at 11:22
  • I am pretty new to NN, but I could implement it using sigmoid function in the output layer for now. I take a maximum from 10 outputs... – user5808583 Oct 26 '17 at 07:53

0 Answers0