1

I'm building an LSTM sequential Binary Classification Model, the data is highly imbalanced like say Fraud detection case.

After building an LSTM model on Sequential Vectorised data, I'm getting a very low recall of 0.005.

# build LSTM layers
model = Sequential()
model.add(LSTM (100, dropout=0.2, input_shape=(time_steps, features)))
model.add(Dense(50, activation='relu'))
model.add(Dense(25, activation='relu'))
model.add(Dense(10, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=[Recall()])
print (model.summary())
history=model.fit(train_X, train_Y, validation_data=(test_X, test_Y), 
epochs=10,batch_size=64)

Please help me, with how to optimize the recall for this model.

Thank you

Jagrut Panchal
  • 13
  • Optimizing recall is easy: every time, predict the class you want to recall. If you do this, you will never miss a case and will have perfect recall. If this approach is unacceptable (which I suspect it is), perhaps you can explain why. // You might be interested in the links I posted in response to the answer be Baradrist. It turns out that metrics like acccuracy, precision, recall, sensitivity (same as recall), and specificity are surprisingly problematic. – Dave Jul 19 '22 at 10:16

1 Answers1

0

The problem is that in your training data, the "positive" class (since recall is the problem, I guess "fraud" is the "positive") is underrepresented and the algorithm has not enough data to pick up on this signal. There is even a valid easy strategy for it: It can get very high accuracy by only predicting the abundant class per default, since it will be wrong only very seldom.

What you can do now is to over- or undersample your data to make the classes artificially more equal. There are numerous techniques for that (see here), plus you can also augment the positive samples that you use. There is also the option to weight your samples, meaning that positive samples get a very stronger influence on the error metric (scaled up relative to its reciprocal proportion in the data) assigned for backpropagation compared to the abundant class. For Keras this looks loke this. The alternative is to use another model (for outlier detection, sometimes autoencoders are employed (here and here)).

Baradrist
  • 1
  • 1
    Welcome to across Validated! Statisticians do not see class imbalance as such a problem and don’t particularly like accuracy as a metric. There is no need to use undersampling, oversampling, or artificial balancing to solve a non-problem. https://stats.stackexchange.com/questions/357466 https://www.fharrell.com/post/class-damage/ https://www.fharrell.com/post/classification/ https://stats.stackexchange.com/a/359936/247274 https://stats.stackexchange.com/questions/464636/ https://stats.stackexchange.com/questions/558942/ https://twitter.com/f2harrell/status/1062424969366462473?lang=en – Dave Jul 19 '22 at 09:46