5

I am currently working on replicating financial deep learning papers to see if they actually would work in real markets.

I have this question that bothers me. Is it okay to use regular loss functions such as MSE, RMSE, MAE, MAPE,..., or other absolute difference loss functions in Finance? It seems odd.

For example, let's compare two scenarios.

A: a financial deep learning model that only predicts long position predicted +5% and the market actually moved up +10%

B: the same model predicted +5%, but this time the market moved +4.5% resulting in a loss.

However the model would learn and would give more scores to scenario B since the absolute difference is less.

So I want to customize loss function or explore the properties of different loss functions.

I was thinking something like...but not exactly like below since it would still prefer 6%(actual) - 5%(predicted) over 10%(actual) - 5%(predicted) Custom Function in TensorFlow

class Error(Loss):

def call(self, y_true, y_pred):
    y_pred = tf.convert_to_tensor_v2(y_pred)
    y_true = tf.cast(y_true, y_pred.dtype)
    return tf.reduce_mean(y_true - y_pred), axis=-1)
hyukkyulee
  • 171
  • 2
    Your question sounds like you are "only" asking about how to implement a custom loss function in TensorFlow etc. That is programming, and off-topic here. But your self-answer indicates that you are also interested in the properties of different loss functions, and that is on-topic. Perhaps you would like to edit your question to clarify what you are looking for. – Stephan Kolassa Jan 04 '22 at 09:08
  • @Stephan Kolassa You saw through my mind. I have edited the question so it clarifies that I want to explore the properties of different loss functions. Thank you! – hyukkyulee Jan 04 '22 at 09:45
  • the same model predicted +5%, but this time the market moved +4.5% resulting in a loss A financial loss? As far as I can tell, this is a 4.5% gain, which might be lower than predicted but is still a gain. – Dave Jan 17 '24 at 16:26

2 Answers2

3

In my opinion, it makes the most sense to first consider which functional of the (unknown, and typically only implicitly considered) future distribution we want to elicit: the mean, the median, a particular quantile, or some more exotic functional. Only once we know that can we decide on a loss function, and we should choose a loss function that is optimized in expectation by the sought-after functional. I have laid out my arguments in Kolassa (2020, IJF), relying mainly on previous work of Tilmann Gneiting. This is also the approach I follow in my answer to What are the shortcomings of the Mean Absolute Percentage Error (MAPE)?.

Stephan Kolassa
  • 123,354
2

What I think is a good solution so far, Improving the Prediction of Asset Returns With Machine Learning by Using a Custom Loss Function

  • Dr. Dessain had the same question and answered in his paper. "Dessain (2021) offers arguably the most comprehensive overview to date, with 190 articles reviewed over the period 2010 – June 2021, but with a narrow focus on the sole performance metrics used to compare algorithms predicting asset returns. He demonstrates that the mean squared error (MSE), root mean squared error (RMSE), mean absolute error (MAE) or similar error-based performance metrics are not appropriate for assessing the results of algorithms predicting asset returns. The underlying reasoning is that error-based metrics treat all errors equally and do not differentiate an error that triggers a bad investment decision (an investment resulting in a negative return or a missed opportunity with no investment when the asset has led to a positive return) from a prediction error which does not have any adverse consequence, leading to a positive return or to a non-investment that avoided a negative return. The deductive reasoning is confirmed with an extensive empirical analysis."
  • his custom loss functions aim to give more penalty on loss than profit. As you have expected, asymmetric design of the loss function is the key. Here is the code for custom loss functions
    • please note that asymmetric designs has inherit limits. Models will learns to be 'safe.' In other words, yhat will be lower. This could make taking positions almost impossible since the real price(y) will be always higher.
    • please note that in order to use AdjMSELoss2, you must adjust 10000 to be higher or lower. Most of the time it should be lowered since if (10000 * outputs * labels) is too big, mean(loss) becomes too low or nan. Then, model cannot be learned. 
      def __init__(self):
    super(AdjMSELoss2, self).__init__()

      def forward(self, outputs, labels):
    outputs = torch.squeeze(outputs)
    beta = 2.5
    loss = (outputs - labels)*2
    adj_loss = beta - (beta - 0.5) / (1 + torch.exp(10000  torch.mul(outputs, labels)))
    loss = beta * loss /(1+adj_loss) 
    return torch.mean(loss)```
  • He also posted a performance metric, which has many advantages over other ratios like sharpe ratio. D-ratio "solely focuses on the added value of the algorithm." If you want to know more about D-ratio, please go check out his another paper, Machine learning models predicting returns: why most popular performance metrics are misleading and proposal for an efficient metric. I highly recommend it. He also posted code for D-ratio on his github

What I think is the best solution in reality.

  • only use deep learning to represent the environment
  • use reinforcement learning to help a human trader. In other words, Machine Learning should not be used in actual trading, but only to help traders. Good use of machine learning could be calculating hedging, option price, and order executions.

Any other paper or solution to the loss function? Please leave in the comments or answers.

hyukkyulee
  • 171