[Learn about machine learning from the Keras] — 17.How to use and customize loss function

When the model is doing Compiler, specify the loss function as follows:

model.compile(optimizer="rmsprop",
loss="sparse_categorical_crossentropy",
metrics=["accuracy"])

We can specify the string “sparse_categorical_crossentropy” to let the framework find the corresponding module to use. Is there any other way to specify the loss function, or is there an opportunity to define another algorithm for calculation?

Through observation, if the above method is used, the LossContainer object will be created and initialized when the model is compiled. This object will be assigned to the compiled_loss attribute of the model, and the specified loss parameter will also be included in this object.

When the LossesContainer is created, the build action is not executed, and the loss included in it is not completely established (if the specified loss function name is passed).

Until the model training process (refer to the model fit article), after the calculation of each layer of a certain round of training is completed, the loss function will be needed. At this time, you will go to keras.engine.training.compute_loss to find the loss function because the LossContainer is detected. The build action has not been executed, so the build of LossesContainer will be done first at this time.
The following process will be carried out along the way:
keras.engine.compile_utils._get_loss_object
-> keras.losses.get
-> keras.losses.deserialize
-> keras.saving.legacy.serialization.deserialize_keras_object

The corresponding loss function object will be searched for in the module object in the frame.
The loss function object in the framework also finds the corresponding loss object according to the passed string identifier. The mapping content is as follows:

function :
‘squared_hinge’ :keras.losses.squared_hinge
‘hinge’ :keras.losses.hinge
‘categorical_hinge’ :keras.losses.categorical_hinge
‘huber’ :keras.losses.huber
‘log_cosh’ :keras.losses.log_cosh
‘categorical_crossentropy’ :keras.losses.categorical_crossentropy
‘categorical_focal_crossentropy’ :keras.losses.categorical_focal_crossentropy
‘sparse_categorical_crossentropy’ :keras.losses.sparse_categorical_crossentropy
‘binary_crossentropy’ :keras.losses.binary_crossentropy
‘binary_focal_crossentropy’ :keras.losses.binary_focal_crossentropy
‘kl_divergence’ :keras.losses.kl_divergence
‘poisson’ :keras.losses.poisson
‘cosine_similarity’ :keras.losses.cosine_similarity
‘bce’ :keras.losses.binary_crossentropy
‘BCE’ :keras.losses.binary_crossentropy
‘mse’ :keras.losses.mean_squared_error
‘MSE’ :keras.losses.mean_squared_error
‘mae’ :keras.losses.mean_absolute_error
‘MAE’ :keras.losses.mean_absolute_error
‘mape’ :keras.losses.mean_absolute_percentage_error
‘MAPE’ :keras.losses.mean_absolute_percentage_error
‘msle’ :keras.losses.mean_squared_logarithmic_error
‘MSLE’ :keras.losses.mean_squared_logarithmic_error
‘kld’ :keras.losses.kullback_leibler_divergence
‘KLD’ :keras.losses.kullback_leibler_divergence
‘kullback_leibler_divergence’ :keras.losses.kullback_leibler_divergence
‘logcosh’ :keras.losses.logcosh
‘huber_loss’ :keras.losses.huber_loss

class:
‘Loss’ :’keras.losses.Loss’
‘LossFunctionWrapper’ :’keras.losses.LossFunctionWrapper’
‘MeanSquaredError’ :’keras.losses.MeanSquaredError’
‘MeanAbsoluteError’ :’keras.losses.MeanAbsoluteError’
‘MeanAbsolutePercentageError’ :’keras.losses.MeanAbsolutePercentageError’
‘MeanSquaredLogarithmicError’ :’keras.losses.MeanSquaredLogarithmicError’
‘BinaryCrossentropy’ :’keras.losses.BinaryCrossentropy’
‘BinaryFocalCrossentropy’ :’keras.losses.BinaryFocalCrossentropy’
‘CategoricalCrossentropy’ :’keras.losses.CategoricalCrossentropy’
‘CategoricalFocalCrossentropy’ :’keras.losses.CategoricalFocalCrossentropy’
‘SparseCategoricalCrossentropy’ :’keras.losses.SparseCategoricalCrossentropy’
‘CosineSimilarity’ :’keras.losses.CosineSimilarity’
‘Hinge’ :’keras.losses.Hinge’
‘SquaredHinge’ :’keras.losses.SquaredHinge’
‘CategoricalHinge’ :’keras.losses.CategoricalHinge’
‘Poisson’ :’keras.losses.Poisson’
‘LogCosh’ :’keras.losses.LogCosh’
‘KLDivergence’ :’keras.losses.KLDivergence’
‘Huber’ :’keras.losses.Huber’

If the function object is retrieved, the loss function object will be integrated through the keras.losses.LossFunctionWrapper class, so that subsequent calculations can be called through keras.losses.LossFunctionWrapper.call.

Custom loss function class
So how do you implement a class if you customize the loss function? Here is a simple example to implement:

(1) Inherit keras.losses.LossFunctionWrapper and pass in the loss function object (or customize it yourself):

class MySparseCategoricalCrossentropy(LossFunctionWrapper):
    def __init__(
     self, fn , from_logits=False, ignore_class=None, reduction=losses_utils.ReductionV2.AUTO,  
     name="sparse_categorical_crossentropy",
     ):
     super().__init__(
     fn,
     name=name,
     reduction=reduction,
     from_logits=from_logits,
     ignore_class=ignore_class,
     )

(2) Inherit keras.losses.Loss and implement the call function:

class MySparseCategoricalCrossentropy(keras.losses.Loss):
    def call(self, y_true, y_pred):
     return keras.losses.sparse_categorical_crossentropy(y_true, y_pred, from_logits=False,
     axis=-1, ignore_class=None)

Therefore, these customized methods can also add customized processing to perform some additional calculations, more efficient formula operations, etc.

The above are observations and recorded here.