[Learn about machine learning from the Keras] — 18.Compute_loss and metrics of the model

As can be seen from the previous section, when the model is compiled, a loss function can be specified, or a customized loss function can be customized. Once the loss function is determined, subsequent model training can be calculated using the content of this loss function. The operation of calculating the loss function is the compute_loss function located in the model; and the original built-in content is to use the LossesContainer mentioned in the previous chapter to execute the call function.

The compute_loss function can be customized, or you can add some tracking attributes to help observe the effect of model training.

Let’s look at the example provided by keras:

from tensorflow.keras.models import Sequential
from tensorflow.keras import layers
from tensorflow.keras.models import Model
import tensorflow as tf

class MyModel(Model):
    def __init__(self, *args, **kwargs):
        super(MyModel, self).__init__(*args, **kwargs)
        self.loss_tracker = tf.keras.metrics.Mean(name='loss')
        
    def compute_loss(self, x, y, y_pred, sample_weight):
        loss = tf.reduce_mean(tf.math.squared_difference(y_pred, y))
        loss += tf.add_n(self.losses)
        self.loss_tracker.update_state(loss)
        return loss
        
    def reset_metrics(self):
        self.loss_tracker.reset_states()
        
    @property
    def metrics(self):
        return [self.loss_tracker]
        
tensors = tf.random.uniform((10, 10)), tf.random.uniform((10,))
dataset = tf.data.Dataset.from_tensor_slices(tensors).repeat().batch(1)
inputs = tf.keras.layers.Input(shape=(10,), name='my_input')
outputs = tf.keras.layers.Dense(10)(inputs)
model = MyModel(inputs, outputs)
model.add_loss(tf.reduce_sum(outputs))
optimizer = tf.keras.optimizers.SGD()
model.compile(optimizer, loss='mse', steps_per_execution=10)
model.fit(dataset, epochs=2, steps_per_epoch=10)
print('My custom loss: ', model.loss_tracker.result().numpy())

When the self-made model is initialized, a new loss_tracker attribute object is added. This object is the entity of keras.metrics.base_metric.Mean.
Inherited keras.metrics.base_metric.Reduce, keras.metrics.base_metric.Metric.

In the model implementation compute_loss, it calculates its own loss value. After passing the result into the update_state function of the model.loss_tracker object, it will find keras.metrics.base_metric.Reduce.update_state to store the result in loss_tracker.total.

When the model is being fit, the model’s reset_metrics will be executed in each epoch loop. Here, the Metric object used by this model, that is, the model.loss_tracker object, will execute its own reset_states function for this Metric. When model.train_function is actually executed, after the predicted value of each layer is calculated, model.compute_loss will be used to calculate the loss function calculation result. After the custom calculation is completed, the update_state function of the model.loss_tracker object is used. Save the results.

Through this example, we can know that the model’s compute_loss will be related to metrics. In fact, when actually training the model, compute_loss will obtain the set metrics data set, and iteratively execute update_state one by one based on each Metric object.

The above is an example provided by keras, and its operation is recorded here through observation.