[Learn about machine learning from the Keras] — 11. Custom Model

From the previous article on Model, we can know that Model inherits base_layer, which means that Model can also be used as a Layer and can execute build and call. Now let’s try to customize the Model class so that the model can be declared diversified and flexibly.

Model classes that can be inherited include keras.engine.training.Model, keras.engine.sequential.Sequential, keras.engine.functional, keras.engine.base_layer, etc. No matter which class, constructing the model is the most important and requires the most implementation. The main tasks are the addition, deletion and modification of layers, model construction, model training, model calculation framework, etc. You can refer to the article on the operation of model execution fit. Basically, almost all of these classes inherit keras.engine.base_layer, so it is not an exaggeration to say that the model can also be a type of Layer. The main difference between model classes and layers is that the model is the behavior of the dominant management layer like a Container, while the layer is the role that performs detailed work.

Model construction and training usually go through the call function of the model class and layer, so here it is mostly necessary to iterate through the entire model to call and concatenate one by one, and the keras.engine.training.Model.step_function function of the model entity will be executed. . So the important parts that custom models can implement can be the Call and step_function functions.

Let’s look at the model classes again:

First example, custom model class:

InputsShape = keras.Input(shape=(None,784))

class MyModel(keras.Model):
    def __init__(self,inputs):
        super().__init__()
        dense1_Output = layers.Dense(64, activation="sigmoid")(inputs)
        dense2_Output = layers.Dense(10, activation="softmax")(dense1_Output)
        self.SubModel = keras.Model(inputs, dense2_Output)
        
    def call(self, inputs):
        return self.SubModel(inputs)
        
model = MyModel(inputs = InputsShape)

model.compile(optimizer="rmsprop",
                loss="sparse_categorical_crossentropy",
                metrics=["accuracy"])
                
model.fit(train_images, train_labels)

1.create MyModel instance (model = MyModel(inputs = InputsShape) )

Create a custom model class. Because it inherits keras.engine.training.Model, the initial construction will still use its own
MyModel.init, the following program is executed:

self.SubModel = Model(inputs, dense2_Output)

A new keras.engine.training.Model class is delegated and assigned to self.SubModel, so this part will continue to run as shown above.
Go to the orange area and generate a description of the Model entity as in the Model vs Sequence article.

2. execute training function by fit (model.fit(train_images, train_labels))

When executing training, the most important execution part is Train_Step, which will be executed from MyModel.call and then iterated to each layer layer.call. It will also check whether each layer has been built to generate settings and initial Weight. Since the init declaration delegates a new keras.engine.training.Model, the layer calls are used in series, which means that the build action has been executed (refer to the initial execution of Layer settings in the Model vs Sequence article), so Model.call continues The orange part executes the delegated keras.engine.training.Model.call(inputs). The process in the orange part has been built, so it will go to keras.engine.functional.Functional._run_internal_graph to iteratively execute the Layer.call function of each layer one by one. Finally, the result tensor is passed out, and actions such as calculating loss and updating weight update_state are continued.

The second example is to treat the Model as a Layer and implement its Call function to return the Layer entity, or use another class that can be initialized and implement __Call__ to return the Layer entity.

Treat the Model as a Layer program:

import tensorflow as tf
from tensorflow.keras import layers
from tensorflow.keras.models import Model

class MyLayer(Model):
    def __init__(self):
        super().__init__()
        self.dense = layers.Dense(10, activation="softmax")
    def call(self, inputs):
        return self.dense(inputs)
        
inputs = layers.Input(shape=(None, 784))
features = layers.Dense(64, activation="relu")(inputs)
outputs = MyLayer()(features)

model = tf.keras.Model(inputs=inputs, outputs=outputs)

model.compile(optimizer="rmsprop",
                loss="sparse_categorical_crossentropy",
                metrics=["accuracy"])
                
model.fit(train_images, train_labels)

illustrate:

Here, a sub-class is customized as MyModel and inherits keras.engine.training.Model, which mainly implements the call function, while keras.engine.training.Model inherits keras.engine.base_layer. When the model is initialized, it will receive the parameters of the inputs layer and outputs layer:

model = keras.Model(inputs=inputs, outputs=outputs)

The incoming parameters, especially outputs, are returned using a custom class; this custom class MyModel first creates a Dense layer when it is initialized, and the MyModel.call function uses the call function of this Dense layer to receive additional concatenated data. Layer, use Dense.call to connect all the layers (refer to the article Model vs Sequence for another way to set the layer), and return it to keras.Model for use.

Therefore, before the final return from init to call of MyModel, you can perform some layer settings and input data processing preprocess actions.

The above is an example of a custom module. Through operation, you can understand which actions can be expanded or additionally performed in the future, and the timing of execution.