Share on facebook
Share on twitter
Share on linkedin
Share on pinterest

KERAS Callbacks

Keras comes with a long list of predefined callbacks that are ready to use. Keras callbacks are functions that are executed during the training process.

According to Keras Documentation, A callback is a set of functions to be applied at given stages of the training procedure. You can use callbacks to get a view on internal states and statistics of the model during training. You can pass a list of callbacks (as the keyword argument callbacks) to the .fit() method of the Sequential or Model classes. The appropriate methods of the callbacks will then be called at each stage of the training. 

In this article, we’ll discuss some of the commonly used callbacks in Keras.

MODEL CHECKPOINT:

The first callback we are going to discuss is the model checkpoint.

Sometimes, training a deep neural network might take days. We don’t want to lose all our progress if there’s a power outage.

By using model checkpoint callback, we can save our model at regular intervals.

The model checkpoint callback saves the weights of the model along with the structure once we made progress, like hitting a new validation accuracy or loss. We could also save the model for every epoch.

The signature of the callback is as follows

  • monitor: quantity to be monitored
  • save_best_only: if TRUE saves only the best model according to the quantity monitored
  • Save_weights_only: if TRUE only the weights of the model are saved
  • mode:  one of {auto, min, max}. In min mode, training will stop when the quantity monitored has stopped decreasing; in max mode, it will stop when the quantity monitored has stopped increasing;  auto mode, the direction is automatically inferred from the name of the monitored quantity.
  • period: after how many epochs you need to checkpoint the model

EARLY STOPPING:

By stopping the training of our model early, we can prevent our model from overfitting.

For instance, our model might keep reducing its loss in the training data and keep increasing its loss in the validation data. This is a sign of overfitting.

By using the early stopping callback, we can monitor specific metrics like validation loss or accuracy. As soon as the chosen metric stops improving for a fixed number of epochs, we are going to stop the training.

  • min_delta: minimum change in the monitored quantity to qualify as an improvement
  • patience: number of epochs that produced the monitored quantity with no improvement after which training will be stopped

REDUCE LR ON PLATEAU:

This callback can be used to reduce the learning rate of our model in case the loss is plateaued.

  • factor: a value of 0.1 divides the learning rate by 10

CSV LOGGER:

This callback logs the output of the model to a CSV file.

  • filename: name of the CSV file

TENSORBOARD:

Tensorboard is a framework that can be used to troubleshoot or monitor our machine learning algorithms.

It offers a suite of visualization tools for visualizing the network’s computational graph as well as the model’s training progress.

Sometimes it might be helpful to keep track of network parameters while training a neural network. With the help of TensorBoard, we can visualize metrics like loss, accuracy, and batch training time over iterations.

In Keras, we can use the Tensorboard callback to save the network parameters and analyze them.

  • log_dir: path to save the log files
  • write_graph: if TRUE, visualize graph in tensorboard
  • update_freq: ‘batch’ or ‘epoch’ or integer. When using ‘batch’, writes the losses and metrics to TensorBoard after each batch. The same applies to ‘epoch’. If using an integer, let’s say 10000, the callback will write the metrics and losses to TensorBoard every 10000 samples.

Tensorboard can be invoked from the command line as follows

CUSTOM CALLBACK:

Along with the list of predefined callbacks that are ready to use, we can also create custom callbacks in Keras.

We can create a simple custom callback using the LambdaCallback.

The signature of the lambda callback is as follows.

  • on_epoch_begin: called at the beginning of every epoch.
  • on_epoch_end: called at the end of every epoch.
  • on_batch_begin: called at the beginning of every batch.
  • on_batch_end: called at the end of every batch.
  • on_train_begin: called at the beginning of model training.
  • on_train_end: called at the end of model training.

Now let’s do some coding

First, we’ll import all the necessary packages and the dataset. We’ll use the fashion MNIST dataset.

Next, let’s build our CNN model.

Once we defined our model, we can now create the callbacks

We’ll create a custom callback which stops the training of the model once the validation accuracy reaches 90%

Finally, we can pass the callbacks object into the model fit method as a list.

Below is the complete code

CONCLUSION:

In this article, we discussed some of the most commonly used callbacks in Keras.

This is not the complete list of callbacks that Keras offers. I’d encourage you to visit Keras Documentation to take a look at available callbacks in Keras.

Reference:

https://keras.io/callbacks/

Love What you Read. Subscribe to our Newsletter.

Stay up to date! We’ll send the content straight to your inbox, once a week. We promise not to spam you.

Leave a Reply

Your email address will not be published. Required fields are marked *

Subscribe Now! We'll keep you updated.