How to build artificial neural networks with Keras and TensorFlow

What is deep learning?

What is an activation function?

Sigmoid function

Softmax activation function

Rectified linear unit (ReLU)

How does a neural network learn?

Gradient descent

How backpropagation works

What is TensorFlow?

Why TensorFlow?

TensorFlow vs. Keras

TensorFlow basics

Tensors

Variables

Automatic differentiation

How TensorFlow works

How TensorFlow models are defined

How to train artificial neural networks with Keras

Data pre-processing

Data transformation

How to build the artificial neural network

How to visualize model performance

Add dropout regularization to fight overfitting

How to accelerate network training with batch normalization

How to stop model training at the right time with early stopping

How to save the best model with checkpoints

Make predictions on the test set

Check the confusion matrix

Make a single prediction

How to save and load Keras models

How to evaluate the Keras model with cross-validation

How to tune model hyperparameters in Keras

How to tune the network parameters

How to build CNN in TensorFlow

What is CNN?

How do CNNs work?

Convolution

Padding

Apply ReLU

Pooling

Deep learning with TensorFlow and Keras

Dropout regularization

Flattening

Full connection

Activation function

Convolutional Neural Networks (CNN) in TensorFlow

How to install TensorFlow

How to confirm TensorFlow is installed

What are Keras and tf.keras?

Develop multilayer CNN models

Data preprocessing

Model definition

Compiling the model

Train the model

How to plot model learning curves

Model evaluation

How to halt training at the right time with Early Stopping

How to accelerate training with batch normalization

How to create custom callbacks for TensorFlow CNN

How to visualize a deep learning model

How to save and load your model

Running CNNs with TensorFlow in the real world

Loading the images

Generate a tf.data.Dataset

Buffered dataset prefetching

Image augmentation

Model definition

Compiling the model

Training the model

Model evaluation

Monitoring the model’s performance

Visualize CNN graph with TensorBoard

How to profile with TensorBoard

Making predictions

CNN architectures

Model without weights

Model with weights

TensorFlow Recurrent Neural Networks

What is a Recurrent Neural Network?

Backpropagation through time

Types of Recurrent Neural Networks

Weaknesses of RNNs

1. Vanishing gradient problem

2. Exploding gradient problem

Long-Short Term Memory (LSTM)

Applications of LSTM

Bidirectional LSTM

Time series analysis with LSTM in TensorFlow

Create LSTM network in Keras

Compile the LSTM model

LSTM model evaluation

Intent classification with LSTM

Text vectorization

Create LSTM network

LSTM model evaluation

Transfer learning guide

What is transfer learning?

Advantages of using pre-trained models

Types of transfer learning

Inductive transfer learning

Unsupervised transfer learning

Transductive transfer learning

Homogeneous transfer learning

Heterogeneous transfer learning

What is the difference between transfer learning and fine-tuning?

Why use transfer learning?

When do you use transfer learning?

When does transfer learning not work?

How to implement transfer learning?

Transfer learning in 6 steps

Step 1: Obtain the pre-trained model

Step 2: Create a base model

Step 3: Freeze layers so they don’t change during training

Step 4: Add new trainable layers

Step 5: Train the new layers on the dataset

Step 6: Improve the model via fine-tuning

Where to find pre-trained models?

Keras pre-trained models

Transfer learning using TensorFlow Hub

Pretrained word embeddings

Stanford’s GloVe pre-trained word embeddings

Below is an example of an implementation for the GloVe pre-trained word embeddings.

Google’s Word2vec

Fasttext

Hugging Face

Transfer learning with PyTorch

How can you use pre-trained models?

Prediction

Feature extraction

Fine-tuning

Example of transfer learning for images with Keras

Transfer learning with image data

Getting the dataset

Loading the dataset from a directory

Data pre-processing

Create a base model from the pre-trained Inception model

Create the final dense layer

Train the model

Fine-tuning the model

Example of transfer learning with natural language processing

Pretrained word embeddings

Loading the dataset

Data pre-processing

Vectorizing the words

Using GloVe Embeddings

Create the embedding layer

Create the model

Training the model

TensorBoard

Advantages of using Tensorboard

How to use TensorBoard

How to install TensorBoard

PIP installation

Conda installation

Docker installation

Using TensorBoard with Jupyter notebooks and Google Colab

How to run TensorBoard

How to use TensorBoard callback

How to launch TensorBoard

Running TensorBoard remotely

TensorBoard dashboards

TensorBoard scalars

TensorBoard images

TensorBoard graphs

TensorBoard distributions

TensorBoard histograms

Fairness indicators

What-If Tool (WIT)

Displaying data in TensorBoard

Using the TensorBoard embedding projector

Plot training examples with TensorBoard

Visualize images in TensorBoard

Displaying text data in TensorBoard

Log confusion matrix to TensorBoard

Hyperparameter tuning with TensorBoard

TensorFlow Profiler

Overview page

Trace viewer

Input pipeline analyzer

TensorFlow stats

GPU kernel stats

Memory profile page

How to enable debugging on TensorBoard

Using TensorBoard with deep learning frameworks

TensorBoard in PyTorch

TensorBoard in Keras

TensorBoard in XGBoost

TensorBoard in JAX and Flax

Download TensorBoard data as Pandas DataFrame

Tensorboard.dev

Limitations of using TensorBoard

How to build TensorFlow models with the Keras Functional API

Keras Sequential models

Keras Functional models

Defining input

Connecting layers

Functional API Python syntax

Creating the model

Training and evaluation of Functional API models

Save and serialize Functional API models

How to convert a Functional model to a Sequential API model

How to convert a Sequential model to a Functional API model

Standard network models

Multilayer perception

Convolutional Neural Network

Recurrent Neural Network

Shared layers model

Shared input layer

Shared feature extraction layer

Multiple input and output models

Multiple input model

Multiple output model

Use the same graph of layers to define multiple models

Keras Functional API end-to-end example

Data download

Data processing

Add image path column

Create face attributes columns

Label encoding

Generate tf.data dataset

Visualize the training data

Define Keras Functional network

Plot and inspect the Keras Functional model

Compile Keras Functional network

Training the Functional network

Evaluate TensorFlow Functional network

Make predictions with Keras Functional model

Keras Functional API strengths and weaknesses

Functional API best practices

How to create custom training loops in Keras

Batch the dataset

How to create model with custom layers in Keras

Define the loss function

Define the gradients function

Create an optimizer

Create custom training loop

Visualize the loss

Evaluate model on test dataset

Use the trained model to make predictions

How to train deep learning models on Apple Silicon GPU

Training deep learning models on Apple Silicon

TensorFlow

Install Tensorflow-metal PluggableDevice

Train TensorFlow model on Apple Silicon GPU

PyTorch

Object detection with TensorFlow 2 Object detection API

Object detection datasets

Preparing datasets for object detection

What is TensorFlow 2 Object Detection API?

Install TensorFlow 2 Object Detection API on Google Colab

Install TensorFlow 2 Object Detection API locally

Download object detection dataset

Download Mask R-CNN model

Edit the object detection pipeline config file

Convert the images to TFRecords

Train the model

Model evaluation and visualization

Download model from Google Colab

Object detection with Mask R-CNN

Load an image from file into a NumPy array

Visualize detections

Create model from the last checkpoint

Map labels for inference decoding

Run detector on test image

Image segmentation with Mask R-CNN

Set label map

Set test image paths

Create inference function

Perform segmentation and detection

Final thoughts