You have previously trained a 2-layer Neural Network (with a single hidden layer). This week, you will build a deep neural network, with as many layers as you want!

- In this notebook, you will implement all the functions required to build a deep neural network.
- In the next assignment, you will use these functions to build a deep neural network for image classification.

**After this assignment you will be able to:**

- Use non-linear units like ReLU to improve your model
- Build a deeper neural network (with more than 1 hidden layer)
- Implement an easy-to-use neural network class

# Packages

Let’s first import all the packages that you will need during this assignment.

**numpy**is the main package for scientific computing with Python.**matplotlib**is a library to plot graphs in Python.- dnn_utils provides some necessary functions for this notebook.
- testCases provides some test cases to assess the correctness of your functions.
- np.random.seed(1) is used to keep all the random function calls consistent. It will help us grade your work. Please don’t change the seed.

1 | import numpy as np |

# Outline od the Assignment

Note that for every forward function, there is a corresponding backward function. That is why at every step of your forward module you will be storing some values in a cache. The cached values are useful for computing gradients. In the backpropagation module you will then use the cache to calculate the gradients. This assignment will show you exactly how to carry out each of these steps.

# Initialization

You will write two helper functions that will initialize the parameters for your model. The first function will be used to initialize parameters for a two layer model. The second one will generalize this initialization process to L layers.

## 2-layer Neural Network

Create and initialize the parameters of the 2-layer neural network.

Instructions:

- The model’s structure is: LINEAR -> RELU -> LINEAR -> SIGMOID.
- Use random initialization for the weight matrices. Use np.random.randn(shape)*0.01 with the correct shape.
- Use zero initialization for the biases. Use np.zeros(shape).

1 | # GRADED FUNCTION: initialize_parameters |

## L-layer Neural Network

Implement initialization for an L-layer Neural Network.

1 | # GRADED FUNCTION: initialize_parameters_deep |

# Forward propagation module

## Linear Forward

Now that you have initialized your parameters, you will do the forward propagation module. You will start by implementing some basic functions that you will use later when implementing the model. You will complete three functions in this order:

1 | # GRADED FUNCTION: linear_forward |

## Linear-Activation Forward

1 | # GRADED FUNCTION: linear_activation_forward |

**Note**: In deep learning, the “[LINEAR->ACTIVATION]” computation is counted as a single layer in the neural network, not two layers.

## L-Layer Model

For even more convenience when implementing the L-layer Neural Net, you will need a function that replicates the previous one (linear_activation_forward with RELU) L−1 times, then follows that with one linear_activation_forward with SIGMOID.

1 | # GRADED FUNCTION: L_model_forward |

Great! Now you have a full forward propagation that takes the input X and outputs a row vector $A^{[L]}$ containing your predictions. It also records all intermediate values in “caches”. Using $A^{[L]}$, you can compute the cost of your predictions.

# Cost function

Now you will implement forward and backward propagation. You need to compute the cost, because you want to check if your model is actually learning.

Compute the cross-entropy cost J, using the following formula:

1 | # GRADED FUNCTION: compute_cost |

# Backward propagation module

Just like with forward propagation, you will implement helper functions for backpropagation. Remember that back propagation is used to calculate the gradient of the loss function with respect to the parameters.

## Linear backward

Use the 3 formulas above to implement linear_backward().

1 | # GRADED FUNCTION: linear_backward |

## Linear-Activation backward

Implement the backpropagation for the LINEAR->ACTIVATION layer.

1 | # GRADED FUNCTION: linear_activation_backward |

## L-Model Backward

Now you will implement the backward function for the whole network. Recall that when you implemented the L_model_forward function, at each iteration, you stored a cache which contains (X,W,b, and z). In the back propagation module, you will use those variables to compute the gradients. Therefore, in the L_model_backward function, you will iterate through all the hidden layers backward, starting from layer L. On each step, you will use the cached values for layer l to backpropagate through layer l.

Implement backpropagation for the [LINEAR->RELU] × (L-1) -> LINEAR -> SIGMOID model.

1 | # GRADED FUNCTION: L_model_backward |

## Update Parameters

1 | # GRADED FUNCTION: update_parameters |

# Conclusion

Congrats on implementing all the functions required for building a deep neural network!

We know it was a long assignment but going forward it will only get better. The next part of the assignment is easier.

In the next assignment you will put all these together to build two models:

- A two-layer neural network
- An L-layer neural network

You will in fact use these models to classify cat vs non-cat images!