The Machine Learning Blog

Artificial Intelligence and Mathematics. Part 1: Anatomy of a Neural Network

"By verbally stating that Mathematics are difficult, we are saying it with a language whose syntax is as complex, no more, no less, than the one that governs mathematics"

Neural networks (DNN) are one of the most used machine learning models nowadays. DNN can be visualized as a set of nodes and lines forming a network, see figure 1. Its similarity with the organization of neurons in the brain is evident, but this topic will be discussed later.

Figure 1. Simplified diagram of Deep Neural Network (DNN)

In a neural network, two nodes and one line correspond to a basic unit of the model and are a graphic representation of a mathematical function of type:

f (x) = kx

For Figure 2, the input variable(x) corresponds to the green node, the output variable f(x) to the blue node and the line that joins them corresponds to the constant k:

Figure 2. Fundamental unit of a DNN

Every mathematical function can be visualized as a box that receives one or more input elements and delivers a single output element:

Figure 3. Graphical representation of a mathematical function

The function "f "is a mathematical relationship between one or more input variables and an output variable.

Thus, the function:

f (x) = 3.x

take input values x, multiplies them by 3 and delivers output values f(x):

Figure 4. Input elements (x) and output f(x) defined by the function f

The value "3" on each arrow corresponds to the value of the constant (parameter of the function) that characterizes the same in each case.

In the neural network of Figure 1, each output element corresponds to the sum of several functions, each with its own parameter that defines it:

Figure 5. Function with three input variables and one output variable

To illustrate, let's work on some nodes and lines of the Neural Network mentioned above, see Figure 6.

Figure 6. Partial detail of a Neural network

For these elements of the network, we can define a function:

f (Xi) = Sum (Ki.Xi)

This is, the output f(Xi) (orange node) corresponds to the sum of the input variables multiplied each by its corresponding constant (function parameter).

What the previous example shows us is that if we pre-define values ​​for the 32 parameters (values ​​of the total of link lines of the Neural Network) and we have the values ​​of the three input variables (Input Layer), we can chain from left to right the results of the functions, converting the output of one function to the input of the next, until obtaining the values ​​of the last output (Output Layer).

This operation is known as "Forward Propagation", that for trained models (parameters with assigned values), it allows to evaluate or classify input entities, this evaluation being the calculated output of the model.

The following article will be devoted to an illustrative example of the theory presented here.