Deep learning

Computers can learn in a special way.

They use many layers to think. These layers work like a stack.

Each layer helps the computer understand more.

First, a layer might see simple lines. Next, a layer sees shapes like circles. Then, it might see a nose or eyes. Finally, the computer knows it sees a face.

This helps computers do hard tasks. They can listen to speech. They can also look at pictures. They can even play board games.

This way of learning is very smart. It helps computers act like experts. It is a very cool way to work!

Computers can learn in a very smart way. This is called deep learning.

It is a part of machine learning. Deep learning uses neural networks. These are sets of parts that act like tiny neurons.

These parts are stacked in many layers. The word "deep" tells us there are many layers. A network might have three layers or even thousands.

These layers work in a set of steps. Each layer makes the data more clear. Imagine a computer looking at a photo of a face. The first layer sees simple lines. The next layer sees shapes like circles. The third layer might see a nose and eyes. Finally, the last layer knows it is a face.

Deep learning helps computers do many jobs. They can recognize speech. They can also study the climate. They can even play board games. Sometimes, they work as well as human experts. This image shows how a computer can change the style of a picture.

Deep learning is a special way for computers to learn.

It is a part of machine learning. This method uses multilayered neural networks to do many tasks. These networks can help with things like classification or regression. They can also learn how to represent data. Deep learning is very useful in many different fields. It is used in computer vision and speech recognition. It even helps with drug design and climate science. Sometimes, these systems work as well as human experts.

This way of learning works through a hierarchy of layers. Each layer changes the data into something more complex. Imagine a computer looking at a photo of a face. The first layer might only see simple lines and circles. The second layer sees how those edges are arranged. The third layer might recognize a nose or eyes. Finally, the fourth layer knows the image is a face. The computer learns which features to put in each layer on its own. This is different from older ways where people had to design the features by hand.

Scientists have been working on these ideas for a long time. The term "deep learning" was used by Rina Dechter in 1986. Later, Igor Aizenberg and his colleagues used the term in 2000. Early ideas were inspired by how the human brain works. However, current networks are not meant to be perfect models of the brain. The word "deep" refers to the number of layers in the network. This is often called the credit assignment path, or CAP. A deep system has a CAP depth higher than two.

Many different types of networks exist today. Some are called convolutional neural networks, or CNNs. Others are called transformers or recurrent neural networks. In 1965, Alexey Ivakhnenko and Lapa published a working deep learning algorithm. It was called the Group method of data handling. In 1979, Kunihiko Fukushima introduced the Neocognitron. This was an early type of convolutional neural network. Another important part is the ReLU activation function. Kunihiko Fukushima introduced this in 1969. It is a very popular tool used in deep learning today.

Deep learning connects to many things we see in the world. It helps machines understand human language through natural language processing. It can even help in mussel farming with special tools.

These systems can learn from data that has no labels. This is helpful because unlabeled data is very abundant. Deep learning helps computers find patterns that humans might miss. It makes technology feel much more intelligent. This technology continues to change how we use computers every day.

Deep learning is a specialized branch of machine learning.

It uses multilayered neural networks to perform complex tasks. These tasks include classification, regression, and representation learning. The field draws inspiration from biological neuroscience. It involves stacking artificial neurons into layers to process data. The adjective "deep" describes the number of layers in the network. These layers can range from three to several thousand. Deep learning is essential for modern technology. It powers fields like computer vision and speech recognition. It also assists in drug design and climate science.

The mechanism of deep learning relies on a hierarchy of layers. This hierarchy transforms input data into abstract representations. For example, consider an image recognition model. The raw input is often a tensor of pixels. The first representational layer identifies basic shapes like lines or circles. The second layer encodes arrangements of these edges. The third layer might recognize features like a nose or eyes. The fourth layer identifies the entire face. Unlike older machine learning, features are not hand-crafted by humans. The model discovers these useful features automatically from the data. This process allows the system to learn which features to place at which level.

Researchers define "depth" through the credit assignment path, or CAP. The CAP is the chain of transformations from input to output. In a feedforward neural network, the CAP depth equals the number of hidden layers plus one. For recurrent neural networks, the CAP depth can be unlimited. This is because signals may propagate through a layer more than once. While there is no universal threshold for "deep," most researchers agree that a CAP depth higher than two qualifies. A depth of two can act as a universal approximator. This means it can emulate any function. However, adding more layers beyond two helps the model extract better features effectively.

There are many different types of deep learning architectures. Convolutional neural networks, or CNNs, are used for many tasks. Transformers are another common architecture used in modern systems. Recurrent neural networks, or RNNs, have cycles in their connectivity. This differs from feedforward neural networks, which do not have cycles. Other types include deep belief networks and generative adversarial networks. Deep belief networks can be trained in an unsupervised manner. This is a major benefit because unlabeled data is more abundant than labeled data. Neural radiance fields are also a notable architecture in the field.

The history of these ideas spans many decades. The term "deep learning" was introduced by Rina Dechter in 1986. Igor Aizenberg and colleagues used the term in 2000 regarding Boolean threshold neurons. Early neural network ideas were inspired by the human brain. However, current networks are not intended to be high-quality models of biological brain function. In 1965, Alexey Ivakhnenko and Lapa published the Group method of data handling. This was the first working deep learning algorithm. It could train arbitrarily deep networks using regression analysis. In 1979, Kunihiko Fukushima introduced the Neocognitron, an early version of a CNN.

Mathematical proofs have helped define the power of these networks. The universal approximation theorem is a key concept. It concerns the capacity of networks to approximate continuous functions. In 1989, George Cybenko published the first proof for sigmoid activation functions. Kurt Hornik generalized this to multi-layer architectures in 1991. Recent work shows this also holds for non-bounded functions like the rectified linear unit, or ReLU. Kunihiko Fukushima introduced the ReLU activation function in 1969. Today, ReLU is the most popular activation function in deep learning. These mathematical foundations allow networks to handle incredibly complex data.

Deep learning connects to many broader scientific systems. It is used in bioinformatics to understand biological data. In material inspection, it helps ensure quality in manufacturing. It even powers board game programs that can challenge humans. The field also uses backpropagation to train networks efficiently. Backpropagation is an application of the chain rule. While the term was used by Rosenblatt in 1962, the modern form appeared in Seppo Linnainmaa's 1970 thesis. Today, end-to-end stochastic gradient descent is the dominant training technique. This technology continues to push the boundaries of what computers can achieve.