Introduction to Neural Networks

What are Neural Networks?

In the world of artificial intelligence (AI), the term 'neural networks' often emerges in conversations about machine learning. But what are these neural networks? In the simplest terms, they're a set of algorithms designed to recognize patterns, much like the pattern recognition that occurs in the human brain. Today, businesses leverage neural networks to streamline complex tasks, ranging from sorting vast datasets to conducting predictive analyses.

Are they modeled after the human neural networks of the brain?

The human brain is made up of neurons, which are cells that process and transmit information through electrical and chemical signals. These neurons are connected by synapses, and they communicate with each other by sending signals across these connections. When a neuron receives a sufficient amount of signals, it gets activated and passes on the signal.


Neural networks in the context of machine learning are named after the neural networks in our brains because they are a rough, simplified computational model based on how we believe the brain processes information. In artificial neural networks, we have units called 'artificial neurons' or 'nodes,' which are connected by 'weights' that mimic the function of synapses. When we 'train' a neural network, we're adjusting these weights. The 'activation' of these artificial neurons happens through mathematical functions, which are much simpler than the biological processes in our brains.


It is probably more accurate to say that artificial neurons were inspired by, rather than modeled after, actual neurons in the brain, as artificial neurons are greatly simplified compared to biological neurons. They don't capture the full complexity of how brain cells work. Artificial neural networks are 'neural' in the sense that they are inspired by our understanding of biology, but they are not identical to the way our brain's neurons work. They are a simplified abstraction that captures some aspects of biological information processing, which allows them to learn from data and perform tasks like classification, regression, and even play games.

Human Thinking vs. Neural Networks

Consider a situation where you're presented with a vast collection of coins from around the globe, and your task is to sort them efficiently. 


Let’s explore three different methods: relying solely on human judgment, using guided machine learning (supervised neural networks), and applying self-organized machine learning (unsupervised neural networks).

Human-Only Approach

Faced with the challenge of sorting the coins, you would naturally start by organizing them into groups based on observable characteristics such as size, color, and engravings. You would label these groups according to what you know, perhaps naming them “US Currency” or “Euro Coins.” When introduced to new coins, you would determine whether they fit into the existing categories or whether new categories should be created. This process is instinctive, relying on human observation and knowledge.

Supervised Neural Networks

Now, if you were to employ a supervised neural network for the coin-sorting task, the process would start with "training" the neural network with various examples of each coin, specifying details such as the country of origin and denomination. You would show the neural network a coin and label it accordingly: "This is a US quarter," for instance, feeding this information into the system. As the neural network is exposed to more coins, it begins to predict or "classify" each new coin it encounters, drawing on its training to date. When the neural network makes an incorrect prediction—a misclassification—you, as the supervisor, would step in to correct it by providing the right label. Through this iterative process, the network learns and adjusts its parameters, enhancing its ability to correctly classify new coins in the future. In essence, when we talk about the network classifying coins, we mean that it's learning to place them into the correct groups—just as you would—based on the examples it has seen. As it learns, the need for human intervention decreases, and the network's ability to sort and identify coins more rapidly and accurately than a human becomes increasingly valuable. Here are a few of the key benefits that supervised neural networks provide:

  1. Availability of Ground Truth for Accuracy
    Just as a coin collection with labels provided by a coin expert (numismatist) allows for precise categorization, supervised neural networks use labeled data to ensure accurate predictions. The presence of a 'ground truth' allows the network to learn with clear guidance, leading to high levels of accuracy in tasks such as image recognition, speech transcription, or medical diagnosis.
  2. Targeted Learning and Specificity
    The network is trained on specific tasks with clear examples of input-output pairs. This targeted learning allows the network to become highly specialized. For example, it can learn to distinguish between different types of coins (or any other objects) with a high degree of precision, provided it has been trained on enough examples. 
  3. Performance Measurement
    With labeled datasets, it is easier to measure the performance of a supervised neural network. You can directly compare the network's predictions to the true labels to calculate metrics like accuracy, precision, and recall. This is similar to testing the coin-sorting machine by checking its sorted piles against the expert's prior categorizations, providing clear benchmarks for success.
  4. Handling Complex Tasks
    Supervised neural networks can handle complex tasks that have well-defined rules and categories. For instance, they can be trained to detect fraudulent transactions in banking, predict customer turnover, or automate reading and comprehending documents. This is akin to using a sophisticated coin-sorting machine that can not only sort coins by era and region but can also detect counterfeit coins.
  5. Transfer Learning
    Once a supervised neural network is trained on a task, it can often apply what it has learned to similar but slightly different tasks—a process known as transfer learning. This is similar to a coin expert who, after learning to categorize coins from one era, can more easily learn to categorize coins from another era.
  6. Predictive Power
    With their ability to make accurate predictions based on past data, supervised networks are invaluable for applications where prediction is critical. They can forecast sales, weather patterns, or stock market trends, comparable to predicting the value of a coin based on its features known to denote worth.

These benefits demonstrate why supervised neural networks are a cornerstone of many AI applications today. They're essential for tasks where precision is key and labeled data is available, making them a go-to choice for industry and research applications where outcomes need to be predictable and verifiable.

Unsupervised Neural Networks

Turning to unsupervised neural networks, these systems approach the task of coin sorting without any predefined guidance or training. They start by closely examining each coin, autonomously detecting patterns and characteristics—such as size, color, weight, and engravings. Exploring on its own, the neural network groups the coins into various piles or “clusters” (a term we use in AI ) based on the similarities it identifies by itself.


There’s no need for a human to confirm its guesses; the system independently establishes its criteria for grouping and even devises its own unique labels for each cluster. With each new coin it examines, the neural network fine-tunes its technique, enhancing its capability to sort and categorize coins more precisely over time. This approach particularly excels in scenarios where the data lacks predefined categories, empowering the neural network to not only discover groupings a human might have discovered, but also to unearth additional patterns and insights that might elude the human eye, such as metallic content, wear patterns, magnetic properties, weight differences, etc., offering businesses newfound knowledge that was once hidden within their data. Here are a few of the key benefits that unsupervised neural networks bring to the table:

  1. Autonomous Discovery 
    Unsupervised neural networks are akin to a self-taught coin expert who can find patterns in a collection of unsorted coins. They don’t require labeled data to start uncovering structures within the dataset. This allows them to detect natural groupings, correlations, or patterns without prior knowledge or human intervention.
  2. Innovative Insights 
    These networks can reveal unexpected relationships within the data, similar to discovering a new classification system for coins based on previously unseen characteristics like subtle design patterns. This can lead to novel insights, uncovering hidden trends that might not be immediately apparent to human analysts.
  3. Efficiency with Unlabeled Data  
    Unsupervised learning excels when it comes to dealing with vast amounts of raw, unlabeled data. It's like having an assistant who can sort through millions of miscellaneous coins efficiently, something that would be overwhelmingly time-consuming for individuals to undertake manually.
  4. Flexibility in Data Exploration 
    Much like a versatile tool that can sort coins, analyze mineral compositions, or organize library books by content, unsupervised neural networks can be applied across various domains. They can be used for dimensionality reduction, feature extraction, or anomaly detection in different fields without needing specific outcomes defined beforehand.
  5. Scalability and Adaptation 
    Handling large-scale data is where unsupervised neural networks truly shine. They can scale to analyze large datasets much faster than humanly possible, adapting to the complexity and size of the data, similar to processing a growing collection of coins from different time periods and origins.
  6. Pattern Recognition without Bias 
    Since these networks do not rely on predefined labels, they can sometimes detect patterns that are not influenced by human bias. This is comparable to an unbiased sorting of coins based solely on physical attributes, not on perceived historical importance or rarity.

The unique capabilities of unsupervised neural networks make them highly useful for exploratory data analysis, understanding complex datasets, and providing a starting point for further analysis or supervised learning tasks. They're particularly valuable when the cost or feasibility of labeling data is prohibitive, or when the goal is to explore data without preconceived notions of what will be found.

topic next button
Pete Slade
November 26, 2023