Parsing AI: A Glossary of AI Concepts
Artificial intelligence (AI) has been embedded in a wide range of products, from smart speakers to motor vehicles to industrial robots. AI may seem ubiquitous already, but an almost limitless number of products that do not yet feature AI could profit from the addition.
While some OEMs have the resources to develop in-house AI expertise, not all have that luxury. AI is a complex engineering discipline, and a rapidly evolving one. Collaborating with a partner that has ample experience with selecting the data sets, tools, software and hardware components appropriate for your application will reduce development time and risk.
A good first step toward exploring the adoption of AI technology is gaining familiarity with concepts and the specific engineering terminology associated with them. One of the difficulties with learning about AI is that some of the concepts are interdependent, and terms that aren’t strictly interchangeable can be used for each other. This can streamline verbal communications, but it can be confusing to the uninitiated. Compounding the confusion, any two organisations might have slightly different definitions for any given term.
Our purpose here is to be neither comprehensive nor fully definitive. It is to introduce the uninitiated to some of the most common concepts and terms in AI.
Traditional computing vs. AI
Traditional computers are programmed with explicit instructions that produce specific, predictable, and reproducible results when executed on properly formatted data.
An AI employs algorithms that allow it to respond to new inputs, consistent with what it has already learned, without any additional programming. The algorithmic approach makes it possible for changes in input to result in changes in the output – it is possible for the system to adapt, or learn. In this manner, AI aims to mimic the capabilities of human brains.
Training and inference
An AI is trained by having it process prepared data. Organisations can train their AIs on proprietary data, on standardised data sets, or a combination of both, as long as it is representative of the data the AI will handle when performing its eventual function.
Inference typically refers to the process of a trained AI performing its function.
This division currently has profound ramifications for companies that design processors. Graphics processing units (GPUs), designed for parallel processing enormous data sets, have proven highly suitable for training.
Figure 1: Machine learning (ML) can be used in equipment used in a variety of applications, including factory automation, medical, retail, human-machine interfaces (HMI), smart cities, and smart appliances. The use of ML creates the potential that any device it is embedded in can become increasingly capable and useful over time.
GPUs are commonly used for inference as well, but CPU and microcontroller vendors argue their products can handle inference workloads as well (if not better) and almost always more cost-effectively. Also, there are companies that design processors specifically for either learning or inference. These devices can be labelled neural processing units (NPU).
Discover more about CPUs, GPUs, and NPUs in our overview article "Artificial intelligence and the question of processors".
Neural network models
AIs are neural networks. A neural network is composed of processing nodes called neurons. Neurons are organised into layers. Any neural network typically includes an input layer, an output layer, plus any number of layers in between. Internal layers are described as hidden in that they connect only to each other and the I/O layers.
While a neural network with the minimum number of layers can make useful, approximate predictions and decisions, additional layers typically help refine and optimise results for greater accuracy.
There are multiple models for implementing a neural network.
ANN – artificial neural network. ANN is commonly used as a generic synonym for any neural network. As such, it is less frequently encountered than it used to be, but it is still used.
CNN – convolutional neural network. CNNs have multiple hidden layers, including at least one layer that performs convolutions. CNNs can use many fewer neurons per layer to process vast streams of data, making them particularly suitable for workloads involving images or video. CNNs are typically DNNs; DNNs do not have to be CNNs. Also, CNNs are synonymous with SIANNs.
DNN – deep neural network. DNNs are defined by having at least three hidden layers – making them “deep”. They are the foundation for a wide variety of AI-based applications. Common examples include voice-enabled devices such as smart speakers and smart TV remotes, self-driving cars, and generative AIs, including many of those now being incorporated in search engines.
GAN – generative adversarial network. A GAN is actually two neural networks trained on the same data sets and paired to contest with each other. The result is new data – albeit based on the training set. This approach is the foundation of generative AI, which can train on language, images, video, or code and generate new instances of any of it.
LLM – large language model. An LLM is optimised for general-purpose language generation, typically for text generation, though software code generators usually fall into this category as well. LLMs can be based on GANs, but many different companies have their own models upon which they base their LLMs.
SIANN – shift (or space) invariant invariant artificial neural network. (See convolutional neural network).
Some of the relatively less common model variations include optical neural networks (ONN), quantum neural networks (QNN), recurrent neural networks (RNN), and spiking neural networks (SNN).
ML – Machine learning. ML algorithms are commonly designed to handle structured, labelled data to make predictions. The data is usually labelled by humans. If input data is unstructured, it is usually formatted before being processed.
DL – Deep learning. DL is a subset of machine learning. DL relies on algorithms optimised to handle unlabelled and unstructured data. DL generally relies on DNNs of one type or another.
Learning models
There is another way of looking at ML that focuses on the nature of the data rather than on the models for implementing the neural network (CNN, DNN, QNN, etc.). This makes distinctions between supervised learning, unsupervised learning, and reinforcement learning.
- Supervised learning uses labelled datasets to categorise or make predictions. This generally maps to ML.
- Unsupervised learning does not require labelled datasets. Instead, it detects patterns in the data and makes categorisations based on those patterns. Generally maps to DL.
- Semi-supervised learning combines the use of labelled and unlabelled data sets.
- Reinforcement learning uses feedback to produce more accurate results. Generally maps to DL.
An example of how all of these distinctions apply involves GAN. GANs were originally proposed as a generative model for unsupervised learning, but over time they were applied to semi-supervised learning, fully supervised learning, and reinforcement learning.
