Learning Objectives
- Trace an inference pipeline end to end.
- Distinguish model weights, activations, and input data.
- Explain why tensors move repeatedly between memory and compute.
- Understand operator mapping to CPU/GPU/NPU.
- Explore how model complexity affects latency and memory.
Theory & Fundamentals
Input preparation. Images, audio, text, or sensor signals are converted into numerical tensors in the format expected by the model.
Weights. Trained model parameters are stored in memory and used by each layer during inference.
Operators. Layers invoke operations such as convolution, matrix multiplication, normalization, and activation functions.
Output. The final tensor is interpreted as a class probability, bounding box, token probability, anomaly score, or control quantity.
Unique Visualization
This figure is specific to AC-11 and illustrates the core architecture or dataflow discussed in this lesson.
Interactive Experiment
Compute demand—
Weight memory—
Activation pressure—
Inference class—
Increase model size and observe both compute and memory demands.
Real-World Case Study
For defect classification, a camera frame is resized and normalized, the NPU executes convolutional layers, and the output probabilities are converted to ACCEPT or REJECT.
Case-study task: Identify the most important compute, memory, power, and data-flow requirements in this example.
Engineering Challenge
Identify which parts of an inference pipeline are compute-heavy and which are data-movement-heavy.
Common Misconceptions
- Headline specifications should not be interpreted without workload context.
- Compute, memory, data movement, software, and power interact as one system.
- More hardware resources can show diminishing returns when another subsystem is limiting performance.
Interactive MCQ Quiz
1. What are model weights?
2. What is an activation?
3. What happens before inference?
Select your answers and submit the quiz.
Nexa AI Chip Tutor™
The future connected tutor can explain How an AI Model Runs on a Chip, interpret this module's experiment, and answer learner questions based on the current settings.
Suggested prompt: “Explain the experiment in AC-11 and tell me why the bottleneck changes when I move the sliders.”
Downloads
References & Further Study
- Computer architecture and digital systems textbooks
- Semiconductor and processor manufacturer educational documentation
- AI hardware, edge-computing, and embedded-systems documentation