Learning Objectives
- Define TOPS, FLOPS, latency, throughput, utilization, and efficiency.
- Distinguish theoretical peak compute from achieved application performance.
- Explore the effect of numerical precision.
- Calculate simple throughput from latency.
- Understand why benchmark conditions matter.
Theory & Fundamentals
TOPS and FLOPS. These metrics describe operation rates under specified arithmetic assumptions. INT8 TOPS and FP16 FLOPS are not directly interchangeable.
Latency. Latency is the time required for one inference or task. Real-time control often cares strongly about latency.
Throughput. Throughput describes how many tasks complete per unit time. Batch processing may favor throughput even when individual latency is higher.
Efficiency. Performance per watt matters because power and cooling are limited, especially in edge devices.
Unique Visualization
This figure is specific to AC-12 and illustrates the core architecture or dataflow discussed in this lesson.
Interactive Experiment
Ideal throughput—
Throughput/W—
TOPS/W—
Realtime class—
Relate latency, throughput, peak compute, and efficiency.
Real-World Case Study
Two accelerators can advertise similar TOPS but deliver different frame rates if one has better memory bandwidth, software optimization, or supported operator coverage.
Case-study task: Identify the most important compute, memory, power, and data-flow requirements in this example.
Engineering Challenge
Calculate ideal throughput for 20 ms, 10 ms, and 5 ms inference latency. Then explain why real throughput may differ.
Common Misconceptions
- Headline specifications should not be interpreted without workload context.
- Compute, memory, data movement, software, and power interact as one system.
- More hardware resources can show diminishing returns when another subsystem is limiting performance.
Interactive MCQ Quiz
1. What does latency measure?
2. What does throughput measure?
3. Can equal TOPS guarantee equal application speed?
Select your answers and submit the quiz.
Nexa AI Chip Tutor™
The future connected tutor can explain TOPS, FLOPS, Latency & Throughput, interpret this module's experiment, and answer learner questions based on the current settings.
Suggested prompt: “Explain the experiment in AC-12 and tell me why the bottleneck changes when I move the sliders.”
Downloads
References & Further Study
- Computer architecture and digital systems textbooks
- Semiconductor and processor manufacturer educational documentation
- AI hardware, edge-computing, and embedded-systems documentation