CUDA Programming on Jetson Nano
Purpose of CUDA
Compute Unified Device Architecture (CUDA) is used for parallel computing on CUDA-enabled GPUs.
Architecture
CPU is meant for sequential execution of instructions GPU is meant for parallel execution of control logic.
By using CUDA’s parallel processing feature, it breaks down tasks into thousands of smaller threads that can be executed independently–this would lead to higher efficiency and parallelism.
TLDR: Basically, CUDA allows parallel processing of tasks to speed their applications by using GPU accelerators.
Concurrency
"Dealing lots of things at once"
Executing multiple tasks at the same time by breaking those tasks into smaller parts; however, it is not doing it simultaneously. Concurrency is achieved by context switching (switching in-between tasks quickly).
Context Switch - save and restore state of a task when it is switched out for another one.
Ex: You are cooking a recipe on your own. You can start chopping vegetables while the rice is being cooked by the rice cooker. After chopping vegetables, you can start cooking the steak.
NOTE: Is desirable if we need to perform more than one task but a single resource is available.
Parallelism
"Doing lots of things at once"
Performing/executing multiple tasks simultaneously (at the same time).
Ex: Having a group of friends help you create a food recipe. One is chopping vegetables, one is cooking rice, and the other is cooking the steak.
Asynchronous
It is related to thread execution. If one task is executed, then you can switch to a different task without waiting for that one task to get completed.
Can think of it as setting a timer for a specific task to finish. You can do other things while you wait for that task to finish.
Ex: You are waiting for a program to finish downloading so you move on to other tasks to do.
NOTE: Asynchronous helps achieve concurrency; asynchronous in a multi-threaded environment achieves parallelism.
Data Augmentation
A technique in machine learning to increase the size and diversity of a dataset without collecting new data.
To increase the diversity and size of a dataset, you apply various transformations to existing data.
Why use augmentation? It would make the model more robust and generalize better by exposing it to a wider range of scenarios.