An Artificial Intelligence (AI) accelerator accelerates artificial intelligence applications such as artificial neural networks and machine learning. In the last decade, graphics processing units (GPUs) have seen increasing adoption for these applications since they efficiently perform image processing and mathematical neural network calculations. Fortunately for AI development, GPU manufacturers such as NVIDIA are making GPUs that greatly enhance AI performance, opening the door to many new computing products at the edge.  NVIDIA products have led the market with innovation and are widely used in AI computing. However, other specialty GPU providers offer enhanced performance on portions of AI computing, such as AI inferencing. 

What is a GPU?

A modern Graphical Processing Unit or GPU is similar to a CPU but makes use of parallel processing and is able to handle many processes and threads at the same time. Due to its parallel processing, a GPU is normally used for graphics processing and rendering.

GPUs have been around since the 1970s, primarily used for arcade games such as Sea Wolf and Space Invaders. Graphics cards were not commonly used in PCs until the mid-1980s, when the NEC μPD7220A became the first processor with a Large Scale Integration circuit chip, making it the most popular GPU. The next significant innovation was in the 1990s with S3 Graphics making the S3 86C911. This new GPU used 2D acceleration to gain a massive performance increase compared to its competitors. Today, a GPU is one of the most crucial hardware components of computer architecture.

Initially, the purpose of a video card was to take a stream of binary data from the central processor and render images to display. But modern graphics processing units are engaged in the most complex calculations, like big data research, machine learning, and AI.

While AI has existed since the 1990s in inference and training, AI accelerators did not enter the market until 10 years ago, when the workloads of AI processes became more intensive. Since 2010, AI accelerators such as field-programmable gate arrays (FGPA) and customized application-specific integrated circuits, (ASIC) are being replaced by commercial GPUs, which offer faster time to market and lower development costs. 

The GPU Market

Currently, the GPU industry is dominated by three large companies: NVIDIA, Intel, and AMD. These companies have been around for the longest, with NVIDIA pioneering the discrete GPU market since 2000 and putting them into a firm leadership position.  New entrants into the GPU market are picking a product niche and developing solutions that are efficient and designed to fill the appetite for higher performance with lower costs per watt of power. Up to 90% of the power consumed by an image processing application is accessing the RAM. Today, there are new entrants with differing approaches to building more efficient GPUs, including Mythic.AI, UntetherAI, Hailo, Blaize, etc., for edge AI computing.  These companies focus on avoiding data transfer between computing and memory to drive efficiency.  

GPU vs CPU

In the past, CPUs were used to process information for artificial intelligence. However, as GPUs have become more powerful over time, their ability to process more parallel information faster has made them a much better solution for AI. 

Integrating a GPU into an application:

GPUs are typically shipped as modules to be easily implemented into various applications. Chip-down GPU designs are intensive hardware and software projects; also, GPU vendors historically will only support Tier 1 customers and projects and encourage the rest of the applications to design a carrier that integrates their modules. 

Here are some common GPU module form factors:

PCIe

• This is the first and still most common form factor for GPU modules and is easily integrated with common PC motherboards with up to x16 Gen5 PCIe connections. However, the disadvantage is that they are large and bulky for many edge AI applications and typically require forced air to cool them.

MXM (Mobile Express Module)

• As the name indicates, the MXM form factor was developed to offer graphics processing module capabilities to smaller mobile computers such as laptops. It is also commonly used in edge SFF computing applications in markets such as military, medical, and transportation. The modules can be air cooled or conduction cooled.

M.2

• The M.2 standard replaces the mSATA standard and offers size and speed advantages for storage.  Gen4 PCIe x4 connections to the processor allow it to be also widely used for other computing functions, including GPS, LTE, IO, and smaller GPUs by vendors such as Hailo.  

E1.S EDSFF (Enterprise and Datacenter Small Form Factor)

• E1.S is the choice of next-generation storage modules.  It offers greater density and performance than the M.2 and other earlier form factors.

• Although it is being deployed in the datacenter server industry, it has not yet replaced M.2 as a standard in SFF Edge Computing.  
• Blaize is an example of a GPU being deployed in the ES.1 SFF, enabling up to 512 TOPS in a 1U server and 16-64 TOPS in an SFF computer.

SOM (System on Modules)

• System on Modules has been made popular for industrial edge computing by NVIDIA in products such as JETSON.
• Embedded ARM processors in the SOM enable a solutions provider to build a smaller, low-cost edge AI or graphics processing computer without needing a separate embedded CPU.

Chip-Down

• With many edge platforms, it is the most cost-effective to design a solution with a ‘chip-down’ GPU and a ‘chip-down’ CPU.  Tauro Technologies has designed these systems for several of the GPU vendors mentioned above.

GPU integration into a carrier board:

System design takes many factors into consideration, including cooling, power, I/O, storage, processing, etc., and each system requirement is different. Typically, GPU modules that are not chip-down are integrated into a main board or carrier board that has been customized to meet the application requirements.  

There are 2 main types of processors integrated into carrier boards:

• Intel x86: Due to the engineering complexity of Intel chip-down designs, many edge AI applications choose to use COMe or COM-HPC client-based modules to simplify their carrier design projects. Although the material cost of the module is higher than a chip-down solution, unless the product reaches modest volumes, it is more cost-effective to design with an x86-based module. 

• ARM-based processors: ARM processors are typically lower cost and take less power, making them ideal for many computer vision products when paired with a GPU.  Since there are few modules available based on open standards, most ARM-based products are custom designed with a chip-down processor such as NXP Cortex or Layerscape.  

Summary:

The advent of AI is opening the door for application-specific GPUs that are finely tuned to the objectives of the project. Tauro Technologies has broad experience implementing edge computers optimized for the application. If you wish to discuss a customized high-volume platform or a system which takes advantage of commercially available hardware and tools, reach out to us.

The post Use of GPUs in Edge AI Computing: appeared first on Tauro Technologies.

Serial communications protocols are vital to embedded systems.  While UART, I²C, and SPI have been used for short-distance device communication for decades, the benefits are not entirely apparent.

In order to connect peripherals to a computer, one of the following protocols is typically employed: a Universal Asynchronous Receiver Transmitter, (UART) Inter-Integrated Circuit, (I²C) or Serial Peripheral Interface (SPI). This blog will compare and contrast the features of each protocol and help you determine the best fit for your application.   

UART

Universal Asynchronous Receiver Transmitter (UART) is an asynchronous serial communication device with its roots dating back to the telegraph. There is no clock signal to synchronize or validate the data transmitted from the transmitter and received by the receiver (Asynchronous Serial Communication). It sends 1 bit at a time from least significant to most significant and uses start and stop bits to enable precise clocking. During packet transmission, UART uses what is called a parity bit, to enable checking if the information has changed during transmission. 

In addition, data transmission between devices can be in simplex, half-duplex, or full duplex modes.

Data is transmitted at baud rate measured in bits per second – some of the standard baud rates are 4800 bps, 9600 bps, 19200 bps, 115200 bps etc. Out of these, 9600 bps baud rate is the most commonly used.

UARTs must be set for the same bit speed, character length, parity, and stop bits for proper operation on the transmit and receive side. If the receiving UART detects mismatched settings a flag is sent in the host system memory to indicate a failure. 

The data in UART serial communication is organized into blocks called Packets or Frames. The structure of a typical UART data packet or the standard framing of the data is shown in the following table:

Advantages:

• Management is straightforward through hardware. It is utilized by standard protocols including RS-232/485/422.
• Long-distance up to 1km for RS-422/485 buses.
• Requires only two wires for full-duplex data transmission (other than power lines).
• Parity bit ensures basic error checking is integrated into the data packet frame.
• No need for clock or any other timing signal.

Disadvantages:

• Communication is only between two devices where the baud rate, data bit count, parity bit, and stop bit count need to be identical.
• Typically the size of the data frame is limited to only 9 bits (8 data bits, no parity bit and one stop bit).
• Overrun errors if the buffer space is insufficient.
• Size of data in the frame is limited.

I²C

Unlike UART, an Inter-Integrated Circuit is a synchronous serial communication interface and utilizes the system clock. It means that data bits are transferred one by one at regular intervals of time set by a SCL clock line. It is used primarily for short distance, intra-board communication between low speed controllers and processors and is ideal for applications that link up to many components on a bus. Although I²C is typically implemented with a single master and multiple slaves on the bus, it can also be implemented with multiple masters.  Each slave device has a unique address and I²C enables the master to send and request data from a particular slave device utilizing a start bit to the slave address.  

I2C only uses two wires to transmit data between devices:

• SDA (Serial Data) – The line for the master and slave to send and receive data.
• SCL (Serial Clock) – The line that carries the clock signal (common clock signal between multiple masters and multiple slaves).

The structure of a typical I²C Data Packet or the standard framing of the data is shown in the following table (Note: The bold signals are sent by slave and the other signals by master):

Advantages:

• Addressing function enables multiple masters and slaves.
• Control a network of devices with only 2 I/O pins.
• Simple mechanism for validation of data transfer.
• I²C networks are easy to scale. New devices can simply be connected to the two common I²C bus lines.
• No need for prior agreement on data transfer rate as in UART communication.

Disadvantages:

• Slower speed (up to 100 kbit/s in standard mode, 400 kbit/s in fast mode)
• Half-duplex interface.
• Only one slave can be addressed at a time.

SPI

The Serial Peripheral Interface (SPI) is also a synchronous serial communication device which is used primarily for short distance communication. The main difference between SPI and I²C is that SPI uses a full-duplex communication with master-slave topology. Similar to I²C , SPI can be used to access multiple slave devices.

At the beginning of communication, the bus master configures the clock (typically 50 MHz) and sends data to the slave.  During a single SPI clock cycle, a full duplex of data transmission is completed.  Unlike UART, there are no start and stop bits – this enables continuous data transmission and the communication achieves higher speeds than I²C and UART.

The maximum data rate limit is not specified in the SPI interface. Standard data rates include 10 Mbps transfer rate with some devices reaching 100Mbps transfer rate.

The SPI bus consists of 4 signals below:

• Master – Out / Slave – In (MOSI)
• Master – In / Slave – Out (MISO)
• Serial Clock (SCLK)
• Chip Select (CS) or Slave Select (SS)

Depending on the values of Clock Polarity (CPOL) and Clock Phase (CPHA), there are 4 modes of operation of SPI:

• Mode 0 is active when Clock Polarity is LOW and Clock Phase is LOW (CPOL = 0 and CPHA = 0). Data sampled on rising edge and shifted out on the falling edge.
• Mode 1 is active when Clock Polarity is LOW and Clock Phase is HIGH (CPOL = 0 and CPHA = 1). Data sampled on the falling edge and shifted out on the rising edge.
• Mode 2 is active when Clock Polarity is HIGH and Clock Phase is LOW (CPOL = 1 and CPHA = 0). Data sampled on the falling edge and shifted out on the rising edge.
• Mode 3 is active when Clock Polarity is HIGH and Clock Phase is HIGH (CPOL = 1 and CPHA = 1). Data sampled on the falling edge and shifted out on the rising edge.

The structure of a typical SPI data packet or the standard framing of the data is shown in the following image:

Advantages:

• Full-duplex is default for the SPI protocol.
• Slaves do not require a unique address.
• Not limited to 8-bit word size.
• Real-estate savings on embedded boards.
• High data transfer speed.
• No need for individual addresses for slaves as CS or SS chip-select lines are used.
• Only one master device is supported, removing the possibility of conflicts.
• SPI uses less power than I²C.

Disadvantages:

• No protocol-level error checking function and no hardware slave acknowledgement.
• Short distances (up to 10m).
• Each additional slave requires an additional dedicated pin on the master for CS or SS.
• There is no acknowledgement mechanism and hence there is no confirmation of data receipt.
• Slowest device determines transfer speed.

Summary

In general, you can use UART if you are looking for a simple connection between 2 devices, I²C if you are connecting several devices on the same bus, and SPI becomes the ideal choice if you require a faster interface.  Whether you need UART’s tried and true operation, or want to utilize the expansion offered by I²C or the high speed of SPI, Tauro Technologies can implement a system using the most appropriate interface for your project.  

Interested to know more? Get in touch with us for details

The post Serial Protocols & Their Uses: I2C, UART, SPI appeared first on Tauro Technologies.