This article is co-authored by:
• Tim Dry, Director of Edge and Endpoint Segment Marketing, Samsung Foundry
• Santiago Fernandez Gomez, Vice President of Platform Engineering, Blaize
• Mahadev Kolluru, Corporate Vice President of North America Platform Solutions Sales, VeriSilicon
Blaize’s novel stream processor for Edge AI offers a case study of new opportunities for smaller companies to leverage semiconductor industry resources in pursuit of their goals.
Until now, most processor chips used for AI applications have been adaptations of devices, like GPUs, that were initially developed for other purposes. This is partly because those existing devices have proven effective enough to be useful, and partly because most companies have seen development of AI-specific processors as prohibitively costly, complex, and/or risky.
But, with the AI market poised for intense expansion, diverse new applications at the edge and in endpoints will require silicon that’s more closely tailored for specific tasks and situations, removing the need to go to the cloud for AI processing. We are now starting to see indications of how the semiconductor sector is evolving tmeet those needs, and what it will mean for the evolution of the AI marketplace and those who seek to enter it.
A useful case study is provided by Edge AI startup Blaize, which is pursuing customers in industrial, smart city, automotive sensor fusion, last-mile delivery, and retail applications with its recently launched Pathfinder and Xplorer system-on-module platforms and accompanying software tools.
The company notes that existing Edge solutions for these sectors are “either too small to compute the load or too costly and too hard to productize.” Blaize aims to remedy this with a combination of multi-sensor processing power, compact size, extremely low power consumption, and easy programmability, powered by a novel Graph Streaming Processor (GSP) developed in collaboration with Samsung Foundry and design/IP firm VeriSilicon.
Why a Stream Processor?
Stream processors offer several advantages in real-time analysis of multiple sources of continuous graphical data, like that produced by arrays of cameras on factory floors or retail stores, or by coordinated combinations of sensors. Unlike traditional batch-oriented processors, stream processors do not have to temporarily store data for processing, or re-aggregate batches of data. This greatly reduces memory requirements and latency while boosting processing efficiency for tasks often needed in Edge AI situations.
By way of example, Blaize’s GSP implementation allows the Pathfinder module to provide five independent neural networks for video streams at 10 frames per second (50 fps total), with less than 100ms latency, while also supporting multiple simultaneous workloads across different streams. This enables sensor fusion, object detection, image noise reduction, fusing of video and lidar/radar data, and other functionality depending on the use case, with the platform having 100% programmability. And remarkably, despite providing 16 TOPS of processing power, the GSP draws just 7 watts.
As appealing as that is from a strategic perspective, bringing those capabilities to market in reliable, affordable silicon is a formidable task that, until recently, few companies would be willing or able to tackle. It’s well beyond the scope of typical custom chip projects, requiring a vast range of advanced design and manufacturing capabilities, integration of diverse circuit IP, and management and coordination of large interdisciplinary development teams — traditionally the province of big, specialized enterprises, not startups.
And in addition to fundamental development hurdles (timing, physical layout, memory management, etc.), both business and technical considerations require rapid and dependable time to market. This type of project generally offers only one chance to get it right, which amps up the risk-reward considerations.
Strategic Choices for Successful Development
As a result, when Blaize began its initial engagement with Samsung Foundry and explained the project goals and requirements, the foundry team recommended bringing in VeriSilicon, a one-stop source for custom silicon solutions and semiconductor IP, with an extensive track record including other AI-related projects. As a member of the Samsung Advanced Foundry Ecosystem (SAFE™) partnership program, VeriSilicon had also seen many of its design projects through the production process with Samsung, and drew on that experience for both strategic planning and the tactics used during development.
One of the first and most important decisions was the choice of manufacturing process node. After considering all available options, Samsung Foundry’s 14nm FinFET process emerged as the ideal choice, beating out several later nodes.