Alibaba Builds AI Accelerator Amid Regulatory Pressure

In response to increasing regulatory pressure from Chinese authorities to reduce dependency on Nvidia graphic processing units, Alibaba, the ecommerce powerhouse, has purportedly created an AI accelerator. This cutting-edge piece of hardware has been expressly designed for AI inference tasks, which pertain more to the application of models than to their development. Alibaba’s T-Heat subunit has been steadily enhancing its AI hardware capabilities over the years.

T-Heat’s groundbreaking venture in this sector started in 2019 when it unveiled the Hanguang 800. Differing from the present-generation chips manufactured by Nvidia and AMD, Hanguang primarily worked with traditional machine learning models such as ResNet—diffusion models or large language models leveraged by AI chatbots and image generators today weren’t in its repertoire.

The newest addition to Alibaba’s chip lineup is stated to be versatile enough to manage a broader array of workloads. This technological leap comes at a time when Alibaba has established itself as a dominant entity in the open model landscape, especially after the release of its Qwen3 series in April.

Alibaba’s initial emphasis on inference hardware does not come as a shock. After all, model application typically consumes fewer resources than model development, making it a prudent first step in Alibaba’s shift towards self-developed hardware. It is anticipated that Alibaba will persist in deriving the benefits of Nvidia’s accelerators for model training in the near term.

The upcoming chip is slated to provide compatibility with Nvidia’s software platform, facilitating engineers to adapt pre-existing code. This might seem to refer to CUDA, Nvidia’s bespoke programming language for GPUs, but it is highly unlikely since CUDA is not required for inference tasks.

Instead, it seems plausible that Alibaba is setting its sights on higher-level abstraction layers like PyTorch or TensorFlow. These platforms predominantly offer a hardware-agnostic programming interface. Thus, they present an attractive target for the ecommerce giant.

The construction of the chip will have to be carried out internally due to the United States’ export regulations on semiconductor technology, which effectively prohibit many Chinese firms from conducting business with companies such as TSMC or Samsung Electronics.

The likeliest candidate to manufacture the chip is China’s own Semiconductor Manufacturing International Co. (SMIC), an entity that has a history of producing the Ascend range of NPUs for telecom giant Huawei. However, the journey towards establishing a domestic chip production ecosystem involves more hurdles than just manufacturing.

AI accelerators are heavily reliant on substantial volumes of speedy memory, typically requiring high bandwidth memory (HBM). HBM has faced its own limitations due to external trade constraints, which might compel Alibaba’s chip designers to opt for slower classes of memory such as GDDR or LPDDR.

Alternatively, Alibaba could choose to utilize their currently stored HBM3 and HBM2e or previous generation HBM2, until Chinese memory suppliers can meet the demand. This issue of memory limitations could conceivably create a bottleneck in Alibaba’s chip production.

The development of this indigenous chip arrives amid the backdrop of the Chinese government’s mounting pressure on regional tech behemoths to steer clear of Nvidia’s H20 accelerators. Meanwhile, Nvidia vehemently rejects the existence of such a directive from the government, stirring up controversy regarding the matter.

Several other Chinese AI firms are also on the hunt for feasible replacements. For instance, DeepSeek has recalibrated its models to operate on freshly developed domestic silicon. Enflame, a startup with financial backing from Tencent, is even venturing into the field with their AI chip, the L600.

MetaX has announced the creation of its own AI chip, the C600. However, the production volumes of these new chips might face obstacles due to existing finite memory reserves. Consequently, this could potentially restrict their widespread application and adoption.

Another standout in the domestic scene is Cambricon, which is diligently developing a homemade accelerator known as the Siyuan 690. This ambitious project from Cambricon promises to outperform the older Nvidia accelerators in terms of computational power.

These collaborative advancements by Chinese tech powerhouses underscore the nation’s drive towards self-reliance in critical technology sectors. Considerable efforts are being directed to navigate around global supply chain restrictions and build indigenous capabilities. While this journey is fraught with challenges, the determination displayed by these firms is noteworthy.

The unfolding saga of Alibaba and other Chinese firms adapting and innovating in the face of diversified challenges stands testament to the resilience of the tech industry. As they continue to carve out their niche, the world eagerly watches, anticipating the technological marvels that might emerge from these endeavours.