China’s artificial intelligence groups are selling “AI-in-a-box” products for companies to run on their own premises, in a threat to the AI cloud computing services offered by the country’s big tech groups such as Alibaba, Baidu and Tencent.
Huawei has signed up more than a dozen AI start-ups to bundle and market their large language models with its AI processors and other hardware. Its partners include groups such as Beijing-based Zhipu AI and language specialist iFlytek.
Chinese groups are deploying the boxes to bring the advances of generative AI to on-premise, or private cloud, set-ups, which account for about half of the cloud market in the country.
Huawei estimates the Chinese market for “all-in-one machines”, as they are known locally, will hit Rmb16.8bn ($2.3bn) this year. Analysts at Minsheng Securities forecast the government market for AI boxes could reach Rmb450bn by 2027.
Liu Qingfeng, iFlytek founder, said at a product launch event for its AI box last year that the company’s “all-in-one machine has top-notch performance, is safe and controllable and is ready to use right out of the box”.
The moves diverge from how AI is being commercialised in the west and capitalises on concerns at Chinese companies about protecting their data.
The trend could crimp the ambitions of tech giants that have invested heavily in building AI infrastructure and large language models that can be sold as a service to customers over the cloud. Baidu’s Robin Li has laid out a vision of hundreds of AI apps running on top of the company’s foundational models.
The proliferation of AI-in-a-box could also solidify the split of the Chinese cloud market.
During China’s first rollout of cloud computing, Huawei established itself as the dominant provider for private clouds, especially for deep-pocketed government and state-owned groups. Alibaba and Tencent were left vying to provide computing power mostly to other internet groups.
This became a problem after Beijing cracked down on their tech customer base in 2021, upending the business models of key customers such as online education companies. Alibaba’s cloud arm has averaged single-digit quarterly growth since then.
“Baidu and Alibaba focus on the public cloud, but the ecosystem in China and the US are very different,” said a local investor. “They will need to adapt to survive.”
Baidu last week said generative AI and foundational models contributed sales of Rmb324mn in the first quarter. The company has also begun to sell all-in-one machines, but analysts say tech giants have less of an advantage in this market, where it is harder to utilise their data centres with vast pools of processing power.
Dylan Patel, chief analyst at research group SemiAnalysis, said that while some Chinese companies might want on-premise AI, it would be inefficient compared with public cloud or using APIs to connect to large language models.
“Usage is going to be very sporadic, which means you’re going to have all this very expensive AI hardware that is not utilised properly,” he said.
Research notes from Chinese investment banks have played up security lapses among western AI groups, such as when OpenAI’s ChatGPT accidentally shared users’ search histories or Samsung employees allegedly leaked trade secrets to the chatbot.
“Organisations need to be able to protect their data, and building a private cloud is the way to prevent leaking valuable data,” iFlytek’s Liu told potential customers.
Kent Fan, a banker at tech advisory China Renaissance, said all-in-one boxes helped get around China’s computing power shortages brought about by Washington’s export controls on advanced chips.
Procurement records show China’s large state-owned groups have a preference for private cloud AI.
A smart city services provider in Chengdu recently put out a Rmb2mn tender to run China Unicom’s large language model on Huawei’s all-in-one machine hardware.
China National Nuclear is also looking to build an on-premise system bundling hardware with a model that has more than 10bn parameters, variables used to train a system and ultimately shape its output. Start-up Zhipu AI is in the running for that project, according to a person familiar with the matter.
Zhipu charges Rmb1.8mn a year for a 12bn-parameter model packaged with eight Huawei Ascend 910 chips, according to Minsheng Securities. The Ascend 910 is Huawei’s most advanced AI chip with the processing power to train large models and handle “inferencing”, or responding to user queries.
Huawei, Baidu, Alibaba and Tencent declined to comment.
