This story appears within the August/September 2024 issue of Forbes Magazine. Subscribe to
Armed with newly raised $640 million, Groq believes it may well challenge certainly one of the world’s Most worthy firms with a chip designed from the bottom up specifically for artificial intelligence.
From Richard NievaForbes Employee
JOnathan Ross’ first inkling that something was flawed got here in February, when he was talking to a crowd of Norwegian MPs and tech executives in Oslo. Ross, the 42-year-old CEO of AI chip startup Groq, was in the midst of a demo he hoped would give the floundering company a lift: an AI chatbot that would answer questions almost immediately, faster than a human can read. But for some reason, things were running somewhat slower. That unsettled Ross, who was in the midst of unveiling a European data center operated by Groq that might showcase the special chips accountable for those superfast answers. “I just kept checking the numbers,” he recalls. “People didn’t know why I was so distracted.”
The reason was an influx of recent users. The day before Ross’ Oslo meeting, a viral tweet from an enthusiastic developer raving about “a lightning-fast AI answering machine” circulated. This sent a torrent of recent traffic to the net demo and crashed the corporate’s servers. It was an issue, but an excellent one.
When he founded Groq eight years ago, Ross’s idea was to design AI chips specifically for what is understood within the industry as “inference”: the a part of artificial intelligence that mimics human considering by applying what it has learned to latest situations. This is how your smartphone can discover your dog as a corgi in a photograph it’s never seen before, or a picture generator imagines Pope Francis in a Balenciaga coat. This could be very different from the opposite computing power of AI: training the massive models from scratch.
But by the point OpenAI released ChatGPT in late 2022, sparking a worldwide AI hype, demand for superfast inference was limited and the corporate was sluggish. “Groq almost died many times,” says Ross from the startup’s semiconductor lab in San Jose, California, recalling a low point in 2019 when the startup was a month away from running out of cash. “We may have launched Groq a little too early.”
But now that demand for computing power to construct and run AI models is so great that it’s contributing to a worldwide power shortage, Groq’s time seems to have come – either as a possible blockbuster or as a takeover goal for the established chip giants. On Monday, the corporate announced exclusively, Forbes The company raised $640 million in a large Series D funding round, giving it a valuation of $2.8 billion (up from $1.1 billion in 2021). The round, led by BlackRock Private Equity Partners, also includes participation from Cisco Investments and the Samsung Catalyst Fund, a enterprise capital arm of the electronics giant focused on infrastructure and AI.
“Groq almost died several times.”
The demand for computing power is so insatiable that Nvidia’s market capitalization has risen to $3 trillion, with revenue of $60.9 billion in 2023. Groq continues to be tiny by comparison, with revenue of just $3.4 million in 2023 and a net lack of $88.3 million, in accordance with financial documents obtained by ForbesBut as interest in its chips grows, the corporate has forecast a perhaps optimistic $100 million in revenue this 12 months, sources say, although they doubted the corporate could hit that concentrate on. Groq declined to comment on those numbers.
With the AI chip market expected to succeed in $1.1 trillion by 2027, Ross sees a possibility to grab a chunk of Nvidia’s incredible 80 percent market share by specializing in inference. That market is predicted to be price about $39 billion this 12 months and grow to $60.7 billion over the subsequent 4 years, in accordance with research firm IDC. “Computers are the new oil,” Ross says.
Challengers like Groq are optimistic because Nvidia’s chips weren’t originally designed for AI. When CEO Jensen Huang introduced its graphics processing units (GPUs) in 1999, they were designed for graphics-intensive video games. It was a completely satisfied coincidence that they turned out to be one of the best chips for training AI. But Groq and a brand new wave of next-generation chip startups, including Cerebras (valuation $4 billion) and SambaNova (valuation $5.1 billion), see a possibility. “Nobody started with a blank sheet of paper and decided to build a GPU for this kind of work,” says Andrew Feldman, CEO of Cerebras.
It’s not only startups that need to dethrone Nvidia. Both Amazon and Microsoft are constructing their very own AI chips. But Groq’s chips, called Language Processing Units (LPUs), are so fast that the corporate believes it has an actual likelihood. In a pitch deck for investors, the corporate touts them as 4 times faster, five times cheaper and thrice more energy efficient than Nvidia’s GPUs when used for inference. Nvidia declined to comment on that claim.
“Their inference speeds are proven to be better than anything else on the market,” says Aemish Shah, co-founder of General Global Capital, which has invested in several of Groq’s funding rounds.
Groq began selling its chips two years ago and has since attracted customers corresponding to Argonne National Labs, a government research facility with origins within the Manhattan Project, which has used Groq chips to review nuclear fusion, the shape of energy that powers the sun. Aramco Digital, the technology arm of the Saudi oil company, has also partnered to make use of Groq chips.
In March, Groq launched GroqCloud, where developers can rent access to its chips without having to purchase them outright. To attract developers, Groq offered free access: 70,000 signed up in the primary month. Now there are 350,000 and counting. On June 30, the corporate launched payments and just hired Stuart Pann, a former Intel executive and now Groq COO, to quickly ramp up revenue and operations. Pann is optimistic about growth: greater than 1 / 4 of GroqCloud customer tickets are requests to pay for more computing power.
“The Groq chip is really a slap in the face,” says Meta chief scientist Yann LeCun, Ross’ former computer science professor at NYU who recently joined Groq as a technical advisor. Late last month, CEO Mark Zuckerberg announced that Groq can be certainly one of the businesses providing chips for the inference computations of Meta’s latest model, Llama 3.1, calling the startup “innovators.”
“The Groq chip really goes for the throat.”
Ross cut his teeth at Google, where he worked on the team that developed the corporate’s semiconductor “tensor processing units” optimized for machine learning. He left in 2016 to co-found Groq with colleague Doug Wightman, the corporate’s first CEO. That 12 months, Groq raised $10 million, led by enterprise capital fund Social Capital. But from then on, it was difficult to seek out latest investors. Groq co-founder Wightman left the corporate several years later and didn’t reply to interview requests.
There are still loads of skeptics. One enterprise capitalist who turned down the corporate’s Series D funding round called Groq’s approach “novel” but didn’t consider its mental property was defensible long-term. Mitesh Agrawal, cloud chief at $1.5 billion AI infrastructure startup Lambda, says his company has no plans to supply Groq or other specialized chips in its cloud. “It’s very hard to think beyond Nvidia right now,” he says. Others Ask the cost-effectiveness of Groq’s chips at scale.
Ross knows it’s a tricky battle. “It’s like we’re Rookie of the Year,” he says. “We’re still a long way from Nvidia. So all eyes are on us. And then you ask yourself: What are you going to do next?”
Additional reporting by Rashi Shrivastava, Alex Konrad and Kenrick Cai.
MORE FROM FORBES
