Deepak Pathak (left) and Abhinav Gupta (right), former professors at Carnegie Mellon University, founded Skild AI in 2023 to construct a “general-purpose brain” for AI
Separate AI
Robots, whether bipedal humanoids doing easy factory work or four-legged military “robot dogs” designed for urban warfare, need brains. In the past, these have been highly specialized and purpose-built. But a Pittsburgh-based robotics startup claims it has developed a single, off-the-shelf intelligence that might be built into various robots to enable basic functions.
Founded in May 2023 by Abhinav Gupta and Deepak Pathak, two former Carnegie Mellon University professors, Skild AI has developed a basic model for what it calls a “general-purpose brain” that might be built into various robots, enabling them to do things like climb steep slopes, walk over objects blocking their path, and recognize and pick up items.
The company announced Tuesday that it had raised $300 million at a $1.5 billion valuation in a Series A funding round led by Lightspeed Ventures, Softbank, Coatue and Amazon founder Jeff Bezos, with participation from CRV, Felicis Ventures, Menlo Ventures, Amazon and General Catalyst, amongst others.
Raviraj Jain, the Lightspeed partner who also led the corporate’s seed round in July 2023, said Forbes He was impressed by Skild AI’s models when he first saw them stress tested last April. Robots using them were capable of perform tasks in environments that they had never seen before and weren’t designed for demos. “The robots were able to climb stairs at the time, and I think it’s really crazy how well they could do that, because it’s a very complex stability problem,” he said.
Even more impressive, the robots that used Skild’s AI models also demonstrated “emergent abilities” – entirely latest skills that they weren’t taught. These are sometimes easy, like picking up an object that slipped out of your hand or rotating an object. But they reveal the model’s ability to perform unexpected tasks, an inclination that happens in advanced artificial systems equivalent to large language models.
Skild did this by training its model on a large database of text, images and videos – a database that it says is 1,000 times larger than its competitors’ databases. To create this massive database, the co-founders, each former AI researchers at Meta, combined a mixture of information collection techniques that they had developed and tested over years of research.
One option was to rent human contractors to remotely control robots and collect data on those actions. Another option was to have the robot perform random tasks, record the outcomes, and learn through trial and error. The AI model was also trained using hundreds of thousands of publicly available videos.
As a graduate student at UC Berkeley, Pathak developed a technique to instill “artificial curiosity” in robots by rewarding the system for outcomes that occur when it cannot predict the implications of its actions. “The more uncertain the agent is about predicting the effects of its actions, the more curious it becomes to explore,” he explained. The technique motivated the AI to run more scenarios and collect more data.
Be Research on curiosity-driven learning was published in 2017 and has been cited greater than 4,000 times, he said. Pathak also developed a technique for robots to take written information from large language models like GPT (for instance, easy methods to open a can of milk) and convert it into actions.
“In 2022, we have found a way to bring these things together into a single coherent system,” said Pathak. “The idea of learning from videos, learning from curiosity, learning from real data, but combined with the knowledge from simulation.”
Skild AI’s basic model, trained with 1,000 times more data than its competitors, might be built into a wide range of robots, allowing them to do things like climb steep slopes, walk over obstacles of their path, or detect and pick up objects.
Separate AI
Skild AI faces stiff competition from various robotics firms which have entered the market with billions of dollars in enterprise capital due to the AI boom. Industry giant OpenAI recently revived its robotics team to offer models to robotics firms. Forbes first reported. Then there are firms like humanoid robotics company Figure AI, led by billionaire CEO Brett Adcock, and Covariant, an OpenAI spin-off that’s developing ChatGPT for robots and has raised over $200 million for it.
Co-founder Gupta claims that Skild AI’s access to large amounts of information sets it other than others within the space, but declined to reveal exactly how much data its model is trained on.
Ken Goldberg, a professor of robotics and automation at UC Berkeley, agrees that data is vital to scaling robotics. However, robots require a selected sort of data that may not widely available on the web. And data from simulations doesn’t all the time translate to the actual world.
“The whole idea that’s so exciting about robotics right now is that we can do something analogous to large language models and large visual language models, where both can access data at internet scale, where there are billions of examples,” he said. That’s not a sure bet for robotics, but Skild AI goals to unravel the issue by combining all of its data collection techniques with more information from simulations.
Pathak and Gupta envision a future for his or her company just like OpenAI, where different use cases and products might be built on Skild’s basic model by fine-tuning it. “This is exactly how we want to revolutionize the robotics industry,” Gupta said, adding that they ultimately want to attain artificial general intelligence (a hypothetical AI system that may match or surpass human capabilities) for robots, but with which humans can interact within the physical world.
“There is a GPT-3 moment coming in the world of robotics,” said Stephanie Zhan, partner at Sequoia Capital and already an investor in Skild AI. “It will trigger a monumental shift that will bring advances similar to those of digital intelligence to the physical world.”
MORE FROM FORBES
