“iPhone moment” for the ai era – Nvidia supports Swedish supercomputer

The Swedish supercomputer Berzelius will get new superpowers – with the help of the American company Nvidia. Equipped with half an exaflop, the goal is to accelerate the development of artificial intelligence. “This is the iPhone moment of the AI ​​era,” says Nvidia CEO Jensen Huang during his visit to Stockholm.

Chat GPT is on everyone’s lips, including Anders Ynnerman’s and Jensen Huang’s. The duo is on site at an event in central Stockholm, where an upgraded version of Berzelius, the supercomputer in Linköping, is being presented.

– We have all seen the revolutionary change Chat GPT has brought about, says Anders Ynnerman, who leads the work with the research project Wallenberg Artificial Intelligence, Autonomous Systems and Software Program (Wasp).

Chat GPT is based on an AI architecture called large language models. However, researchers at Stanford have suggested that their potential is so far-reaching that they should be called “foundation models”. A basic, broad and general ai architecture in other words.

Read more: ChatGPT shows what ai can do – “Becoming hard to resist”

AI models of that kind have to be trained on large amounts of data before they are put to use. The more training data, the more the model is scaled up, the greater the functionality, it has been shown. Now Knut and Alice Wallenberg’s foundation has pushed to SEK 300 million so that Sweden can keep up with the pace. The money will be used to upgrade Berzelius from 60 to 94 nodes of Nvidia’s DGX A100 system. The work is expected to be completed in the second quarter of 2023, making Berzelius one of the hundred most powerful supercomputers in the world.

The Swedish supercomputer Berzelius. Photo: Wasp

– The discussion about building Sweden’s most advanced supercomputer began two years ago when Marcus (Wallenberg) and I talked about how artificial intelligence can improve and accelerate some of the most important areas of science: the life sciences, climate research and materials research, says Jensen Huang.

Large language models immediately bring to mind spoken language. Generative ai tools such as Chat GPT rely on the model predicting which words will follow when fed with a text. But as Jensen Huang points out, it doesn’t have to be like Swedish, English or Spanish. A language model can just as easily be trained in the language of biology.

Read more: Google at full throttle as Microsoft invests heavily in Open AI

– Researchers can generate proteins with valuable functions or discover chemicals that are not found in nature but have medicinal properties.

Advances in the life sciences are highlighted as a particularly important area for Wasp’s ambitions with Berzelius. Recently, the British company Deepmind showed that their AI model can predict how proteins fold, a problem that until then had eluded science for over half a century. Wasp wants to build on that knowledge.

– When researchers gain insight into how proteins fold and thus their function, we gain deep insights into how the human body functions. Maybe we can even find a cure for cancer, says Anders Ynneman.

Anders Ynnerman. Photo: LiU

At the same time, work continues with GPT-SW3, the Swedish ai model that has been named after the company Open AI’s GPT-3, the predecessor of Chat GPT. In addition to supplying new hardware to Berzelius, Nvidia assists with software used in the development of the model.

In future, it will be equipped with 175 billion parameters, which is as many as GPT-3 boasts. One of Sweden’s largest banks is said to be already figuring out how GPT-SW3 can be used as a chatbot. Coincidentally, SEB was present during the event, as well as representatives of Volvo Cars, Ericsson, Ikea, Saab, Scania and H&M.

“Still need Swedish version”

However, the question is whether it is necessary to have a Swedish version, given that Chat GPT already has a command of the Swedish language. Anders Ynneman maintains that the work with GPT-SW3 is still relevant.

– I was amazed at how good Chat GPT is in Swedish. I asked it to write a poem about my daughter’s rabbit and it was wonderful. When I asked it to be translated into Swedish, it also turned out very well. But the quality is a little better in English so we still need a Swedish version.

At the moment, discussions are ongoing about who and who should be able to use GPT-SW3. First and foremost, the target group is researchers at universities, start-up companies and industrial companies. But given the impact Open AI’s technology has had on the public, the question arises whether the Swedish version should be a useful tool that everyone can use.

– It is too early to say. It would require a regulatory framework that is not yet in place due to the rapid pace of development. But it is on our agenda to discuss, says Anders Ynneman.

One challenge is the cost. Open AI is now said to be about to start charging for its tool. At the moment it is free and often the traffic is so high that some users cannot access it.

– Training our model is expensive, but that cost is covered by the Wallenberg Foundation. But scaling up the model so that it becomes available to the entire Swedish population is also associated with a cost.

Jensen Huang meets reporters in Stockholm. Photo: Peter Ottsjö

A bit into the question and answer session with the reporters in place, Jensen Huang – as always dressed in black and a leather jacket – asks us to reflect on what he sometimes calls the dawn of the AI ​​era, or the breaking point of artificial intelligence.

– I hope you appreciate this breakthrough. The last decade has been mainly about scientific advances for ai. With Chat GPT, people have started to explore for themselves and fantasize about the future. This is the iPhone moment of the AI ​​era. These models are magical in that they can generalize and solve problems they were never trained for. The number of new startups building applications on top of this technology has exploded. This is only the beginning.