MediaTek Dimensity has changed in the era of generative AI mobile phones!Taking the initiative to lead industry changes

Recently, MediaTek held the first Dimensity Developer Conference 2024 (MDDC 2024), with the theme of “AI for Everything”, and led the mobile ecological innovation based on generative AI through multiple achievements.

At this conference, MediaTek and its partners jointly launched the “Dimensity AI Pioneer Plan”, released the “Generative AI Mobile Phone Industry White Paper”, announced the flagship 5G generative AI mobile chip Dimensity 9300+, and also demonstrated the generative AI mobile chip Dimensity 9300+. AI terminal-side deployment solution “Dimensity AI Development Kit”, Starspeed Engine game ecosystem, and multi-scenario terminal-side AI applications.

The Dimensity AI Pioneer Program is led by MediaTek and jointlyAlibaba Cloud, Baichuan Intelligence, Transsion, 01Wanyuan, OPPO, Honor, vivo, Xiaomi, etc.Numerous cloud services, large models, and terminal manufacturers are available to developers around the world.

This plan will integrate the resources of MediaTek and industry ecological partners to provide rich development resources, technical support and business opportunities for developers exploring device-side generative AI applications and scenario innovation.

At present, a large number of application development manufacturers and developers have joined the Dimensity AI Pioneer Program, and some of them took the stage to help out.Little Red Book, Meitu, Kugou Music, Huya Live, Douyin, Tencent AI Lab, Soul, National Karaoke, RWKV, COCOS, etc.

Many applications simultaneously demonstrated their new AI experiences at the venue, which we will share with you in subsequent sessions.

MediaTek joins hands with authoritative market research agency CounterpointAlibaba Cloud Tongyi Qianwen, Baichuan Model, Huya, Kugou Music, Zero One Thing, OPPO, Soul, Tencent AI Lab, Tencent Hunyuan, vivojointly released the “Generative AI Mobile Phone Industry White Paper”, jointly defining the concept and typical characteristics of “Generative AI Mobile Phones”.

According to the definition in this white paper,Generative AI mobile phones can use large-scale, pre-trained generative AI models to achieve multi-modal content generation, situational awareness, and have increasingly enhanced human-like capabilities.

The white paper also elaborates on the trend of deep integration of generative AI and smartphones, and deeply discusses the AI ​​strategies of chip manufacturers, mobile phone manufacturers, large model manufacturers, and developers in the generative AI mobile phone ecosystem, as well as the software and hardware technology of generative AI mobile phones. Panoramic view, and looks forward to the development trend of generative AI mobile phones.

As an underlying hardware supplier, MediaTek can provide powerful hardware support for generative AI phones, includingAdvanced mobile computing platform, integrated or independent NPU unit, large-capacity and high-bandwidth memory, stable and high-speed network connection, hardware-level and system-level security defense.

The newly released Dimensity 9300+ is a typical representative.

Dimensity 9300+ continues to use TSMC’s third-generation 4nm advanced process and full-core CPU architecture, includingFour X4 large cores, with a maximum frequency of 3.4 GHz, and four A720 large cores, with a main frequency of 2.0GHzthere is also a 12-core Immortalis-G720 GPU, 18MB cache, and supports LPDDR5T 9600Mbps high-frequency memory.

Dimensity 9300+ has a built-in seventh-generation AI engine APU 790, which has powerful generative AI computing power.It is the first to support AI speculative decoding acceleration technology on the device side, and the generation speed can be increased by 120%.

Supports Dimensity AI LoRA Fusion 2.0 technology, that is, dual LoRA fusion, which increases generation efficiency by 100% and saves memory space by 50%, making the generative AI experience more efficient and personalized.

Supports cutting-edge and mainstream generative AI large models, covering text, images, music and other AI multi-modal scenarios, including Alibaba Cloud Tongyi Qianwen, Baichuan, Baidu Wenxinyiyan, Google Gemini Nano, Zero One Thousand Things, Meta Llama 2/3, ERNIE-3.5-SE.

especiallyIt is the first in the industry to run the Llama 2 7 billion parameter large model at high speed on the end side at 22 tokens per second.

In addition, it also supports the AI ​​framework ExecuTorch, which can accelerate the development of end-side generative AI applications.

At the same time, MediaTek’s StarSpeed ​​engine has been fully upgraded to provide game developers withMAGT adaptive technology software development kit and hardware ray tracing technology software two major development kitsthrough key technologies such as precise performance management, light tracing effect optimization, and network quality monitoring, it improves the fidelity and fluency of the picture, the responsiveness of touch and network, and the durability of the device's battery life.

Starspeed Engine can bridge the gap between hardware platforms and games, build a bridge for communication with each other, and achieve real-time on-demand performance response.

After MAGT technology is turned on, game loading speed can be accelerated by 11%, allowing for longer-lasting full frames.

Combined with precise system monitoring and in-depth joint adjustments with head games, it can ensure full game frames while further reducing power consumption by 10%.

In terms of network quality monitoring technology, through cooperation with Tencent Cloud, the connection quality of the game network can be monitored in real time, allowing the game to use Wi-Fi and cellular network concurrency technology more efficiently, making online games more stable.

Compared with the traditional Wi-Fi/cellular network dual-network concurrency technology, it can predict weak networks 100ms in advance with an accuracy of over 90%. When turned on in a Wi-Fi weak network environment, it can save 10% of full frame power consumption. 25% data usage.

In addition, MediaTek is also actively working with industry partners such as Arm, Google, Khronos, and UE Unreal Engine to explore cutting-edge technologies to expand the Dimensity gaming ecosystem, especially to accelerate the implementation of hardware light-tracing games.

Another highlight of this conference is the Tianji AI development kit, which includes four major modules, namelyFast and efficient GenAI best practices, GenAI Model Hub covering the world's mainstream large models, GenAI optimization technology that efficiently improves performance, and Neuron Studio one-stop visual development environment.

MediaTek said that the Dimensity AI development kit can provide developers with a “fast, comprehensive, powerful and easy” professional development experience and accelerate the entire development process of terminal-generated AI applications.

Among them, GenAI best practices accelerate the terminal deployment of large models through model quantification, model compilation and model inference technologies, which can shorten the workload that originally took several weeks to just one day.

GenAI Model Hub adapts to the industry's leading and mainstream large models and provides developers with rich large model resources needed to create generative AI applications.

GenAI optimization technology only includes speculative decoding acceleration, LoRA Fusion, etc.

Neuron Studio integrated development environment can provide a one-stop visual development environment, jumping out of the traditional code development environment, making development become “what you see is what you get”.

At present, the Dimensity AI developer kit has covered many smart terminal devices such as smartphones, smart cars, the Internet of Things, and PC computers, enabling the development of generative AI applications in all scenarios.

Currently, MediaTek has joined hands withAlibaba Cloud Tongyi Qianwen, Cocos, King of Glory, Huya Live, Kugou Music, Meitu, National Karaoke, RWKV, Soul, Tencent AI Lab, Xiaohongshu, etc.Dozens of ecological partners create generative AI applications with completely different experiences.

This is the first time MediaTek has held a developer conference. It can be called an important turning point in the history of MediaTek and a milestone for the entire industry, leading the transformation of the generative AI mobile ecosystem.

It marks that the role of MediaTek Dimensity platform has changed from a technology leader in the past to an innovation leader in the mobile industry. It is no longer limited to its own development, but has expanded its vision and vision to the advancement of the industry.

Since the birth of the Dimensity brand, MediaTek has won widespread favor from terminal manufacturers, application developers, and consumers through continuous technology investment and open cooperation, and has also formed a vigorous mobile ecosystem.

Today, generative AI is quietly changing various computing industries, especially the mobile industry, and is bound to subvert everyone’s life, work, and entertainment experiences.

Under this wave, MediaTek provides not only basic underlying hardware, but also one-stop development tools and optimization technologies for the entire chain. It also shoulders the responsibility of leadership and hub, uniting many industry partners to jointly define New concepts and new technologies jointly promote application and ecological development.

Next, we will experience the mobile terminal-side generative AI applications optimized in cooperation with MediaTek platform one by one.

Let’s watch a live video first:

https://www.douyin.com/video/7366396666703826210

Tik TokIt brings two applications, one isAI real-time special effectsbased on MediaTek APU, combined with algorithm polishing and engineering optimization based on hardware computing power, to bring a better experience in terms of effect, fluency, and power consumption.

two isEnd-side SDXL-Lightning high-definition Vincent graphicsdeveloped by Douyin’s intelligent creation team.

It uses progressive adversarial distillation technology to generate high-quality and high-resolution images in 4 or even 2 steps, and can generate images for timeout-sensitive applications in 1 step.

According to internal testing, the speed of 1024*1024 resolution on the mobile phone is about 2 seconds for 1 step and about 5 seconds for 2 steps.

Huya Livethe first to realizeAI real-time face changingthe AI ​​algorithm based on Tushengtu can enrich the anchor's live broadcast and bring enhanced on-end image quality to users.

It supports twice the super resolution of 450P resolution, turning it into 900p; it also supports 720P original painting enhancement, which can remove the noise of the original painting and enhance the texture details of the picture.

kugou musicsupports lyrics generation, quick image matching, manual image generation and other gameplay methods.

National karaoke,The first device-side model song generation SVC model to be implemented on Dimensity 9300while ensuring high performance, it also improves the reasoning speed, can easily restore beautiful singing voices, sing songs of powerful singers with one click, and can also generate chorus works for you and the singer, which can be called the savior of the tone-deaf.

Soul APPfocusing on AI emotional companionship, is the first to introduce LoRA technology, a single model supports multiple image switching, no need to connect to the Internet, and privacy is safe.

Supports LLM+SD mixed switching, LLM can support multiple rounds of dialogue with only one configuration.

Alibaba Cloud Tongyi Qwen Al Agent (intelligent agent)known as the super AI assistant around you.

Based on the Tongyi large model, Qwen Al Agent can perceive the environment, make decisions and execute instructions, and can think and use tools to complete goals.

Through the Tongyi Qianwen APP (or the application connected to the Tongyi large model), the local model can complete the entire closed loop of task requirements offline -> route planning and design -> task confirmation -> command issuance of corresponding services and execution of tasks. The interactive experience, processing efficiency, accuracy, etc. far exceed traditional interaction methods.

Volcano Engine’s AI Creation SDK Kityou can cut it into a piece, and you can create it at any time whether on your mobile phone, tablet or car.

Among them, EffectOne SDK provides mobile applications and OEM customers with complete end-side picture and video creation technology solutions, including shooting, editing, special effects (2D/3D/AIGC), video templates, etc.

The Moment SDK is a car-side creation suite that provides simpler car-side creation capabilities, including automatic film production capabilities such as one-click filming, cutting of the same style, and highlight filming. It also includes more flexible basic editing capabilities, combined with AIGC, massive Special effects provide a rich selection of templates.

“Nishuihan” mobile game and “Dark Zone Breakout”The first to implement mobile hardware ray tracing technology and MAGT game adaptive control technology.

“Honor of Kings” overseas version player's personal game coachthe first to realize the implementation of game teaching audio companion AI on the Dimensity mobile phone terminal.

It builds a bridge between AI and players through TTS text-to-speech conversion, teaching players MOBA gameplay/scene prediction/outfitting suggestions, combined with accurate voice generation, to create unique timbres and character backgrounds, making it easier for players to immerse themselves.

This includes guidance on hero outfitting at the start of the game, reminders of hero disappearance during the battle, enemy outfit reminders, team battle preparations, etc., kill encouragement after eliminating the opponent's hero, and victory rate broadcast.