The AI-powered smartphone and what it means for developers

• The Qualcomm Snapdragon 888 chip will let handsets take high-resolution multimedia faster and perform AI-related tasks more efficiently
• As more AI-enabled handsets hit the market, new opportunities are emerging for app developers for cutting-edge customer experiences

The unveiling of Qualcomm’s new flagship chip — the Snapdragon 888 — this week for Android smartphones underlines just how big a role artificial intelligence (AI) capabilities will play in upcoming smartphones.

Like how the recent iPhone 12 launch unpacked the tiny transistors and the AI advantages of Apple’s new A14 Bionic chip – promising nearly 30% uptick in CPU, GPU, and AI-powered sensors performances – the Qualcomm Snapdragon 888 is the chipmaker’s first 5-nanometer chip.

The smaller transistors on these 5nm chips mean that performance gains are boosted up to 35% higher than previous generations, and the 888 also has its 5G modem built into it rather than a separate component in previous chips – which means that connectivity is also being enhanced alongside the processing and graphical capabilities.

The combination of 5G coverage and AI-powered sensor input should be no slouch for the next-generation of topline Android devices, but AI will not just be enhancing the camera and other sensors.

AI-enhanced smartphones

“Qualcomm’s addition of a large AI accelerator and TinyML to its next-generation 5G mobile platform is a game-changer for 5G mobile devices,” said Lian Jye Su, the principal analyst at tech market advisory firm, ABI Research.

“As more users switch to 5G mobile devices in 2021, embedded AI will be a critical success factor for enhanced user experience.”

Qualcomm says the new Snapdragon features a completely re-engineered AI Engine, the Hexagon 780. The chipmaker believes this redesign will help software developers working on Snapdragon 888-powered devices to make improvements in a number of areas including live-motion tracking of objects (to assist autofocus in video and photography, for instance); automatically adjusting audio to account for the device’s surrounding ambient environment; improving augmented reality filters in AR-capable apps. The AI computational performance is said to be so remarkable that Qualcomm says that the Snapdragon 888’s AI processing can erase a recorded character from a video scene and insert someone else instead – in real-time.

“With the AI capabilities of Snapdragon 888, OEMs can pack more and more AI enhancements into various applications in their mobile device, including beautification, edge detection, depth sensing and low-light noise reduction in the camera, more secure facial and fingerprint unlock in biometric identification, more fluid gaming performance and smarter app recommendations and battery optimization,” noted ABI’s Lian.

According to Gartner, “Artificial intelligence (AI) features will become a critical product differentiator for smartphone manufacturers that will help them to acquire new customers while retaining current users. As the smartphone market shifts from selling technology products to delivering compelling and personalized experiences, AI solutions running on the smartphone will become an essential part of vendor roadmaps over the next two years.”

For app developers, AI-enabled smartphones offer new opportunities, including:

• Emotion recognition: Emotion sensing systems and affective computing allow smartphones to detect, analyze, process, and respond to people’s emotional states and moods. The proliferation of virtual personal assistants and other AI-based technology for conversational systems is driving the need to add emotional intelligence for better context and an enhanced service experience. Car manufacturers, for example, can use a smartphone’s front camera to understand a driver’s physical condition or gauge fatigue levels to increase safety.
• Natural-language understanding: Continuous training and deep learning on smartphones will improve the accuracy of speech recognition, while better understanding the user’s specific intentions. For instance, when a user says “the weather is cold,” depending on the context, his or her real intention could be “please order a jacket online” or “please turn up the heat.” As an example, natural-language understanding could be used as a near real-time voice translator on smartphones when traveling abroad.
• Personal profiling: Smartphones are able to collect data for behavioral and personal profiling. Users can receive protection and assistance dynamically, depending on the activity that is being carried out and the environments they are in (e.g., home, vehicle, office, or leisure activities). Service providers such as insurance companies can now focus on users, rather than the assets. For example, they will be able to adjust the car insurance rate based on driving behavior.
• AR and AI Vision: With the release of iOS 11, Apple included an ARKit feature that provides new tools to developers to make adding AR to apps easier. Similarly, Google announced its ARCore AR developer tool for Android and plans to enable AR on about 100 million Android devices by the end of next year. Google expects almost every new Android phone will be AR-ready out of the box next year. One example of how AR can be used is in apps that help to collect user data and detect illnesses such as skin cancer or pancreatic cancer.