Renesas Electronics and Syntiant have jointly developed a voice-controlled multimodal AI solution

     Renesas Electronics and Syntiant, a deep learning chip technology provider driving low-power intelligent voice and sensor processing in edge devices, today announced the joint development of a voice-controlled multimodal AI solution. Enable low-power, contactless image processing in visual AI-based iot and edge systems such as self-service cash registers, security cameras, and video conferencing systems, as well as in smart appliances such as vacuum cleaners.

     The new solution combines the Renesas RZ/V Series Vision AI microprocessor (MPU) with a low-power, multimodal, multifunctional Syntiant NDP120 Neural Decision Processor to deliver advanced speech and image processing capabilities. The solution is normally on and can be quickly activated by voice from standby mode to perform visual tasks such as object recognition and facial recognition, as well as other critical functions of the system such as security cameras. For example, when a user-defined voice prompt activates and runs the system, the visual AI will recognize and track the operator's behavior and control the operation or issue a warning when suspicious behavior is detected.

     The multimodal architecture makes it easier to create contactless user experiences for visual AI-based systems. Because software can be developed independently of visual AI capabilities, speech recognition with dedicated, energy-efficient chips reduces standby power consumption and accelerates system development.

     Keito Nikita, senior Vice President of Renesas Electronics and head of the SoC Division of the Internet of Things and Infrastructure Business Unit, said: "We expect to see increased demand for multimodal systems with multiple input streams, including image and voice, as an effective way to improve ease of use and security. The close collaboration between Renesas, with its advanced low-power image AI technology, and Syntiant, a leading supplier of voice AI technology, will accelerate the adoption of low-power, ultra-small intelligent voice AI technology in embedded systems and bring new integrated solutions to customers around the world."

     Kurt Busch, CEO of Syntiant, said: "Voice-based user interfaces give customers a new user experience, bringing the next generation of innovative ideas from concept to reality. Our deep learning NDPS have shipped more than 15 million units worldwide, enabling normally open voice capabilities for a wide range of consumer and industrial iot applications. Our partnership with Renesas creates powerful, low-power voice and image solutions that we believe will accelerate the adoption of our customers across a wide range of devices and cases around the world."

     The Renesas RZ/V series MPU for vision AI integrates Renesas' unique DRP-AI (Dynamically Configurable Processor-AI) accelerator and combines high-precision AI reasoning with industry-leading energy efficiency. Superior energy efficiency eliminates the need for cooling measures such as heat sinks or cooling fans, thereby reducing bill of materials (BOM) costs and enabling visual AI integration into a wide range of embedded applications.

     The Syntiant NDP120 chip integrates sophisticated AI capabilities for a wide range of high-precision hands-free speech applications, including speaker recognition, keyword detection, multi-wakeword and local command recognition. Packaged with the Syntiant Core 2 neural network inference engine, the NDP120 can also run multiple applications simultaneously and reduce power consumption to 1mW of battery power.

     The new voice-controlled multimodal AI solution, part of Renesas Electronics' "Portfolio of Success," features multiple compatible devices from Renesas' broader product portfolio to build higher-level prototyping platforms for customers to accelerate time-to-market and reduce risk. Renesas' Portfolio for Success 'includes a superior portfolio of analog, power and embedded processing products that help customers accelerate design and reduce time to market.