NXP Semiconductors EdgeReady smart HMI solution

The NXP EdgeReady Smart Human Machine Interface (SMHMI) solution leverages the i.MX RT117H crossover MCU to allow developers to quickly and easily enable multi-modal, intelligent, hands-free capabilities including machine learning (ML) vision for face and gesture recognition, far-field voice control and 2D graphical user interface (GUI) in their products. These functions can be mixed and matched to simplify overall system design using just this single NXP high-performance crossover MCU.

Key features

• Based on i.MX RT1176, with NXP’s recognition, AFE and automatic speech recognition SW
• Enable Smart HMI with ML/AI Vision, far-field Voice and GUI locally on a single MCU
• Software framework to support flexible design customization of vision and voice functions
• Software package with source code and libraries, solution kit with out-of-box demos

Additional features

The i.MX RT117H MCU is licensed to run NXP’s face/ gesture recognition library, audio front end library and ML speech engine library, which includes (see software block diagram section for more information):

• Software framework
• Camera drivers, image capture and pre-processing
• Face/gesture detection, alignment and recognition (with quantified results and confidence measure)
• Microphones driver and audio capture
• MQS driver and audio playback
• Voiceseeker audio front end (AFE) for acoustic echo cancellation, noise reduction, beamforming and barge-in
• Conversa software for full duplex speaker phone
• VIT or 3rd party automatic speech recognition (wake words and commands)
• Display drivers, 2D graphics accelerator, support LVGL
• Built-in security, bootloader and application validation
• All necessary wireless drivers, including Bluetooth™ low energy and Wi-Fi
• USB mass storage device updates
• Support secure firmware update over a wireless connection.
• Factory automation scripts
• Supported by MCUXpresso SDK, IDE, Config tools and GUI guider

Applications

• Smart appliances
• Smart home
• Smart building
• Smart industrial

Avaliable tool

The SLN-TLHMI-IOT development kit implements NXP’s i.MX RT117H based smart HMI solution. The kit supports multiple out of box applications (which can be selected by pressing and holding SW1, SW2, or SW3 on the back of the kit when booting the kit) including coffee machine, elevator and smart home panel (the former two are pre-flashed with the kit while the later one could be downloaded from the GitHub). These example applications are built to showcase the complex machine vision, voice and advanced graphical user interface capabilities the kit supports. They also provide a base software platform for rapid integration of unique customer applications and requirements.