Final yr, Microsoft revealed Copilot+ PCs that includes a devoted Neural Processing Unit (NPU) able to over 40 TOPS or extra. These Copilot+ PCs got here with Phi-Silica, an on-device SLM to convey language intelligence capabilities to Microsoft’s personal first-party apps and apps from different Third-party builders.

As we speak, Microsoft revealed Mu, a brand new on-device small language mannequin constructed into Home windows 11. Microsoft’s objective was to create an AI-powered agent inside the Settings app that may perceive a consumer’s pure language queries and combine it into the present search field for a clean consumer expertise. Therefore, Microsoft is utilizing the brand new Mu mannequin to energy the brand new agent function within the Settings app, which was made out there to Home windows Insiders not too long ago within the Dev Channel with Copilot+ PCs.

Like Phi-Silica, Mu is designed to function effectively on NPUs, delivering over 100 tokens per second whereas operating regionally. Within the official weblog publish, Microsoft defined how they designed and educated the Mu language mannequin. Here is an outline of the Mu language mannequin:

• Mu is a 330M encoder-decoder language mannequin.
• The encoder-decoder strategy achieved about 47% decrease first-token latency and 4.7× larger decoding pace in comparison with a decoder-only mannequin of comparable measurement.
• Mu makes use of weight sharing in sure elements to scale back the entire parameter rely.
• Microsoft educated Mu utilizing NVIDIA A100 GPUs on Azure Machine Studying.
• Mu is almost comparable in efficiency to a equally fine-tuned Phi-3.5-mini, regardless of being one-tenth of the dimensions.
• The mannequin is best fitted to multi-word queries. So for brief or partial-word inputs, the Settings app will proceed to floor lexical and semantic search ends in the search field.

customers can obtain Home windows 11 Construct 26120.3964 (KB5058496) or larger to take a look at this new AI-powered agent within the Home windows Settings app.