In short: Small language fashions are typically extra compact and environment friendly than LLMs, as they’re designed to run on native {hardware} or edge units. Microsoft is now bringing yet one more SLM to Home windows 11, as customers apparently want a couple of AI-powered hints to assist them discover particular OS settings and customise their PC expertise.
Microsoft not too long ago introduced Mu, a brand new small language mannequin designed to combine with the Home windows 11 UI expertise. Mu will work alongside Phi Silica – the language mannequin beforehand launched on Copilot+ PCs – to energy the AI agent function in Home windows Settings that the corporate is at the moment growing.
Like Phi Silica, Mu will run totally on the system’s neural processing unit, delivering responses at over 100 tokens per second, which ought to present a easy person expertise in Settings.
Microsoft describes Mu as an environment friendly 330-million-parameter mannequin optimized for small-scale deployments. It’s primarily based on a transformer encoder-decoder structure, the place the encoder converts person enter right into a fixed-length latent illustration, and the decoder generates output tokens from that illustration.
Microsoft designed Mu as a extremely optimized “sibling” to Phi Silica, utilizing a number of strategies to cut back the parameter rely and enhance effectivity. The SLM was educated utilizing an unspecified variety of A100 GPUs on the Azure Machine Studying platform, progressing via a number of phases and making use of strategies that Redmond developed throughout work on the Phi fashions.
After fine-tuning the mannequin, the builders discovered that Mu delivers efficiency practically corresponding to Phi-3.5-mini, regardless of being simply one-tenth its measurement. Moreover, Microsoft collaborated with silicon companions Intel, AMD, and Qualcomm to make sure that Mu is absolutely optimized for operating on native NPU-powered machines.
What was the purpose of all these optimization efforts? In accordance with Redmond, Mu will energy the brand new “agentic” expertise within the upcoming AI-enhanced Settings app. The AI and cloud large goals to offer a brand new “chat” function that lets Home windows customers ask settings-related questions in pure language and obtain efficient solutions, even after they barely know what they’re in search of.
Agentic AI within the Settings app is meant to enhance Home windows’ ease of use. Microsoft acknowledged that coaching an SLM able to dealing with (and modifying) a whole lot of system settings was a problem, and the corporate says it welcomes person suggestions on the brand new function.
Personally, I by no means skilled any usability or discoverability points with Settings throughout eight years of utilizing Home windows 10, but Microsoft stays bullish about remodeling Home windows into an “AI-first” platform.
