Join us as we return to New York on June 5 to work with executive leaders to explore comprehensive methods for auditing AI models for bias, performance, and ethical compliance in diverse organizations. Find out how you can be present here.
Microsoft is investing more in the development of small language models (SLMs). In his own way To build developer conference, the company announced the general availability of its Phi-3 models and viewed an example of Phi-3-vision. However, following Microsoft's Copilot+ PC news, it is introducing an SLM built specifically for this device's powerful Neural Processing Units (NPUs).
Phi-3-Silica will be embedded in all Copilot+ PCs when they go on sale in June. It is the smallest of all Phi models, with 3.3 billion parameters.
Microsoft claims the first token's latency is 650 tokens per second and consumes about 1.5 watts of power, meaning it won't be a resource suck and frees up the PC's CPU and GPU to do other calculations feed. Furthermore, the token generation reuses the NPU's KV cache and runs on the CPU, producing approximately 27 tokens per second.
A Microsoft spokesperson tells VentureBeat that what sets Phi-Silica apart is “its distinction as Windows' inaugural locally implemented language model. It is optimized for use on the NPU of Copilot+ PCs, bringing lightning-fast local inference to your device. This milestone marks a pivotal moment in bringing advanced AI directly to 3P developers optimized for Windows, to start building incredible 1P and 3P experiences coming to end users this fall, increasing productivity and accessibility within the Windows ecosystem.”
VB event
The AI Impact Tour: the AI audit
Join us as we return to New York on June 5 to engage with top executives and delve into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across organizations. Secure your attendance for this exclusive invitation-only event.
Request an invitation
Phi-Silica is the fifth variant of Microsoft's Phi-3 model, joining Phi-3-mini with 3.8 billion parameters, Phi-3-small with 7 billion parameters, Phi-3-medium with 14 billion parameters and Phi-3 vision with 4.2 billion parameters.