Microsoft “Cortana” – next generation personal assistant for Windows Phone 8.1 detailed

By damaster | Posted September 12, 2013 10 comments

Windows Phone BlueEarlier today, we reported a recent leak of Windows Phone 8.1, codenamed “Blue”, and briefly mentioned an upcoming new feature in the OS codenamed “Cortana”. The feature was described as “an overhaul to the TellMe service that [would] give Siri a run for its money”. Not long after, Mary Jo Foley from ZDNet chimed in on the news and added what she knew about “Cortana”. Below is an excerpt:

Cortana, Microsoft’s assistant technology, will be able to learn and adapt, relying on machine-learning technology and the “Satori” knowledge repository powering Bing.

Cortana will be more than just an app that lets users interact with their phones more naturally using voice commands. Cortana is core to the makeover of the entire “shell” — the core services and experience — of the future versions of Windows Phone, Windows and the Xbox One operating systems…

Mary Jo Foley alludes to Microsoft CEO Steve Ballmer’s strategy memo back in July, quoting him that this Cortana “shell” will be able to “broker information among services to bring them together on our devices in ways that will enable richer and deeper app experiences.” An example given is when a user tell their devices to “print my boarding pass on Southwest” and the system will be able to intelligently perform this action.

What’s interesting is that the codename “Cortana” isn’t the first time it has been revealed. Back in June, a debugging app called “zCortanaApp” had already appeared on leaked screenshots of an early build of Windows Phone 8.1. The Verge also posted a set of screenshots today seemingly from the same debugging app, showing some of the capabilities “Cortana” can do. Check them out below:

Cortana

Note: The screenshots above are from a debugging app only and is not meant to represent the final user interface  of “Cortana”

As seen from the screenshots above, it appears that Cortana is able to learn from a set of rules, such as pulling in weather information, and perform them at predetermined events. It is also able to bring in signals, such as location, Bluetooth, calendar, amongst others, and use them as inputs to the personal assistant.

According to WPCentral, in Windows Phone 8.1, “Cortana” will be able to be invoked by hitting the Search button, replacing the current Bing app. Users will then be able to speak or type their request for “Cortana” to carry its task out. But it doesn’t stop here, as WPCentral notes that “Cortana” will not only be confined to the Search button, but it will be embedded throughout the operating system. For example, given the consent to do so, “Cortana” will be able to automatically scan your text messages and help you add calendar reminders where applicable, all done without the user having to open the “Cortana” app.

This new “Cortana” personal assistant, intended to compete against Apple’s Siri or Google Now, will most likely be based on a combination of technologies from Bing’s knowledge platform, TellMe natural language speech technology, as well as some sort of social graph. In fact, back in July, Bing’s Stefan Weitz indicated that Microsoft will not ship a personal assistant feature until they have “something more revolutionary than evolutionary”. It seems like “Cortana” is that revolutionary feature.

Posted September 12th, 2013 at 10:14 am
  • http://inaimathi.de/blog/ InaiMathi

    Cortana is also the name of Master Chief’s personal assistant in the Halo series. Coincidence? ;)

    • damaster

      On purpose :)

  • http://inaimathi.de/blog/ InaiMathi

    Cortana is also the name of Master Chief’s personal assistant in the Halo series. Coincidence? ;)

    • damaster

      On purpose :)

  • Yuan Taizong

    I really hope that the execution of this feature will be great, Microsoft has always made great innovations, but hit just next to the target with marketing or execution, Microsoft had the ability to create a Siri-like experience since 2003, but they thought that it would be handier if the users wouldn’t here their phone speak, although Microsoft’s Tellme Networks on Windows Phone does work accurate and speak back, it doesn’t function half as good as Apple’s Siri or Google’s Google Now, which is sad, but in functionality is does a better work with the tasks it can do, but if Cortana will be half as good as most people claim ”she’ll” be, we can expect a great reason for people to chose Windows Phone, maybe Cortana will teach people that the number of apps in the app marketplace isn’t as important as the native features of the phone itself (like older versions of Windows Mobile before it).

  • Yuan Taizong

    I really hope that the execution of this feature will be great, Microsoft has always made great innovations, but hit just next to the target with marketing or execution, Microsoft had the ability to create a Siri-like experience since 2003, but they thought that it would be handier if the users wouldn’t here their phone speak, although Microsoft’s Tellme Networks on Windows Phone does work accurate and speak back, it doesn’t function half as good as Apple’s Siri or Google’s Google Now, which is sad, but in functionality is does a better work with the tasks it can do, but if Cortana will be half as good as most people claim ”she’ll” be, we can expect a great reason for people to chose Windows Phone, maybe Cortana will teach people that the number of apps in the app marketplace isn’t as important as the native features of the phone itself (like older versions of Windows Mobile before it).

  • Yuan Taizong

    Usually I show my friends the cool things I can do with my Windows Phone 7.8 concerning voice and they all think it’s cool, but after a few mere minutes (or days) it can become just a simply gimmick, if the Satori engine will actually be the engine used, we can expect it to learn naturally from its users, and transfers the data to the Cloud and become more like ”an actual person” over time, I hope something akin to Clever-Bot while maintaining the the functionality of Microsoft’s Tellme Networks on Windows Phone.

  • Yuan Taizong

    I mean in theory.

  • Yuan Taizong

    Usually I show my friends the cool things I can do with my Windows Phone 7.8 concerning voice and they all think it’s cool, but after a few mere minutes (or days) it can become just a simply gimmick, if the Satori engine will actually be the engine used, we can expect it to learn naturally from its users, and transfers the data to the Cloud and become more like ”an actual person” over time, I hope something akin to Clever-Bot while maintaining the the functionality of Microsoft’s Tellme Networks on Windows Phone.

  • Yuan Taizong

    I mean in theory.