Today is the fifth post in the series I started back in March. My overview of 5 chatbots compared Perplexity, ChatGPT, Bing Copilot, Microsoft Copilot, and Gemini. Subsequent blog posts looked at each of these in depth; today’s focuses on Microsoft Copilot.

Getting Information from Your Microsoft Ecosystem

Why do I call it “tricky?” Unlike the others that primarily get their information from the Internet, Microsoft Copilot can also search through most of your Microsoft documents, email, and Teams conversations. This is because it has access to your company’s Microsoft Tenant. (Mind you, the cost for licenses is $30 per user per month.)

For example, I asked about recent interactions with Jackie, a CGNET team member. Here is the output from Copilot, which included footnotes linking directly to the items:

1. Jackie authored a document titled “Use Case 6 – prompt writing,” which was last modified about 15 minutes ago. The document discusses how to phrase requests to Copilot effectively.
2. Jackie also authored “Funny Example – Out of Office” and “Funny Copilot Example” documents, both modified within the last half hour, providing humorous out-of-office message examples.
3. A few hours ago, Jackie sent an email to Erik, mentioning that you and she would discuss scheduling and get back to him.
4. Yesterday afternoon, Jackie responded to an email thread you were included in regarding a Visa transaction, indicating she could provide more exact invoice information.
5. Last Thursday, Jackie mentioned in a Teams message that she had completed the web analytics.
6. Additionally, there was a Teams conversation last Tuesday where Jackie inquired about the recipient of a document related to Africa Re-Insurance.

I find this impressive, yet to be honest, I was already aware of all this and didn’t particularly need to be reminded. So, while impressive, it didn’t significantly add to my productivity. When I asked for more complex requests involving a particular customer over a long time period, the results were inaccurate.

Writing a Document

I asked it to write a blog article on the advantages and disadvantages of Microsoft GPT. The output was useless, unlike what I got from Perplexity and ChatGPT.

Overall Performance

A recent Wall Street Journal article titled “The Great AI Chatbot Challenge: ChatGPT vs. Gemini vs. Copilot vs. Perplexity vs. Claude” compared the performance of five leading AI chatbots: ChatGPT, Microsoft’s Copilot, Google’s Gemini, Perplexity, and Claude. The good news was that  Copilot came in first place for creative writing and middle of the pack for summarization, current affairs, and response time.

The bad news was that Copilot came in last place in overall performance. It also ranked last in terms of category-specific results for Health, Finance, Cooking, Work-Related Writing, and Coding.

Proofreading

Copilot performed similarly to the other chatbots I have used. It didn’t make a huge number of recommendations for structural improvement, but it got the job done in relation to grammar, spelling, and punctuation.

Create an Image

Like several of the others, Copilot failed my Alfa Romeo test spectacularly (“create an image of a 1969 Alfa Romeo Spyder Veloce 1750”). The image below is not at all related to what the sports car looks like, and what is with the jet planes and fire truck? However, I do think the car is cool looking.

But I did like its Easter Bunny rendering.

What’s to Like

• Its uniqueness is intriguing, and for users who are not on top of their Teams, email, and documents, this could be a helpful way to find things or get back up to speed after a vacation.
• It is relatively easy to use.
• It can be accessed from within apps like Teams, Outlook, Word, and Excel.
• Its proofreading is acceptable.

What’s Not to Like

• The cost of $30 per user per month is high, considering that most of it is free, and all you are adding is access to your own Microsoft files, email, conversations, etc.
• Launching within an app, like within Outlook compose, is confusing, and when I did use it, the language sounded stilted, like hyperbole, or somewhat condescending.  The learning curve is high.
• Not useful for creating content.

Overall Grade

In summary, I’d give Microsoft Copilot a “C-”.  But if I were to concentrate on its ability to deliver content from within a user’s  file, I would give it a B-.