Power Platform AI Builder – Count Things with Object Detection

Microsoft’s new AI Builder capability for the Power Platform includes an Object Detection model. But what does this mean and how can it be used to help organisations?

This article’s objective is to kick start your thinking about the use of AI Builder Object Detection. With AI Builder, the opportunities to cost effectively digitise and accelerate a business are now enormous.

This article includes:

  • An example scenario where AI Builder’s Object Detection model can be used to count objects.
  • Tips on how to easily create a prototype AI Builder Object Detection model.
  • A demonstration of a prototype built with PowerApps and AI Builder.

Read below or consume in video format…

An Object Detection Scenario

My local gym runs group fitness classes. The ones where a bunch of people in a room, jump around following an instructor. On a regular basis a person will enter the room half way through a class and count the number of the people in the room. This number is written in a note book. I can only assume that this is then manually typed into a computer somewhere so that fitness class attendance can be measured.

Wouldn’t it be great to automate what is currently a mundane, manual, human task? To increase productivity and free up employees to carry out more meaningful work.

You might also be thinking… wouldn’t this be difficult and expensive to implement? Why would I even consider automating such a small task?

Actually, counting gym class attendance turns out not to be such a small task and the ROI works out pretty good! We will come back to that. But first, let’s look at the solution to make more sense of it.

AI Builder Object Detection Models

Initially, one can only assume that AI Builder’s Object Detection model is good enough to count the number of people from a photo, but you can’t be sure. Usually, testing AI scenarios takes hours. However, with AI Builder I was able to build an Object Detection model to look for people in a group fitness setting, and test that it works, in under 30 minutes.

This is how I did it:

  1. Use PowerApps to create a new Object Detection model
  2. Collect images using a Google image search and a Chrome add-on called Bulk Image Downloader.
  3. Upload the images to AI Builder and tag the people in each image. This is the most time consuming part but you only need 15 images to get started.
  4. Train and test the model.

The results weren’t exactly perfect but good enough to identify that AI Builder was a good fit for this scenario.

Performance of model trained off Googled images
Quick test results of first Object Detection model.

Object Detection with PowerApps

Imagine if the gym employee could simply take a photo of the fitness class on their phone and the number of people were counted automatically by the AI Builder’s Object Detection model. Imagine that the phone then also automatically updated the database that stores the attendee numbers.

A PowerApp with an AI Builder Object Detection control can achieve this.

Note that I am assuming that the gym’s database is the Common Data Service (CDS) or something that PowerApps or Microsoft Flow can connect to and update.

Power Platform AI Builder
A very simple PowerApp with the AI Builder control (phone landscape mode).

I’m not going to go into the detail on how to build this. If you are familiar with building Canvas PowerApps then adding in the AI Builder Object Detection control is dead easy. There is plenty of content out there covering how to build a PowerApp with an AI Builder control.

Check out these resources:

Object Detection with Microsoft Flow

The PowerApps solution described above is OK. It will save the employee time. However, it still requires the employee to go to each room and take a photo.

Let’s step this solution up a gear. A permanent IP camera in each group fitness room can take photos at regular intervals. The photos can then be processed using Microsoft Flow and AI Builder to count the number of people and update the database.

Power Platform AI Builder
Bypass human data entry using AI Builder.
(see the video for futher context of this diagram)

AI Builder Solution ROI

Usually, an AI solution like this would not be considered due to it’s complexity and cost. Especially for such a simple task. However, I did some maths based on what I know about my gym and the ROI estimates are eye opening.

Roughly $100,000 per year to pay people to count other people, vs a US$84 Flow Plan 1 license. You choose…

The diagram below shows my rough workings. Check out the video for a more thorough explanation.

Power Platform AI Builder
Rough ROI workings

Note: The ROI will only be realised if the people who currently do the counting use their skills in other meaningful ways to drive value for the business. Putting in an automated solution such as this does not remove the employees salary, it only enables the employee to do other things. Always consider the people side of things.

PowerApps & AI Builder Prototype Demo

Key Considerations

When evaluating whether AI Builder is right for you make sure that you take the following into consideration. At this stage, I recommend using AI Builder for prototyping only and testing ideas.

  • AI Builder is currently in preview until October 2019. It is not production ready.
  • AI Builder is only available in the US and EU regions.
  • There are no details on licensing yet (that I am aware of). AI Builder might be included in PowerApps or Flow licenses – or it might not.

Conclusion

I hope that you are inspired by this article to start looking at AI Builder and what it can do for your business. AI Builder combined with the Power Platform really is a game changer and opens up a whole world of opportunities. AI is now for everyone. Be creative, get out there and build some prototypes.

Reference:

Sheild, H. (2019). Power Platform AI Builder – Count Things with Object Detection. Available at: https://dynamicscitizendeveloper.com/2019/07/05/power-platform-ai-builder-count-things-with-object-detection/ [Accessed: 1st April 2020].

Find more great Power Platform content here.

Share this on...

Rate this Post:

Share:

Topics:

AI