AI Image Processing at Home (Hot Dog/Not Hot Dog)

Home automation is wonderful.  Up until recently, I used things like motion sensors and contact sensors to trigger various things.  While these devices still have their place, they are "dumb."  For example, I could have a motion sensor in my backyard that triggers lights, but any motion from squirrels or my dog unnecessarily triggers.

Recently, a youtube channel I came across detailed how to easily set something up that can determine objects in your camera and act accordingly.

The meat of this uses 3 pieces of software:

- Blue Iris https://blueirissoftware.com/

This is the only piece of software that requires cash money.  Also, it runs in Windows.  These might be deal breakers for some (it was almost for me), but after trying it out, I decided to go with it.  It turned out to be worth the price (around $60) and it's much easier to work with than ZoneMinder.  There are also efforts to make this work in Docker (more on that later).

- Deep Stack AI Server https://deepstack.cc/

This software is free and available as a Docker container.  https://hub.docker.com/r/deepquestai/deepstack

- AI Tool (Windows - https://github.com/gentlepumpkin/bi-aidetection) or node-deepstackai-trigger (Docker - https://github.com/danecreekphotography/node-deepstackai-trigger)

The AI Tool runs on windows while node-deepstackai-triggers runs in a Docker container.  This is the important piece of the puzzle and sits in-between Deep Stack AI Server and Blue Iris.  The basic flow is:

Blue Iris captures a still image into a folder – >AI Tool/node-deepstackai-trigger sees the image > image is sent to Deep Stack AI > results are returned to AI Tool/node-deepstackai-trigger > actions are taken based on results

Actions can be any type of home automation you have going on.  With node-deepstack-ai, you can send out a web request or a MQTT message.  In a particular set up, I set up my front door car to recognize people.  After everything happens, my front door records a high resolution video and makes a chime/announcment on select Alexa devices in my home.  Previously, I would get false alarms if any motion was detected on the camera.  Now, I have a "smarter" camera that gives me finer control on my home automation.  This is all very exciting stuff for home automation.  You can run all this processing locally and not depend on pay/Internet services.

Below is an example of another automation I have with a camera pointed at my pool.  When it notices a person, I trigger Blue Iris to record a high resolution video.  Blue Iris can also trigger other things as well so you can chain these things together (or, just add another trigger Uri to the array).

Example snippet of configuration for my pool camera:

    {
      "name": "poolsd",
      "watchPattern": "/aiinput/poolsd*.jpg",
      "cooldownTime": 5,
      "enabled": true,
      "threshold": {
        "minimum": 0,
        "maximum": 100
      },
      "handlers": {
        "webRequest": {
          "triggerUris": ["http://192.168.1.28:81/admin?camera=poolhd&trigger&user=user&pw=pass"]
        }
      },

To set this all up, I recommend watching The Hook Up youtube channel video on how to set this all up.  The wiki for node-deepstackai-trigger also has good detailed info:  https://github.com/danecreekphotography/node-deepstackai-trigger/wiki

Hopefully, we can have something set up to train the AI with our own personal faces.  We can get really creative with home automation then.  I would like my doors to open when my camera sees my ugly mug.

# Windows?

I prefer to use the node-deepstackai-trigger docker container since it is easy to keep running and configure.  It also pairs well in a docker-compose set up with Deep Stack AI server.  The final piece of the puzzle is getting Blue Iris running in Docker.  jshridha has created a container, but ther is a lot of work to be done.  It isn't stable for my tastes, but we are working to fix things.  I have already submitted some PRs to update things and I hope we can eventually get this working properly.   https://github.com/jshridha/docker-blueiris

Funny last note:  As I was writing this on my patio, I noticed some log entries from node-deepstackai-trigger:

2020-09-01T15:52:07-07:00 [Trigger patiodoorsd] /aiinput/patiodoorsd.20200901_155151275.jpg: Detected object laptop is not in the watch objects list [person, dog]
2020-09-01T15:52:07-07:00 [Trigger patiodoorsd] /aiinput/patiodoorsd.20200901_155151275.jpg: Not triggered by laptop (95.88411500000001)

lol - that's pretty cool.