Uncategorized

Comparison of Wyzev3 Sensor vs RPi IMX327

Leave a comment

One of the best low-light sensors for the raspberry pi is the Sony IMX327 (available here: https://www.inno-maker.com/product/mipi-cam-327/ for around $90). The wyzev3 offers similar starlight performance for $20 – a fraction of the cost. But how do they compare?

IMX327
Wyzev3

As can be seen, both images capture all the overall scene well. The white balance of the IMX327 was not adjusted leaving the overall scene seeming a bit “warmer” than the wyzev3.

As for detail a few important differences pop out:

  • The wyze seems to saturate around light sources. Especially pronounced near the streetlamp and the car’s break lights.
  • The wyze has a slightly wider field of view
  • The wyzev3 doesn’t seem to capture some detail, such as the lettering on the stop sign, as well as the IMX327 – this is especially apparent when zooming in on the full size image.
  • The wyze seems better at keeping details crisp in “busier” sections of the scene – examples include the ground around the forest on the right of the image
  • The wyze seems to suffer more motion blur. I believe that the wyze acheives some of its performance by combining multiple frames. This improves low light performance but smears together details.

While the IMX327 seems to have potentially better quality, cost almost hands the victory to the wyzev3. By the time you add a fully-equipped pi4 to the IMX327 your cost is close to $160. Cutting that in half you could point four times as many wyze cameras at the problem and have vastly better combined video coverage. Add to this the wyzev3 includes IR-LEDs (near and far), microphones, a speaker, and can be set up by a non-PhD.

The only caveat here is that wyze3 is a “closed” product. Unless open source firmwares can be loaded on it, it will never be as secure as the pi solution – users of wyzev3 are at the mercy of wyze to protect their data. To this end there is some hope that wyze will release an RTSP version of their firmware, as they did for their v2 product.

HomeAssistant

HA MQTT Auto Discovery

Leave a comment

As described on their official MQTT Auto Discovery page, Home Assistant allows one to create sensors on the fly. This is particularly important if you are, say, trying to add some non-trivial number of devices that speak MQTT.

I saw a lot of posts about people struggling to get this working. I too had a few issues and felt it worth stating my take on it. Spoiler alert: HA MQTT auto discovery works perfectly, so long as you get over some of the setup nuances. I thought I’d document these for posterity:

  • If you added MQTT support through the GUI, auto discovery is set to false by default. There doesn’t appear to be any way to fix this through the GUI. If you then add MQTT to your configuration yaml, the GUI config overrides it! The only way to overcome this is to delete MQTT from the GUI and then add it in the configuration yaml, restart HA, etc..
  • To publish a value there is a two step process
    1. Publish to the configuration topic to define the sensor. The concept of how this works is well documented on the main link given above – however it is important to note that you do NOT have to provide the configuration for all values in a entity with multiple values. This is actually a nice feature – and works so long as you always publish the config before sending the value. It was not clear to me (and i didnt investigate) if there was any good way to minimize the need to broadcast configurations; e.g. how do you know HA saw your config before you send state? To be safe i just publish the config and state one after another everytime. This doesnt seem optimal but it works.
    2. Publish the state via the state topic passed in during the config publish. The only thing to note here is that, unfortunately, it does not appear that values get rooted under the objectid you provide. Insted they are placed under a sensor with the name you provide – seemingly HA completely ignores the objectid…

Otherwise this feature works very well. Some tips I found for debugging this:

  • Enabling logging on mqtt works well. To do this add to configuration.yaml:
logger:
  default: warning
  logs:
    homeassistant.components.mqtt: debug

You can then tail (if under docker) config/home-assistant.log

  • Using the “Publish a packet” or “Listen to a topic” pages under MQTT->configure (in the HA configuration->integrations page) is good for eliminating any client broker issues you might see
  • Additionally, if you are using eclipse-mosquitto as your mqtt server, you can directly view (HA aside) publishes. In docker it would be:
docker exec -it <mosquittodockerid> /bin/sh
mosquitto_sub -u user -P pass -h localhost -t topic

Where “topic” above is whatever you are publishing, e.g. “homeassistant/sensor/grinch/state”

Lastly, in looking for a good c++ mqtt client, I tried paho and mosquitto. I found mosquitto worked best – both for the server (which i run in a docker) and c++ client. My decision was based on this simple requirement: the library should build (paho failed), work with c++11 (other, albeit cool-looking, c++ libraries required c++14), and be easy. Mosquitto fit the bill perfectly. I am sure – perhaps – there could be other libraries that might work as well – i just never found them.

Containers, HomeAssistant

InfluxDB Queries on HomeAssistant Data

Leave a comment

I recently added in influx db + grafana + mariadb to one of my HA instances. I was surprised at how easy it was to get HA happy. Essentially nothing more than running the docker instances for each of these components and adding minimal yaml to HA.

When i went to query the data there were a few surprises that I thought I’d note. First, as noted in https://community.home-assistant.io/t/missing-sensors-in-influxdb/144948, HA puts sensors that have a unit name inside a measurement of that name. Thus if i show measurements that match my model_y, for example, all i get are binary (e.g. unittless) sensors:

> show measurements with measurement =~ /sensoricareabout/
name: measurements
name
----
binary_sensor.sensoridontcareabout1
climate.sensoridontcareabout2
device_tracker.sensoridontcareabout3
...

However if i query series, similarly, i see the measurement i care about:

> show series where "entity_id" =~ /sensoricareabout/
key
---
°F,domain=sensor,entity_id=sensoricareabout_foo_1
°F,domain=sensor,entity_id=sensoricareabout_foo_2

Thus, oddly, to query the data i want i form a query something like this:

 select value from "°F" where entity_id = 'sensoricareabout_foo_1'

A couple things. So, it is odd to me to have to query from what I essentially view as a table °F… but that is probably more HA’s fault than influxdb. Second, i think the quoting is a little weird. Note above that the measurement is double quoted, whereas the tag is single quoted. This is important. You can sometimes (maybe?) leave the double quotes off the former but the latter must strictly be single quotes…

The moral of the story is that if you want to find the data you are looking for you need to look under ‘%’ or the actual unit of measurement. For example in grafana i would:

  • click on explore
  • select ‘%’
  • Add a where clause to filter by entity_id
  • Possibly find my series there…
    • But if not remove the where, and switch from % to the unit of measurement, and repeat by adding the where clause

This all works well but is a little unintuitive and might make one think HA has forgotten to put the data into influx.

Computer Vision, Deep Learning, Raspberry Pi, Web Programming

Architecture For DIY AI-Driven Home Security

Leave a comment

The raspberry pi’s ecosystem makes it a tempting platform for building computer vision projects such as home security systems. For around $80, one can assemble a pi (with sd card, camera, case, and power) that can capture video and suppress dead scenes using motion detection – typically with motion or MotionEye. This low-effort, low-cost solution seems attractive until one considers some of its shortfalls:

  • False detection events. The algorithm used to detect motion is suseptible to false positives – tree branches waving in the wind, clouds, etc. A user works around this by tweeking motion parameters (how BIG must an object be) or masking out regions (don’t look at the sky, just the road)
  • Lack of high level understanding. Even in tweeking the motion parameters anything that is moving is deemed of concern. There is no way to discriminate between a moving dog and a moving person.

The net result of these flaws – which all stem from a lack of real understanding – is wasted time. At a minimum the user is annoyed. Worse they are fatigue and miss events or neglect responding entirely.

By applying current state of the art AI techniques such as object detection, facial detection/recognition, one can vastly reduce the load on the user. To do this at full frame rate one needs to add an accelerator, such as the Coral TPU.

In testing we’ve found fairly good accuracy at almost full frame rate. Although Coral claims “400 fps” of speed – this is inference, not the full cycle of loading the image, running inference, and then examining the results. In real-world testing we found the full-cycle results closer to 15fps. This is still significantly better than the 2-3 fps one obtains by running in software.

In terms of scalability, running inference on the pi means we can scale endlessly. The server’s job is simply to log the video and metadata (object information, motion masks, etc.).

Here’s a rough sketch of such a system:

This approach is currently working successfully to provide the following, per rpi camera:

  • moving / static object detection
  • facial recognition
  • 3d object mapping – speed / location determination

This is all done at around 75% CPU utilization on a 2GB Rpi 4B. The imagery and metadata are streamed to a central server which performs no processing other than to archive the data from the cameras and serve it to clients (connected via an app or web page).

Uncategorized

Why a Tesla Model Y Is The Best Car Ever

Leave a comment

Lots of people have opinions. These are mine as to the supremecy of the model y:

  • Purchase experience. In case the entire world has not clearly understood what the past 20 years has increasingly shown: people would rather order online in person. Let that sink in deeply if it causes you pause. Here’s how Tesla makes that work:
    • You can test drive a tesla – this is entirely optional. I test drove a model S and model Y before deciding sedans are the worst and the Y, being “not a sedan”, is good for me. I am now convinced all the test drive did was excite me make the purchase – however i now know, as will be described next, that this was entirely unnecessary.
    • When you go to pick up your car a few weeks later (took 4 weeks for me, at a time when tesla was advertising 6-8), you spend ~1 hour inspecting the car and ensuring the title is transferred. Better yet, you get the car for 1,000 miles or 7 days (whichever comes first) – then you can send it back at zero cost. This is an amazing deal. I’m not sure you get that kind of a guarantee with any major car company.
    • Tesla will fix any delivery-related issue at no cost. When i accepted delivery I noted some superficial items that needed fixed:
      • Greese from seat on rear carpet. Tesla tech cleaned it up on
      • Scuffed portable charger. The charger looked like someone had been carrying it around in their (large) pockets for a few weeks. The tech switched it. I didnt like the one he found any better so he switched it again. The guy was VERY accomodating.
      • Windshield washer fluid connector was disconnected. I didn’t discover this until AFTER i drove home, but thankfully the connector was under the hood and i simply stuck it together
      • Paint missing on bolts around rear liftgate and front doors. I didn’t notice the liftgate bolt problem until recently so i have not bothered to fix it, but the paint on the front door bolts was fixed on the spot during delivery.
  • Service. After accepting delivery I noticed a few issues: Tesla came TO MY HOUSE and fixed them at zero cost. Turns out they had to come twice due to missing parts. The issues they fixed were minor:
    • There was a scratch on part of the passenger sun visor. They replaced it
    • The inner liner had slumped down (near sunroof). They reconnected it. If it happens again they recommended taking it in.
    • The weather seals around the back doors were detaching – they put on new ones
    • The Tesla floor markers inside the front doors were wiggly – they removed them and put new ones on
  • Autopilot.
    • Having autopilot on is best described in one word: soothing. Its like having a buddy drive for you – almost. You do have to let autopilot know you’re not asleep by jerking the wheel every 15-30 seconds, but even then your roll as driver is more as a supervisor of a much more aware computer. It has more eyes, including sonar and radar, and keeps track of 1) what is around you, 2) where you’re going (your route), 3) and when you need to fuel (it even picks where you should stop to supercharge)
    • At least once autopilot saved me when i became drowsy. Good lukc getting that in a normal car
    • It does an excellent job on highways – making complicated interchanges correctly where I as a human have failed.
  • Speed. While tesla does sell a performance model Y, the standard long range left me feeling like i was manning a rocket ship. After a month of driving the tesla i went back to our other gas powered car. I used to think it was a fast car – but it feels like a joke compared to the tesla. Im convinced if i were raised on teslas and then asked to drive any standard gas powered car i would think it was broken – it doesnt even appear to accelerate compared to the tesla.
  • Charging. There are two main components here:
    • Supercharging. I took a trip across the country to see the how the Y and supercharger network would do together. It was absolutely a pleasure; we ALWAYS had enough energy to make it between superchargers. The only problem we found was coming back from UT, we wanted to drop down to Nauvoo, IL, however we were on HWY 80, and the only way to hit Nauvoo and keep on our trip was to drop straight down from Davenport, then go back to Davenport. The better, time savings route would have been to drop from 80 to nauvoo, then continue on HWY 74 – however this was not possible as there wasnt a supercharger near enough to let us continue on this route; so we ended up ditching nauvoo. Moral: while the superchrger network is currently very good, it needs ot be several times larger – kind of like the network of gas station.
    • Home charging. I spend seconds connecting my charger to my wall at home. No more gas stations. No more oil changes. The best.
  • Being part of the future. I love time/life-saving technology. There is nothing more human, in my opinion, than trying to improve. The tesla is like a breath of fresh air to an industry that hasn’t changed much in a generation. It is layers of improvement all at once in a single impressive package. It hits on several themes:
    • Less is more. As in less buttons is more better. My odyssey has over 60 buttons in the front. The tesla has 6ish – 4 for windows, 2 for the door locks. Everything else is on that one big beautiful screen.
    • Audio commands that work. I can tell it to navigate, call people on the phone, send text messages, all while NOT diverting my eyes from the road if i dont want to.
    • Over the air updates. I have received one update every week or so, improving things such as efficiency, autopilot, etc. I truly feel like the car is getting better as i own it.
    • Electricity vs gas. Other than the fact that it takes a lot longer to charge than to get a full tank of gas, tesla has shown the model Y is superior in almost every way. While we clearly have a long ways to go – maybe faster superchargrs or longer-range batteries would help – for most people their daily commute is entirely drivable with the model Y today, where the recharging is done overnight at home.

As I was contemplating purchasing a new odyseey or the model Y I wasn’t sure if the model Y would be as good as the odyssey. I took a risk on the model Y. After the past few months I now have the opposite view: there is no way anyone could tempt me to sink a dollar into a new gas powered car. The features gas powered cars offer are fairly irrelevant – fancy interiors and updated trims. But who wants to smell gas odors in their garage just because you had to move your car? Who really wants to go to the gas station or have their oil changed? Who really wants to visit a dealership to find a car? Or to take it to a repairshop to fix its vastly more complicated (and, from personal experience, breakable) interiors? Not I.

In short: the big car makers approach (end to end) is leaving the gas powered industry in Tesla’s dust. My message to them (in case they’re reading this humble blog): be like Tesla or you’re going to die.

Databases

Adding in Custom Indices to Elastiflow

Leave a comment

Let’s say you have an elastiflow docker instance set up. This stack pushes all flow info into an index named “elastiflow-<version>-<Year>.<Month>.<Day>”. What if you wanted to use the same ELK stack for both elastiflow AND other stuff?

This is possible, of course!

Clone the elastiflow git repo

Cd into the repo

Add a new input filter to logstash/elastiflow/conf.d/10_input_syslog.conf . For example to bring in syslog:

input {
  udp {
    host => "0.0.0.0"
    port => 10514
    codec => "json"
    type => "rsyslog"
    tags => ["rsyslog"]
  }
}

filter { }

Modify logstash/elastiflow/conf.d/30_output_10_single.logstash.conf


output {
  if "rsyslog" in [tags]  {
    elasticsearch {
      user => "${ELASTIFLOW_ES_USER:elastic}"
      password => "${ELASTIFLOW_ES_PASSWD:changeme}"
      hosts => [ "172.10.4.1:9200" ]
      index => "logstash-%{+YYYY.MM.dd}"
      template => "${ELASTIFLOW_TEMPLATE_PATH:/etc/logstash/elastiflow/templates}/logstash.template.json"
      template_name => "logstash-1.0.0"
    
    }
  } else {
    elasticsearch {
      id => "output_elasticsearch_single"
      hosts => [ "${ELASTIFLOW_ES_HOST:127.0.0.1:9200}" ]
      ssl => "${ELASTIFLOW_ES_SSL_ENABLE:false}"
      ssl_certificate_verification => "${ELASTIFLOW_ES_SSL_VERIFY:false}"
      # If ssl_certificate_verification is true, uncomment cacert and set the path to the certificate.
      #cacert => "/PATH/TO/CERT"
      user => "${ELASTIFLOW_ES_USER:elastic}"
      password => "${ELASTIFLOW_ES_PASSWD:changeme}"
      index => "elastiflow-3.5.3-%{+YYYY.MM.dd}"
      template => "${ELASTIFLOW_TEMPLATE_PATH:/etc/logstash/elastiflow/templates}/elastiflow.template.json"
      template_name => "elastiflow-3.5.3"
      template_overwrite => "true"
    }
  } 
}

Rebuild the image:

docker build --tag logstash-elastiflow-custom:1.0 .

Now bring up your stack, e.g. “docker-compose up -d”

Now let’s test it. We can generate a new syslog message by, say, logging into the syslog server. If we do this the server shows the following message:

Mar 31 08:37:37 zoobie-2-1 sshd[2625]: Accepted publickey for magplus from 172.10.4.32 port 61811 ssh2: RSA SHA256:2dui2biubddjwbdjbd

If we go to kibana -> Management and create an index, we should see a new logstash index. Add it to kibana. Then view the index in the discover view. It should look like this:

node.hostname:elk.myhomenet.sys node.ipaddr:172.10.4.1 event.type:rsyslog event.host:syslogserver.myhomenet.sys @version:3.5.3 facility:auth @timestamp:Mar 31, 2020 @ 08:24:52.000 sysloghost:zoobie-2-1 severity:info programname:sshd procid:2575 logstash_host:syslogserver.myhomenet.sys tags:rsyslog message: Accepted publickey for magplus from 172.10.4.32 port 61736 ssh2: RSA SHA256:2dui2biubddjwbdjbd _id:IJ6xanIBxE6Ab_zHIO3i _type:_doc _index:logstash-2020.03.31 _score:0

And there you have it!

NOTE: This example does not cover setting up syslog forwarding, which is required to get syslog into logstash. For a good example of this go to this Digital Ocean tutorial on syslog and logstash

Deep Learning

Armchair Deep Learning Enthusiast: Object Detection Tips #1

Leave a comment

Recently I’ve been using the Tensorflow Object Detection API. Much of my approach follows material provided on pyimagesearch.com, this API allows us to perform. Rather than rehash that material, I just wanted to give a few pointers that I found helpful.

  • Don’t limit yourself to the ImageNet Large Scale Visual Recognition (ILSVRC) dataset. This dataset is credited with helping really juice up the state of the art in image recognition, however there are some serious problems for armchair DL enthusiasts like myself
    1. It isn’t easy to get the actual images for training. You have to petition to have access to the dataset, and your email must not look “common”. Uh-huh – so what are your options? Sure, you can get links to the images on the imagenet website and download them yourself Sure you could get the bounding boxes – but how do you match them up with the images you manually downloaded. I don’t want any of this – I just want the easy button; download it and use it. Well – the closest thing you’ll get to that is to grab the images from academic torrents: http://academictorrents.com/collection/imagenet-lsvrc-2015. but even that doesn’t feel satisying – if imagenet is about moving the state of the art forward, and assuming i even had the capability o do so, they sure aren’t making it easy for me to do that!
    2. It seems outdated. The object detection dataset hasnt changed since 2012. That is probably good for stability but the total image size (~1M images) no longer seems big. Peoples hairstyles, clothing, etc. are all changing – time for an update!
    3. Oh, that’s right – there is no official “person” synset inside the ILSVRC image set! So don’t worry about those out of date hair styles or clothes!
    4. There are better datasets out there. Bottom line – people are moving to other data sets and you should too.
      1. Open Images being one of the best
      2. Oh, and you can download subsets of this easily using a tool like https://github.com/harshilpatel312/open-images-downloader.git.
  • The TFOD flow is easy to follow – provided you use the right tensorflow version.
    • You TFOD is not compatible with tensorflow 2.0. You have to use the 1.x series.
    • I am using anaconda to download tensorflow-gpu version 1.15.0. To do this type “conda install tensorflow-gpu=1.15.0” (inside an activated anaconda instance)
    • You then grab the TFOD library, as per the website
  • Make sure you actually feed TFOD data, else you get weird hanging-like behavior.
    • At some point i found that tensorflow was crashing because a bounding box was outside the image.
    • In the process of fixing tht i introduced a bug that caused zero records to be sent to tensorflow
    • When i then ran a training loop I saw tensorflow progress as usual until it reported it had loaded libcublas: “Successfully opened dynamic library libcublas.so.10.2”
    • I thought this was a tensorflow issue, and even found a github issue for it Successfully opened dynamic library libcublas.so.10.0′ – however this was all a red herring. It was NOT because of tensorflow, it was just because my bounding box fix had eliminated all bounding boxes. Once i fixed that alll was well
  • Make sure you provide enough discriminatory data. E.g. if you want to find squirrels, don’t just train on the squirrel set, otherwise your detector will think almost anything that looks like an object is a squirrel. Add in a few other data sets and you will find that squirrels are squirrels and everything else is hit or miss.

Deep Learning

How I Set Up DLIB

Leave a comment

It is fairly easy to check out and build dlib. Getting it to work in a performance-optimized manner – python bindings included -takes a little more work.

Per the dlib github one can build the bindings by simply issuing:

python setup.py install

First problem I found is that the setup process decided to latch on to an old version of CUDA. That was my bad – fixed by moving my PATH variable to point to the new cuda’s bin dir.

Second problem is that during compilation I saw the following:

Invoking CMake build: 'cmake --build . --config Release -- -j12'
[  1%] Building NVCC (Device) object dlib_build/CMakeFiles/dlib.dir/cuda/dlib_generated_cusolver_dlibapi.cu.o
[  2%] Building NVCC (Device) object dlib_build/CMakeFiles/dlib.dir/cuda/dlib_generated_cuda_dlib.cu.o
/home/carson/code/2020/facenet/dlib/dlib/cuda/cuda_dlib.cu(1762): error: calling a constexpr __host__ function("log1p") from a __device__ function("cuda_log1pexp") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

As it suggests this is resolved by passing in a flag to the compiler. To do this modify the setup.py line:

python setup.py install --set USE_AVX_INSTRUCTIONS=1 --set DLIB_USE_CUDA=1 --set CUDA_NVCC_FLAGS="--expt-relaxed-constexpr"

Everything went just peachy from there except when I attempted to use dlib from within python I got an error (something like):

dlib 19.19.99 is missing cblas_dtrsm symbol

After which i tried importing face_recognition and got a segfault.

I fixed this by install openblas-devel, then re-ran the setup.py script as above. Magically this fixed everything.

Again, not bad – dlib seems cool – just normal troubleshooting stuff.

Deep Learning, Video

GPUs Are Cool

Leave a comment

Expect no witty sayings or clever analyses here – I just think GPUs are cool. And here are a few reasons why:

Exhibit A: Machine Learning Training a standard feed forward neural net on CIFAR-10 progresses at 50usec/sample; my 2.4 Ghz i7 takes almost 500usec/sample. The total set takes around 5 min to train on the GPU vs over a 35 min on my CPU. On long tasks this means a difference of days to weeks.

Exhibit B: Video transcoding In order to make backups of all my blu-ray disks, I rip and transcode them using ffmpeg or handbrake. Normally Im lucky to get a few dozen frames per second – completely making out my CPU during the process. By compiling ffmpeg to include nvenc/cuda support I get 456 fps (19x faster). As the screenshots show, my avg cpu usage was below 20% – and even GPU usage stayed under 10%. Video quality was superb (i couldnt tell the difference). 

ffmpeg -vsync 0 -hwaccel cuvid -i 00800.m2ts -c:a copy -c:v h264_nvenc -b:v 5M prince_egypt.mp4
  • CPU Load During NVENC encoding
  • GPU Load During NVENC encoding
RAW frame from blu-ray
Same frame after ffmpeg/nvenc transcoding

My setup:

  • GPU: RTX 2070 Super (8GB ram)
  • CPU: i7-8700K (6 core HT @3.7Ghz)
  • RAM: 32GB
  • Disk: 1TB PM981 (NVME)