Containers, HomeAssistant

InfluxDB Queries on HomeAssistant Data

Leave a comment

I recently added in influx db + grafana + mariadb to one of my HA instances. I was surprised at how easy it was to get HA happy. Essentially nothing more than running the docker instances for each of these components and adding minimal yaml to HA.

When i went to query the data there were a few surprises that I thought I’d note. First, as noted in https://community.home-assistant.io/t/missing-sensors-in-influxdb/144948, HA puts sensors that have a unit name inside a measurement of that name. Thus if i show measurements that match my model_y, for example, all i get are binary (e.g. unittless) sensors:

> show measurements with measurement =~ /sensoricareabout/
name: measurements
name
----
binary_sensor.sensoridontcareabout1
climate.sensoridontcareabout2
device_tracker.sensoridontcareabout3
...

However if i query series, similarly, i see the measurement i care about:

> show series where "entity_id" =~ /sensoricareabout/
key
---
°F,domain=sensor,entity_id=sensoricareabout_foo_1
°F,domain=sensor,entity_id=sensoricareabout_foo_2

Thus, oddly, to query the data i want i form a query something like this:

 select value from "°F" where entity_id = 'sensoricareabout_foo_1'

A couple things. So, it is odd to me to have to query from what I essentially view as a table °F… but that is probably more HA’s fault than influxdb. Second, i think the quoting is a little weird. Note above that the measurement is double quoted, whereas the tag is single quoted. This is important. You can sometimes (maybe?) leave the double quotes off the former but the latter must strictly be single quotes…

The moral of the story is that if you want to find the data you are looking for you need to look under ‘%’ or the actual unit of measurement. For example in grafana i would:

  • click on explore
  • select ‘%’
  • Add a where clause to filter by entity_id
  • Possibly find my series there…
    • But if not remove the where, and switch from % to the unit of measurement, and repeat by adding the where clause

This all works well but is a little unintuitive and might make one think HA has forgotten to put the data into influx.

Computer Vision, Deep Learning, Raspberry Pi, Web Programming

Architecture For DIY AI-Driven Home Security

Leave a comment

The raspberry pi’s ecosystem makes it a tempting platform for building computer vision projects such as home security systems. For around $80, one can assemble a pi (with sd card, camera, case, and power) that can capture video and suppress dead scenes using motion detection – typically with motion or MotionEye. This low-effort, low-cost solution seems attractive until one considers some of its shortfalls:

  • False detection events. The algorithm used to detect motion is suseptible to false positives – tree branches waving in the wind, clouds, etc. A user works around this by tweeking motion parameters (how BIG must an object be) or masking out regions (don’t look at the sky, just the road)
  • Lack of high level understanding. Even in tweeking the motion parameters anything that is moving is deemed of concern. There is no way to discriminate between a moving dog and a moving person.

The net result of these flaws – which all stem from a lack of real understanding – is wasted time. At a minimum the user is annoyed. Worse they are fatigue and miss events or neglect responding entirely.

By applying current state of the art AI techniques such as object detection, facial detection/recognition, one can vastly reduce the load on the user. To do this at full frame rate one needs to add an accelerator, such as the Coral TPU.

In testing we’ve found fairly good accuracy at almost full frame rate. Although Coral claims “400 fps” of speed – this is inference, not the full cycle of loading the image, running inference, and then examining the results. In real-world testing we found the full-cycle results closer to 15fps. This is still significantly better than the 2-3 fps one obtains by running in software.

In terms of scalability, running inference on the pi means we can scale endlessly. The server’s job is simply to log the video and metadata (object information, motion masks, etc.).

Here’s a rough sketch of such a system:

This approach is currently working successfully to provide the following, per rpi camera:

  • moving / static object detection
  • facial recognition
  • 3d object mapping – speed / location determination

This is all done at around 75% CPU utilization on a 2GB Rpi 4B. The imagery and metadata are streamed to a central server which performs no processing other than to archive the data from the cameras and serve it to clients (connected via an app or web page).

Uncategorized

Why a Tesla Model Y Is The Best Car Ever

Leave a comment

Lots of people have opinions. These are mine as to the supremecy of the model y:

  • Purchase experience. In case the entire world has not clearly understood what the past 20 years has increasingly shown: people would rather order online in person. Let that sink in deeply if it causes you pause. Here’s how Tesla makes that work:
    • You can test drive a tesla – this is entirely optional. I test drove a model S and model Y before deciding sedans are the worst and the Y, being “not a sedan”, is good for me. I am now convinced all the test drive did was excite me make the purchase – however i now know, as will be described next, that this was entirely unnecessary.
    • When you go to pick up your car a few weeks later (took 4 weeks for me, at a time when tesla was advertising 6-8), you spend ~1 hour inspecting the car and ensuring the title is transferred. Better yet, you get the car for 1,000 miles or 7 days (whichever comes first) – then you can send it back at zero cost. This is an amazing deal. I’m not sure you get that kind of a guarantee with any major car company.
    • Tesla will fix any delivery-related issue at no cost. When i accepted delivery I noted some superficial items that needed fixed:
      • Greese from seat on rear carpet. Tesla tech cleaned it up on
      • Scuffed portable charger. The charger looked like someone had been carrying it around in their (large) pockets for a few weeks. The tech switched it. I didnt like the one he found any better so he switched it again. The guy was VERY accomodating.
      • Windshield washer fluid connector was disconnected. I didn’t discover this until AFTER i drove home, but thankfully the connector was under the hood and i simply stuck it together
      • Paint missing on bolts around rear liftgate and front doors. I didn’t notice the liftgate bolt problem until recently so i have not bothered to fix it, but the paint on the front door bolts was fixed on the spot during delivery.
  • Service. After accepting delivery I noticed a few issues: Tesla came TO MY HOUSE and fixed them at zero cost. Turns out they had to come twice due to missing parts. The issues they fixed were minor:
    • There was a scratch on part of the passenger sun visor. They replaced it
    • The inner liner had slumped down (near sunroof). They reconnected it. If it happens again they recommended taking it in.
    • The weather seals around the back doors were detaching – they put on new ones
    • The Tesla floor markers inside the front doors were wiggly – they removed them and put new ones on
  • Autopilot.
    • Having autopilot on is best described in one word: soothing. Its like having a buddy drive for you – almost. You do have to let autopilot know you’re not asleep by jerking the wheel every 15-30 seconds, but even then your roll as driver is more as a supervisor of a much more aware computer. It has more eyes, including sonar and radar, and keeps track of 1) what is around you, 2) where you’re going (your route), 3) and when you need to fuel (it even picks where you should stop to supercharge)
    • At least once autopilot saved me when i became drowsy. Good lukc getting that in a normal car
    • It does an excellent job on highways – making complicated interchanges correctly where I as a human have failed.
  • Speed. While tesla does sell a performance model Y, the standard long range left me feeling like i was manning a rocket ship. After a month of driving the tesla i went back to our other gas powered car. I used to think it was a fast car – but it feels like a joke compared to the tesla. Im convinced if i were raised on teslas and then asked to drive any standard gas powered car i would think it was broken – it doesnt even appear to accelerate compared to the tesla.
  • Charging. There are two main components here:
    • Supercharging. I took a trip across the country to see the how the Y and supercharger network would do together. It was absolutely a pleasure; we ALWAYS had enough energy to make it between superchargers. The only problem we found was coming back from UT, we wanted to drop down to Nauvoo, IL, however we were on HWY 80, and the only way to hit Nauvoo and keep on our trip was to drop straight down from Davenport, then go back to Davenport. The better, time savings route would have been to drop from 80 to nauvoo, then continue on HWY 74 – however this was not possible as there wasnt a supercharger near enough to let us continue on this route; so we ended up ditching nauvoo. Moral: while the superchrger network is currently very good, it needs ot be several times larger – kind of like the network of gas station.
    • Home charging. I spend seconds connecting my charger to my wall at home. No more gas stations. No more oil changes. The best.
  • Being part of the future. I love time/life-saving technology. There is nothing more human, in my opinion, than trying to improve. The tesla is like a breath of fresh air to an industry that hasn’t changed much in a generation. It is layers of improvement all at once in a single impressive package. It hits on several themes:
    • Less is more. As in less buttons is more better. My odyssey has over 60 buttons in the front. The tesla has 6ish – 4 for windows, 2 for the door locks. Everything else is on that one big beautiful screen.
    • Audio commands that work. I can tell it to navigate, call people on the phone, send text messages, all while NOT diverting my eyes from the road if i dont want to.
    • Over the air updates. I have received one update every week or so, improving things such as efficiency, autopilot, etc. I truly feel like the car is getting better as i own it.
    • Electricity vs gas. Other than the fact that it takes a lot longer to charge than to get a full tank of gas, tesla has shown the model Y is superior in almost every way. While we clearly have a long ways to go – maybe faster superchargrs or longer-range batteries would help – for most people their daily commute is entirely drivable with the model Y today, where the recharging is done overnight at home.

As I was contemplating purchasing a new odyseey or the model Y I wasn’t sure if the model Y would be as good as the odyssey. I took a risk on the model Y. After the past few months I now have the opposite view: there is no way anyone could tempt me to sink a dollar into a new gas powered car. The features gas powered cars offer are fairly irrelevant – fancy interiors and updated trims. But who wants to smell gas odors in their garage just because you had to move your car? Who really wants to go to the gas station or have their oil changed? Who really wants to visit a dealership to find a car? Or to take it to a repairshop to fix its vastly more complicated (and, from personal experience, breakable) interiors? Not I.

In short: the big car makers approach (end to end) is leaving the gas powered industry in Tesla’s dust. My message to them (in case they’re reading this humble blog): be like Tesla or you’re going to die.

Databases

Adding in Custom Indices to Elastiflow

Leave a comment

Let’s say you have an elastiflow docker instance set up. This stack pushes all flow info into an index named “elastiflow-<version>-<Year>.<Month>.<Day>”. What if you wanted to use the same ELK stack for both elastiflow AND other stuff?

This is possible, of course!

Clone the elastiflow git repo

Cd into the repo

Add a new input filter to logstash/elastiflow/conf.d/10_input_syslog.conf . For example to bring in syslog:

input {
  udp {
    host => "0.0.0.0"
    port => 10514
    codec => "json"
    type => "rsyslog"
    tags => ["rsyslog"]
  }
}

filter { }

Modify logstash/elastiflow/conf.d/30_output_10_single.logstash.conf


output {
  if "rsyslog" in [tags]  {
    elasticsearch {
      user => "${ELASTIFLOW_ES_USER:elastic}"
      password => "${ELASTIFLOW_ES_PASSWD:changeme}"
      hosts => [ "172.10.4.1:9200" ]
      index => "logstash-%{+YYYY.MM.dd}"
      template => "${ELASTIFLOW_TEMPLATE_PATH:/etc/logstash/elastiflow/templates}/logstash.template.json"
      template_name => "logstash-1.0.0"
    
    }
  } else {
    elasticsearch {
      id => "output_elasticsearch_single"
      hosts => [ "${ELASTIFLOW_ES_HOST:127.0.0.1:9200}" ]
      ssl => "${ELASTIFLOW_ES_SSL_ENABLE:false}"
      ssl_certificate_verification => "${ELASTIFLOW_ES_SSL_VERIFY:false}"
      # If ssl_certificate_verification is true, uncomment cacert and set the path to the certificate.
      #cacert => "/PATH/TO/CERT"
      user => "${ELASTIFLOW_ES_USER:elastic}"
      password => "${ELASTIFLOW_ES_PASSWD:changeme}"
      index => "elastiflow-3.5.3-%{+YYYY.MM.dd}"
      template => "${ELASTIFLOW_TEMPLATE_PATH:/etc/logstash/elastiflow/templates}/elastiflow.template.json"
      template_name => "elastiflow-3.5.3"
      template_overwrite => "true"
    }
  } 
}

Rebuild the image:

docker build --tag logstash-elastiflow-custom:1.0 .

Now bring up your stack, e.g. “docker-compose up -d”

Now let’s test it. We can generate a new syslog message by, say, logging into the syslog server. If we do this the server shows the following message:

Mar 31 08:37:37 zoobie-2-1 sshd[2625]: Accepted publickey for magplus from 172.10.4.32 port 61811 ssh2: RSA SHA256:2dui2biubddjwbdjbd

If we go to kibana -> Management and create an index, we should see a new logstash index. Add it to kibana. Then view the index in the discover view. It should look like this:

node.hostname:elk.myhomenet.sys node.ipaddr:172.10.4.1 event.type:rsyslog event.host:syslogserver.myhomenet.sys @version:3.5.3 facility:auth @timestamp:Mar 31, 2020 @ 08:24:52.000 sysloghost:zoobie-2-1 severity:info programname:sshd procid:2575 logstash_host:syslogserver.myhomenet.sys tags:rsyslog message: Accepted publickey for magplus from 172.10.4.32 port 61736 ssh2: RSA SHA256:2dui2biubddjwbdjbd _id:IJ6xanIBxE6Ab_zHIO3i _type:_doc _index:logstash-2020.03.31 _score:0

And there you have it!

NOTE: This example does not cover setting up syslog forwarding, which is required to get syslog into logstash. For a good example of this go to this Digital Ocean tutorial on syslog and logstash

Deep Learning

Armchair Deep Learning Enthusiast: Object Detection Tips #1

Leave a comment

Recently I’ve been using the Tensorflow Object Detection API. Much of my approach follows material provided on pyimagesearch.com, this API allows us to perform. Rather than rehash that material, I just wanted to give a few pointers that I found helpful.

  • Don’t limit yourself to the ImageNet Large Scale Visual Recognition (ILSVRC) dataset. This dataset is credited with helping really juice up the state of the art in image recognition, however there are some serious problems for armchair DL enthusiasts like myself
    1. It isn’t easy to get the actual images for training. You have to petition to have access to the dataset, and your email must not look “common”. Uh-huh – so what are your options? Sure, you can get links to the images on the imagenet website and download them yourself Sure you could get the bounding boxes – but how do you match them up with the images you manually downloaded. I don’t want any of this – I just want the easy button; download it and use it. Well – the closest thing you’ll get to that is to grab the images from academic torrents: http://academictorrents.com/collection/imagenet-lsvrc-2015. but even that doesn’t feel satisying – if imagenet is about moving the state of the art forward, and assuming i even had the capability o do so, they sure aren’t making it easy for me to do that!
    2. It seems outdated. The object detection dataset hasnt changed since 2012. That is probably good for stability but the total image size (~1M images) no longer seems big. Peoples hairstyles, clothing, etc. are all changing – time for an update!
    3. Oh, that’s right – there is no official “person” synset inside the ILSVRC image set! So don’t worry about those out of date hair styles or clothes!
    4. There are better datasets out there. Bottom line – people are moving to other data sets and you should too.
      1. Open Images being one of the best
      2. Oh, and you can download subsets of this easily using a tool like https://github.com/harshilpatel312/open-images-downloader.git.
  • The TFOD flow is easy to follow – provided you use the right tensorflow version.
    • You TFOD is not compatible with tensorflow 2.0. You have to use the 1.x series.
    • I am using anaconda to download tensorflow-gpu version 1.15.0. To do this type “conda install tensorflow-gpu=1.15.0” (inside an activated anaconda instance)
    • You then grab the TFOD library, as per the website
  • Make sure you actually feed TFOD data, else you get weird hanging-like behavior.
    • At some point i found that tensorflow was crashing because a bounding box was outside the image.
    • In the process of fixing tht i introduced a bug that caused zero records to be sent to tensorflow
    • When i then ran a training loop I saw tensorflow progress as usual until it reported it had loaded libcublas: “Successfully opened dynamic library libcublas.so.10.2”
    • I thought this was a tensorflow issue, and even found a github issue for it Successfully opened dynamic library libcublas.so.10.0′ – however this was all a red herring. It was NOT because of tensorflow, it was just because my bounding box fix had eliminated all bounding boxes. Once i fixed that alll was well
  • Make sure you provide enough discriminatory data. E.g. if you want to find squirrels, don’t just train on the squirrel set, otherwise your detector will think almost anything that looks like an object is a squirrel. Add in a few other data sets and you will find that squirrels are squirrels and everything else is hit or miss.

Deep Learning

How I Set Up DLIB

Leave a comment

It is fairly easy to check out and build dlib. Getting it to work in a performance-optimized manner – python bindings included -takes a little more work.

Per the dlib github one can build the bindings by simply issuing:

python setup.py install

First problem I found is that the setup process decided to latch on to an old version of CUDA. That was my bad – fixed by moving my PATH variable to point to the new cuda’s bin dir.

Second problem is that during compilation I saw the following:

Invoking CMake build: 'cmake --build . --config Release -- -j12'
[  1%] Building NVCC (Device) object dlib_build/CMakeFiles/dlib.dir/cuda/dlib_generated_cusolver_dlibapi.cu.o
[  2%] Building NVCC (Device) object dlib_build/CMakeFiles/dlib.dir/cuda/dlib_generated_cuda_dlib.cu.o
/home/carson/code/2020/facenet/dlib/dlib/cuda/cuda_dlib.cu(1762): error: calling a constexpr __host__ function("log1p") from a __device__ function("cuda_log1pexp") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

As it suggests this is resolved by passing in a flag to the compiler. To do this modify the setup.py line:

python setup.py install --set USE_AVX_INSTRUCTIONS=1 --set DLIB_USE_CUDA=1 --set CUDA_NVCC_FLAGS="--expt-relaxed-constexpr"

Everything went just peachy from there except when I attempted to use dlib from within python I got an error (something like):

dlib 19.19.99 is missing cblas_dtrsm symbol

After which i tried importing face_recognition and got a segfault.

I fixed this by install openblas-devel, then re-ran the setup.py script as above. Magically this fixed everything.

Again, not bad – dlib seems cool – just normal troubleshooting stuff.

Deep Learning, Video

GPUs Are Cool

Leave a comment

Expect no witty sayings or clever analyses here – I just think GPUs are cool. And here are a few reasons why:

Exhibit A: Machine Learning Training a standard feed forward neural net on CIFAR-10 progresses at 50usec/sample; my 2.4 Ghz i7 takes almost 500usec/sample. The total set takes around 5 min to train on the GPU vs over a 35 min on my CPU. On long tasks this means a difference of days to weeks.

Exhibit B: Video transcoding In order to make backups of all my blu-ray disks, I rip and transcode them using ffmpeg or handbrake. Normally Im lucky to get a few dozen frames per second – completely making out my CPU during the process. By compiling ffmpeg to include nvenc/cuda support I get 456 fps (19x faster). As the screenshots show, my avg cpu usage was below 20% – and even GPU usage stayed under 10%. Video quality was superb (i couldnt tell the difference). 

ffmpeg -vsync 0 -hwaccel cuvid -i 00800.m2ts -c:a copy -c:v h264_nvenc -b:v 5M prince_egypt.mp4
  • CPU Load During NVENC encoding
  • GPU Load During NVENC encoding
RAW frame from blu-ray
Same frame after ffmpeg/nvenc transcoding

My setup:

  • GPU: RTX 2070 Super (8GB ram)
  • CPU: i7-8700K (6 core HT @3.7Ghz)
  • RAM: 32GB
  • Disk: 1TB PM981 (NVME)

Uncategorized

Hosting a Minecraft Server

Leave a comment

We decided to try out minecraft as a family. Our setup includes a half dozen devices (some iOS, some android, Xbox one, etc) with users spread out across the country.

Rather than pay for a realms subscription, we chose to host our own server. Besides being cheaper this gives us flexibility and control over the world data.

Server Setup

Run the docker container (we used the Bedrock Minecraft Server):

docker run -d -e EULA=TRUE --name cworld -e GAMEMODE=creative -e DIFFICULTY=normal -e SERVER_NAME=fenhome2 -v /home/<user>/data:/data  -p 127.0.0.1:19132:19132/udp itzg/minecraft-bedrock-server

Note 1 – I purposefully run the minecraft server only the localhost – not on the public IP. More on this later.

Note 2 – I specify a volume for the data. This lets you preserve minecraft state across server restarts and allows you to port it to other servers easily. Replace <user> with your username on your server.

For the sake of security we would like to allow access to only the clients that we specifically want to allow. To do this I found the easiest thing is to use one firewall-cmd per allowed host:

firewall-cmd --direct --add-rule ipv4 nat PREROUTING 0 -s <allowedHostIp>/32 -p udp --dport 19132 -j DNAT --to-destination  <dockerIp>:19132

Substitute the ip of the allowed host in place of <allowedHostIp>. To find your ip just go to http://whatismyip.com. The <dockerIp> can be found with the following command: docker exec -it cworld ip addr | grep 172

Connecting Android/iOS Clients

Its really easy to connect Android and iOS clients. Just click on the servers tab (shown here):

Then click on the “Add Server” button.

You will fill in the server name with whatever you want, however the hostname and port will be the hostname of your server and port (19132 in this example). See this example:

Connecting XBox One Clients

XBox clients are more challenging than Android/iOS because they don’t have the “Add Server” button. However there is still a way to make it work! The magic involves adding a “network proxy” on your network that tricks the XBox into thinking there is a local server. The XBox connects to this proxy which forwards traffic to your server.

  1. Download phantom from https://github.com/jhead/phantom
  2. Inside your phantom checkout run make
  3. Copy phantom_windows.exe to your windows box.
  4. On your windows box type: cmd
  5. In your console type cd <path to downloaded phantom-windows.exe">
  6. Type phantom-windows.exe -server <server>:19132
  7. If prompted by windows to allow phantom through your firewall say “yes.”

IMPORTANT: You will need to ensure phantom is running for the duration of your game play.

If you trust me (why would you?!) you can use the copy of phantom I built (md5 5e7595f82f117b545c511629df3304ba)

Add a Second Server for Survival Mode

If you want to start a second server for survival mode games the process is almost identical. The docker image is created as follows:

docker run -d -e EULA=TRUE --name cworld_survival -e GAMEMODE=survival -e DIFFICULTY=normal -e SERVER_NAME=fenhome2_survival -v /home/<user>/data_survival:/data  -p 127.0.0.1:19133:19132/udp itzg/minecraft-bedrock-server

And for the firewall

firewall-cmd --direct --add-rule ipv4 nat PREROUTING 0 -s <allowedHostIp>/32 -p udp --dport 19133 -j DNAT --to-destination  <dockerIp2>:19132

Where <dockerIp2> is the IP of the second minecraft server.

You can now add a second server in your android/iOS clients with the name “<server> survival”, <server> for the ip, and 19133 for the port.

Note that on XBox you will need a second phantom instance with 19133 as the port, as in:

phantom-windows.exe -server <server>:19133

Uncategorized

Installing pytorch/torchvision on centos7

Leave a comment

The installation command shown on pytorch.org didn’t quite work for me on centos7. Instead of using the anaconda installer provided on pytorch, I just used the yum-provided conda:

yum install conda

However when i went to run the command as specified I got the following error:

~$ conda install pytorch torchvision cudatoolkit=10.0 -c pytorch

NoBaseEnvironmentError: This conda installation has no default base environment. Use
'conda create' to create new environments and 'conda activate' to
activate environments.

It seems the problem is that pytorch is assuming you’ve set up a working conda environment – which i had not. You can work around this by simply naming the environment (I suppose you could also have made a base environemnt, but I decided to save that adventure for another day). The following did work for me:

conda create -n pytorch pytorch torchvision matplotlib cudatoolkit=10.0 -c pytorch

Note I also added matplotlib as I was following pytorch examples that requires it. Also, I found I could run any pytorch example by directly referencing the conda environment, instead of activating it and then running the script, as follows:

~/.conda/envs/pytorch/bin/python3.7 train.py