Category Archives: Raspberry Pi

Architecture For DIY AI-Driven Home Security

The raspberry pi’s ecosystem makes it a tempting platform for building computer vision projects such as home security systems. For around $80, one can assemble a pi (with sd card, camera, case, and power) that can capture video and suppress dead scenes using motion detection – typically with motion or MotionEye. This low-effort, low-cost solution seems attractive until one considers some of its shortfalls:

  • False detection events. The algorithm used to detect motion is suseptible to false positives – tree branches waving in the wind, clouds, etc. A user works around this by tweeking motion parameters (how BIG must an object be) or masking out regions (don’t look at the sky, just the road)
  • Lack of high level understanding. Even in tweeking the motion parameters anything that is moving is deemed of concern. There is no way to discriminate between a moving dog and a moving person.

The net result of these flaws – which all stem from a lack of real understanding – is wasted time. At a minimum the user is annoyed. Worse they are fatigue and miss events or neglect responding entirely.

By applying current state of the art AI techniques such as object detection, facial detection/recognition, one can vastly reduce the load on the user. To do this at full frame rate one needs to add an accelerator, such as the Coral TPU.

In testing we’ve found fairly good accuracy at almost full frame rate. Although Coral claims “400 fps” of speed – this is inference, not the full cycle of loading the image, running inference, and then examining the results. In real-world testing we found the full-cycle results closer to 15fps. This is still significantly better than the 2-3 fps one obtains by running in software.

In terms of scalability, running inference on the pi means we can scale endlessly. The server’s job is simply to log the video and metadata (object information, motion masks, etc.).

Here’s a rough sketch of such a system:

This approach is currently working successfully to provide the following, per rpi camera:

  • moving / static object detection
  • facial recognition
  • 3d object mapping – speed / location determination

This is all done at around 75% CPU utilization on a 2GB Rpi 4B. The imagery and metadata are streamed to a central server which performs no processing other than to archive the data from the cameras and serve it to clients (connected via an app or web page).

Installing OpenCV 4.1.1 on Raspberry Pi 4

Recently I purchased a raspberry pi 4 to see how it performs basic computer vision tasks. I largely followed this guide on building opencv https://www.learnopencv.com/install-opencv-4-on-raspberry-pi/

However, in the guide there are several missing dependencies on a fresh version of raspbian buster. There are also some apparent errors in the CMakeLists.txt file which other users already discovered. After patching these fixes and adding the needed missing dependencies, I now have opencv running on my pi. Here’s my complete script below.

IMPORTANT: You must READ the script, don’t just run it! For example, you should check the version of python you have at the time you run this script. When I ran it i was at python 3.7. Also, feel free to bump to a later version (or master) of opencv.

Oh, also during the numpy step it hung and i was too lazy to look into it. It didn’t seem to affect my ability to use opencv – so i didn’t go back and dig. My bad.

#!/bin/bash

cvVersion="4.1.1"
pythonVersion="3.7"
opencvDirRoot=/home/pi/opencv

sudo apt-get -y purge wolfram-engine
sudo apt-get -y purge libreoffice*
sudo apt-get -y clean
sudo apt-get -y autoremove

mkdir -p $opencvDirRoot
cd $opencvDirRoot

# Clean build directories
rm -rf opencv/build
rm -rf opencv_contrib/build

# Create directory for installation
rm -fr installation
mkdir -p installation
mkdir installation/OpenCV-"$cvVersion"


sudo apt -y update
sudo apt -y upgrade
sudo apt-get -y remove x264 libx264-dev
 
## Install dependencies
sudo apt-get install libblas-dev liblapack-dev
sudo apt-get install libeigen3-dev
sudo apt-get -y install qtbase5-dev qtdeclarative5-dev
sudo apt-get -y install build-essential checkinstall cmake pkg-config yasm
sudo apt-get -y install git gfortran
sudo apt-get -y install libjpeg8-dev libjasper-dev libpng12-dev

 
sudo apt-get -y install libtiff5-dev
 
sudo apt-get -y install libtiff-dev

sudo apt-get -y install libavcodec-dev libavformat-dev libswscale-dev libdc1394-22-dev
sudo apt-get -y install libxine2-dev libv4l-dev
cd /usr/include/linux
sudo ln -s -f ../libv4l1-videodev.h videodev.h
cd $opencvDirRoot

sudo apt-get -y install libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev
sudo apt-get -y install libgtk2.0-dev libtbb-dev qt5-default
sudo apt-get -y install libatlas-base-dev
sudo apt-get -y install libmp3lame-dev libtheora-dev
sudo apt-get -y install libvorbis-dev libxvidcore-dev libx264-dev
sudo apt-get -y install libopencore-amrnb-dev libopencore-amrwb-dev
sudo apt-get -y install libavresample-dev
sudo apt-get -y install x264 v4l-utils
sudo apt-get -y install libmesa-dev
sudo apt-get -y install freeglut3-dev



# Optional dependencies
sudo apt-get -y install libprotobuf-dev protobuf-compiler
sudo apt-get -y install libgoogle-glog-dev libgflags-dev
sudo apt-get -y install libgphoto2-dev libeigen3-dev libhdf5-dev doxygen

sudo apt-get -y install python3-dev python3-pip
sudo -H pip3 install -U pip numpy
sudo apt-get -y install python3-testresources

cd $opencvDirRoot
# Install virtual environment
python3 -m venv OpenCV-"$cvVersion"-py3
echo "# Virtual Environment Wrapper" >> ~/.bashrc
echo "alias workoncv-$cvVersion=\"source $opencvDirRoot/OpenCV-$cvVersion-py3/bin/activate\"" >> ~/.bashrc
source "$opencvDirRoot"/OpenCV-"$cvVersion"-py3/bin/activate
#############

############ For Python 3 ############
# now install python libraries within this virtual environment
sudo sed -i 's/CONF_SWAPSIZE=100/CONF_SWAPSIZE=1024/g' /etc/dphys-swapfile
sudo /etc/init.d/dphys-swapfile stop
sudo /etc/init.d/dphys-swapfile start
pip install numpy dlib
# quit virtual environment
deactivate

git clone https://github.com/opencv/opencv.git
cd opencv
git checkout $cvVersion
cd ..

git clone https://github.com/opencv/opencv_contrib.git
cd opencv_contrib
git checkout $cvVersion
cd ..

cd opencv
mkdir build
cd build


# Eigen/Core to eigen3/Eigen/Core
sed -i s,Eigen/Core,eigen3/Eigen/Core/g ../modules/core/include/opencv2/core/private.hpp

# Add these to  opencv/samples/cpp/CMakeLists.txt 
find_package(OpenGL REQUIRED)
find_package(GLUT REQUIRED)

cmake .. -D CMAKE_BUILD_TYPE=RELEASE \
            -D CMAKE_INSTALL_PREFIX=$opencvDirRoot/installation/OpenCV-"$cvVersion" \
            -D INSTALL_C_EXAMPLES=ON \
            -D INSTALL_PYTHON_EXAMPLES=ON \
            -D WITH_TBB=ON \
            -D WITH_V4L=ON \
            -D OPENCV_PYTHON3_INSTALL_PATH=$opencvDirRoot/OpenCV-$cvVersion-py3/lib/python$pythonVersion/site-packages \
        -D WITH_QT=ON \
        -D WITH_OPENGL=ON \
    -D OPENCV_EXTRA_EXE_LINKER_FLAGS=-latomic \
        -D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules \
        -D BUILD_EXAMPLES=ON


make -j$(nproc)
make install


sudo sed -i 's/CONF_SWAPSIZE=1024/CONF_SWAPSIZE=100/g' /etc/dphys-swapfile
sudo /etc/init.d/dphys-swapfile stop
sudo /etc/init.d/dphys-swapfile start

echo "sudo modprobe bcm2835-v4l2" >> ~/.profile
                                                                                                                                        

WiFi Performance on Raspberry Pi 3

I purchased a couple raspberry pi 3s. The major draw is that you no longer need an external wifi adapter – yet the cost is the same as the older pis.

The very first thing I noticed is that ssh traffic was stuttery. This cleared up after performing an apt-get update/upgrade. Everything seemed silky-smooth thereafter.