GPUs Are Cool

Expect no witty sayings or clever analyses here – I just think GPUs are cool.

For example: training a standard feed forward neural net on CIFAR-10 on my GPU (RTX 2070 Super) progresses at 50usec/sample; my 2.4 Ghz i7 takes almost 500usec/sample. The total set takes around 5 min to train on the GPU vs over a 35 min on my CPU.

Although this GPU wasn’t cheap ($500) it also wasn’t super expensive. And saving 30 min for a simple, repetitive task is well worth it.

Hosting a Minecraft Server

We decided to try out minecraft as a family. Our setup includes a half dozen devices (some iOS, some android, Xbox one, etc) with users spread out across the country.

Rather than pay for a realms subscription, we chose to host our own server. Besides being cheaper this gives us flexibility and control over the world data.

Server Setup

Run the docker container (we used the Bedrock Minecraft Server):

docker run -d -e EULA=TRUE --name cworld -e GAMEMODE=creative -e DIFFICULTY=normal -e SERVER_NAME=fenhome2 -v /home/<user>/data:/data  -p 127.0.0.1:19132:19132/udp itzg/minecraft-bedrock-server

Note 1 – I purposefully run the minecraft server only the localhost – not on the public IP. More on this later.

Note 2 – I specify a volume for the data. This lets you preserve minecraft state across server restarts and allows you to port it to other servers easily. Replace <user> with your username on your server.

For the sake of security we would like to allow access to only the clients that we specifically want to allow. To do this I found the easiest thing is to use one firewall-cmd per allowed host:

firewall-cmd --direct --add-rule ipv4 nat PREROUTING 0 -s <allowedHostIp>/32 -p udp --dport 19132 -j DNAT --to-destination  <dockerIp>:19132

Substitute the ip of the allowed host in place of <allowedHostIp>. To find your ip just go to http://whatismyip.com. The <dockerIp> can be found with the following command: docker exec -it cworld ip addr | grep 172

Connecting Android/iOS Clients

Its really easy to connect Android and iOS clients. Just click on the servers tab (shown here):

Then click on the “Add Server” button.

You will fill in the server name with whatever you want, however the hostname and port will be the hostname of your server and port (19132 in this example). See this example:

Connecting XBox One Clients

XBox clients are more challenging than Android/iOS because they don’t have the “Add Server” button. However there is still a way to make it work! The magic involves adding a “network proxy” on your network that tricks the XBox into thinking there is a local server. The XBox connects to this proxy which forwards traffic to your server.

  1. Download phantom from https://github.com/jhead/phantom
  2. Inside your phantom checkout run make
  3. Copy phantom_windows.exe to your windows box.
  4. On your windows box type: cmd
  5. In your console type cd <path to downloaded phantom-windows.exe">
  6. Type phantom-windows.exe -server <server>:19132
  7. If prompted by windows to allow phantom through your firewall say “yes.”

IMPORTANT: You will need to ensure phantom is running for the duration of your game play.

If you trust me (why would you?!) you can use the copy of phantom I built (md5 5e7595f82f117b545c511629df3304ba)

Add a Second Server for Survival Mode

If you want to start a second server for survival mode games the process is almost identical. The docker image is created as follows:

docker run -d -e EULA=TRUE --name cworld_survival -e GAMEMODE=survival -e DIFFICULTY=normal -e SERVER_NAME=fenhome2_survival -v /home/<user>/data_survival:/data  -p 127.0.0.1:19133:19132/udp itzg/minecraft-bedrock-server

And for the firewall

firewall-cmd --direct --add-rule ipv4 nat PREROUTING 0 -s <allowedHostIp>/32 -p udp --dport 19133 -j DNAT --to-destination  <dockerIp2>:19132

Where <dockerIp2> is the IP of the second minecraft server.

You can now add a second server in your android/iOS clients with the name “<server> survival”, <server> for the ip, and 19133 for the port.

Note that on XBox you will need a second phantom instance with 19133 as the port, as in:

phantom-windows.exe -server <server>:19133

Installing pytorch/torchvision on centos7

The installation command shown on pytorch.org didn’t quite work for me on centos7. Instead of using the anaconda installer provided on pytorch, I just used the yum-provided conda:

yum install conda

However when i went to run the command as specified I got the following error:

~$ conda install pytorch torchvision cudatoolkit=10.0 -c pytorch

NoBaseEnvironmentError: This conda installation has no default base environment. Use
'conda create' to create new environments and 'conda activate' to
activate environments.

It seems the problem is that pytorch is assuming you’ve set up a working conda environment – which i had not. You can work around this by simply naming the environment (I suppose you could also have made a base environemnt, but I decided to save that adventure for another day). The following did work for me:

conda create -n pytorch pytorch torchvision matplotlib cudatoolkit=10.0 -c pytorch

Note I also added matplotlib as I was following pytorch examples that requires it. Also, I found I could run any pytorch example by directly referencing the conda environment, instead of activating it and then running the script, as follows:

~/.conda/envs/pytorch/bin/python3.7 train.py

Installing OpenCV 4.1.1 on Raspberry Pi 4

Recently I purchased a raspberry pi 4 to see how it performs basic computer vision tasks. I largely followed this guide on building opencv https://www.learnopencv.com/install-opencv-4-on-raspberry-pi/

However, in the guide there are several missing dependencies on a fresh version of raspbian buster. There are also some apparent errors in the CMakeLists.txt file which other users already discovered. After patching these fixes and adding the needed missing dependencies, I now have opencv running on my pi. Here’s my complete script below.

IMPORTANT: You must READ the script, don’t just run it! For example, you should check the version of python you have at the time you run this script. When I ran it i was at python 3.7. Also, feel free to bump to a later version (or master) of opencv.

Oh, also during the numpy step it hung and i was too lazy to look into it. It didn’t seem to affect my ability to use opencv – so i didn’t go back and dig. My bad.

#!/bin/bash

cvVersion="4.1.1"
pythonVersion="3.7"
opencvDirRoot=/home/pi/opencv

sudo apt-get -y purge wolfram-engine
sudo apt-get -y purge libreoffice*
sudo apt-get -y clean
sudo apt-get -y autoremove

mkdir -p $opencvDirRoot
cd $opencvDirRoot

# Clean build directories
rm -rf opencv/build
rm -rf opencv_contrib/build

# Create directory for installation
rm -fr installation
mkdir -p installation
mkdir installation/OpenCV-"$cvVersion"


sudo apt -y update
sudo apt -y upgrade
sudo apt-get -y remove x264 libx264-dev
 
## Install dependencies
sudo apt-get install libblas-dev liblapack-dev
sudo apt-get install libeigen3-dev
sudo apt-get -y install qtbase5-dev qtdeclarative5-dev
sudo apt-get -y install build-essential checkinstall cmake pkg-config yasm
sudo apt-get -y install git gfortran
sudo apt-get -y install libjpeg8-dev libjasper-dev libpng12-dev

 
sudo apt-get -y install libtiff5-dev
 
sudo apt-get -y install libtiff-dev

sudo apt-get -y install libavcodec-dev libavformat-dev libswscale-dev libdc1394-22-dev
sudo apt-get -y install libxine2-dev libv4l-dev
cd /usr/include/linux
sudo ln -s -f ../libv4l1-videodev.h videodev.h
cd $opencvDirRoot

sudo apt-get -y install libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev
sudo apt-get -y install libgtk2.0-dev libtbb-dev qt5-default
sudo apt-get -y install libatlas-base-dev
sudo apt-get -y install libmp3lame-dev libtheora-dev
sudo apt-get -y install libvorbis-dev libxvidcore-dev libx264-dev
sudo apt-get -y install libopencore-amrnb-dev libopencore-amrwb-dev
sudo apt-get -y install libavresample-dev
sudo apt-get -y install x264 v4l-utils
sudo apt-get -y install libmesa-dev
sudo apt-get -y install freeglut3-dev



# Optional dependencies
sudo apt-get -y install libprotobuf-dev protobuf-compiler
sudo apt-get -y install libgoogle-glog-dev libgflags-dev
sudo apt-get -y install libgphoto2-dev libeigen3-dev libhdf5-dev doxygen

sudo apt-get -y install python3-dev python3-pip
sudo -H pip3 install -U pip numpy
sudo apt-get -y install python3-testresources

cd $opencvDirRoot
# Install virtual environment
python3 -m venv OpenCV-"$cvVersion"-py3
echo "# Virtual Environment Wrapper" >> ~/.bashrc
echo "alias workoncv-$cvVersion=\"source $opencvDirRoot/OpenCV-$cvVersion-py3/bin/activate\"" >> ~/.bashrc
source "$opencvDirRoot"/OpenCV-"$cvVersion"-py3/bin/activate
#############

############ For Python 3 ############
# now install python libraries within this virtual environment
sudo sed -i 's/CONF_SWAPSIZE=100/CONF_SWAPSIZE=1024/g' /etc/dphys-swapfile
sudo /etc/init.d/dphys-swapfile stop
sudo /etc/init.d/dphys-swapfile start
pip install numpy dlib
# quit virtual environment
deactivate

git clone https://github.com/opencv/opencv.git
cd opencv
git checkout $cvVersion
cd ..

git clone https://github.com/opencv/opencv_contrib.git
cd opencv_contrib
git checkout $cvVersion
cd ..

cd opencv
mkdir build
cd build


# Eigen/Core to eigen3/Eigen/Core
sed -i s,Eigen/Core,eigen3/Eigen/Core/g ../modules/core/include/opencv2/core/private.hpp

# Add these to  opencv/samples/cpp/CMakeLists.txt 
find_package(OpenGL REQUIRED)
find_package(GLUT REQUIRED)

cmake .. -D CMAKE_BUILD_TYPE=RELEASE \
            -D CMAKE_INSTALL_PREFIX=$opencvDirRoot/installation/OpenCV-"$cvVersion" \
            -D INSTALL_C_EXAMPLES=ON \
            -D INSTALL_PYTHON_EXAMPLES=ON \
            -D WITH_TBB=ON \
            -D WITH_V4L=ON \
            -D OPENCV_PYTHON3_INSTALL_PATH=$opencvDirRoot/OpenCV-$cvVersion-py3/lib/python$pythonVersion/site-packages \
        -D WITH_QT=ON \
        -D WITH_OPENGL=ON \
    -D OPENCV_EXTRA_EXE_LINKER_FLAGS=-latomic \
        -D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules \
        -D BUILD_EXAMPLES=ON


make -j$(nproc)
make install


sudo sed -i 's/CONF_SWAPSIZE=1024/CONF_SWAPSIZE=100/g' /etc/dphys-swapfile
sudo /etc/init.d/dphys-swapfile stop
sudo /etc/init.d/dphys-swapfile start

echo "sudo modprobe bcm2835-v4l2" >> ~/.profile
                                                                                                                                        

Setting up GPG/qtpass/pwgen for windows

This is a predicate for a distributed password management system. At the end of this exercise you have a secure password manager for one device, but with a small step or two the full solution can be implemented.

  1. Download and install GPG; At least install kleopatra, as this will give you a nice GUI for generating you keys.
  2. Generate a new keypair. Open kleopatra, click on file -> new key pair. Select “Create a personal OpenPGP pair.” Fill in your name (first and last) and email. Click on advanced. Select 4096 bits. Ensure you publish your key so you can do other fun things, like send encrypted email.
  3. Download and install qtpass.
  4. Configure qtpass. Click on users. Select the user you generated a key for earlier. Close the dialog. Exit qtpass. Relaunch it. Select config. I recommend the following:
    • On-demand copy to clipboard. Hide after 10 seconds.
    • Check hide password. Check autoclear, after 10 seconds
    • Password length. Try 20 characters. Some sites won’t use all 20, so youll need to shorten it for them, but most sites do let you go long.
    • Use tray icon.
    • Start minimized
    • Click on programs, set the path to gpg: You can browse to it, but it should be “C:/program files (x86)/GnuPG/bin/gpg.exe”.

At this point, it would be good to create a few folders. The nice thing to note is you don’t have to use qtpass for this – it can be a bit squirrley, Just open a command line prompt and cd into “password-store” – you can do “mkdir ” to create a new directory.

Try adding a new password under a folder – it should let you enter the login name, click on “generate password”, and when you save it should show the password under your folder.

Next step is to add git support, and create a secure git repo on a website. Then you’ll basically have a custom, secure, distributed password store!

The Wonder Of Open Source And Web Development

I am not a web developer. Notwithstanding this fact I have been doing web development almost every year for the past two decades. This has helped me see a recurring trend: every year I find that all the things I hated about web development have been replaced by new and exciting things. These exciting things keep their glittery shine for about six months, tarnish, wither, and become hated. I then retire from web development forever (meaning: until the following year.)

Currently I am in the honeymoon phase with React. This is after a brief love affair with Angular. I think my relationship with React will last longer because it is just so much cooler than Angular… (nevermind when my ex says “that’s what you said about me after you ditched jQuery!” Everyone knows its impossible to ditch jQuery)

The other thing of note is how many stinkin’ development tools are born every year. I was just brushing up on open source ecommerce offerings last week and stumbled upon a dozen or so. Widdling this down by filtering out inactive projects (based on github commit graphs), I found a few that seemed like good candidates. Of course this led me to some of the underlying frameworks supporting these products: composer (php – ihk; I loathe anything relating to php), meteor, electrode (from walmart labs of all places!), hapi.js, etc. What are all these heretofore-unknown-but-now-pivitol-groundbreaking frameworks?

It never ceases to amaze me how much time is spent on doing, then re-doing frameworks and toolkits. Could we solve the worlds shortage of developer problem by passing some law to limit needless duplication of efforts in creating frameworks? Maybe if we linked excess framework creation to carbon emissions Bernie sanders could jump in and help.

Perhaps toolkits and frameworks are the only fitting artifacts that a developer can leave behind for posterity. All the application code written is usually for a company, and therefore kept closed source, will likely never see the light of day. What does any good developer do besides create abstractions?

As an aside I also found it amusing to see a similar trend with amazon – no longer just s3 and ec2, they have all kinds of petabyte scale storage (including one offering that involves driving a semi truck to your business to extract up to 100 petabytes of data), lambda, snowball, blah blah, etc. Not open source, but in the same vein of neverending innovation…

Oh Blue Apron!

Blueapron is just plain awesome. Even if you don’t pay for it – the recipes are free! Here are a few of my all time favorites:

  • Roasted Cauliflower Meunière – Oh browned butter and lemon juice (aka Meunière), delicious is thy name! It is hard to describe how well the flavors agree in this dish. Wonderful kale and roasted cauliflower, along with this sauce of all sauces, topped with an egg of all things (it works!) sprinkled with parmesean panko breadcrumbs. A few thoughts: 1) i can’t believe i lived this long without trying Meunière, 2) panko and parmesean belong together, on my plate, and 3) you can substitute sweet potato for the egg (i’ve done it and its delicious).
  • Manhattan-Style Fish Chowder – More surprises here like: 1) a soup with fish?! 2) Lemons make the dish yet again, 3) this actually tastes far fresher and is just as comforting as the best clam chowder you could dream up. Note that along with the other commentators, i agree that you should halve the Old Bay seasoning.
  • Shokichi Squash Ragù & Mafalda Pasta – This is a stunning dish if done right. No ingredient is optional. I once proved this by omitting the celery. It is important to have the right pasta, otherwise the noodles won’t catch enough of that gorgeous, creamy butternut squash sauce (im sure Steve Jobs would describe it thusly). And oh thank heavens for rosemary.
  • Roasted Cauliflower Steaks – I cannot think of a recipe whose name more completely belies the supreme beauty and taste it affords than this. This recipe has all the tastes: lemon zest on the lower farro/arugula layer, mild roasted cauliflower in the middle, and a symphony of nutty hazelnuts, sweet grapes, bitter shallots, creamy browned butter and citrusy lemon on top. If you can find fennel pollen, great – otherwise finely chopped fennels seeds suffice. This dish is beautiful: red grapes, green arugula, white cauliflower, brown farro. Like the best blue apron recipes, the ingredients are all familiar on their own, but I doubt in even geological time scales I would have dared combine them in one dish.

I am no blue apron salesman, but I do love the idea of freedom from tyranny – and blue apron frees you from the oppression of bland and boring food. Additionally it provides an exciting escape from the meat-centric, canned-food based recipes of your childhood. Go blueapron!

Random Opinions of MATLAB

While working on my Masters at BYU I had a professor that did all his demonstrations in MATLAB. It was as if nobody had told him that they had invented other languages. Or he just really loved MATLAB.

Lately I’ve been doing a project where I’ve had to experience the joy of using MATLAB. As a numerical processing suite it definitely effective: concise, quick, easy. It is also quirky. I thought I’d log a few observations for posterity.

First, the editor has some nice things: autocomplete, lots of suggestions for writing more idiomatic MATLAB. For example:

  • number conversion – i was tempted to use str2num, which does work, but MATLAB suggested str2double, which also works but is mo’ betta
  • string finding – Of course there is a findstr and strfind… you guess which one MATLAB prefers. The MATLAB will correct you. (hint: its strfind)
  • printing – My very first transgression was trying to log statements using disp(sprintf(‘stuff: %s))… For some reason i found sprintf but didn’t yet know that disp(sprintf()) is analogous to fprintf()… but MATLAB wasn’t afraid to correct me

Matlab has a ton of quirks:

  • The editor is SOOO WEIRD. Examples: weird key combos for copy and paste… seriously, Alt+2?? Where are my vim bindings?
  • Performance of deeply nested classes. Deeply meaning depth > 1. I wrote a file reader class. The reader had a data member that allocated a large array. Performance went in the gutter. Then I moved the array to the same level as the file reader, keeping all logic the same, and performance was better. Why oh why would it matter. I don’t know, but it does.
  • Weird behavior when return value not specified. I realized c++ does this. If you don’t specify a return on a non-void function, bad things can happen. I’d expect more from a (presumably) high-level language like MATLAB.
  • paths. I think the notion of having code paths is weird. Why not just have a project with all the paths set up?

Except for the editor, which is appalling, I actually find MATLAB quite usable. I wouldn’t claim it is superior to python, just different.

How to get an ovpn file onto iOS device via HTTP

Recently I wanted to get a VPN connection from a phone to an openvpn server. The openvpn app only provides two or three options for getting your ovpn file: 1) use itunes, 2) go to a website in safari that has the .ovpn file, or 3) send via mail.

Option 1 was out since the phone was locked down enough to not permit me to transfer apps. Not sure why – likely just a policy of the phone (i dont own the phone).

Option 3 is less secure so I skipped that.

Option 2 is easy – I only had to make a few mods off what I found via Mr. Google:

  • Make the .ovpn file have a unified format. Some places on the internet suggested you could have configuraiton lines that looked like the following:
    ca [inline]
    cert [inline]
    key [inline]
    

    However this doesn’t actually work. Instead, just delete those lines and stuff the ca, cert, key, and tls keys under the appropriate tags (they have the same name as the config lines)

  • Ensure your web server has the right mime type. The .ovpn mime type is “application/x-openvpn-profile”, and can be added to your httpd.conf like this:


    <IfModule mime_module>
    ...
    AddType application/x-openvpn-profile .ovpn
    ...

I then pointed my browser at the .ovpn file, at which point safari recognizes that you can open it in openvpn. All worked marvellously thereafter.