DIY Cell Phone

GSM Phone Tracking Methods

Leave a comment

I decided to conduct a few tests with my fona808.

  • Battery. The at+cbc command give you the current charge mode, percent charged, and millivolts. I found on the adafruit fona808 that the charge mode indicator always gave status 0 (“not charging”) even when charged.
  • GSM Location. This is given by the at+cipgsmloc. It spits out lat and lon. I don’t know where it is getting the data from, but based on the plots I made the location it gives is an estimate based on your location to the nearest cell tower.
  • GPS. Given by at+cgnsinf. I found this to be spot-on, always accurate, even with my pea-sized GPS antenna!
  • DIY cell tower triangulation. If you put the fona into ENG mode (“AT+CENG=3”), it will give you the MNC, MCC, LAC, and CellId for the towers around you (i usually got six reports per “AT+CENG?” query). You can then use a site like cellphonetrackers.org to turn the tower info into a lat/lon coordinate. The +CENG messages also given a power level which you can use, in cojunction with the cell tower lat/lon coordinates, to perform the triangulation. The method of doing this is described elsewhere, but basically each power level becomes a weight, w_i = rx_i / (rx_0+rx_1…rx_n), which is multiplied by the lat/lon of the corresponding cell tower item. You then just add up the weighted lat/lons, and viola!

      • My results?

      • I found that i lost about 10% power on my 1 hour trip. This seems terrible – barely 10 hours per charge, extrapolating.
      • The GPS info was perfect. It took about 2 min to acquire
      • The GSM location was too course, but was actually more clean than my DIY location
      • DIY location seems crumby. Im not sure what would fix this. Possible things to look at: 1) Filtering. Maybe I could throw out the lowest power rating, or smooth out the locations somehow. 2) See if there are better cell tower dbs. As far as i know, cell tower info isnt public, so any db is most likely based on reported, possibly inaccurate, values

      Here’s a map with plots of the three localization methods: GPS (magenta), GSM (green), and DIY Triangulation (yellow).

      gsmtrack

DIY Cell Phone

My favorite FONA commands

Leave a comment

These are more like “the commands i have found useful as of present.” I bear no real affinity for them, except that I do appreciate the data they yield.

AT+COPS? Ensure you are connected to the network (it gives “+COPS: 0” if you are not)
at+ccid Get the SIM number; you need this for activation
AT+CMGF=1 This sets us into text mode. I haven’t used the other mode (PDU) yet.
at+sapbr=3,1,”contype”,”gprs” You can set the connection type to GPRS (data) or CSD (circuit switched) – i think this is why you can call or do data on gsm networks, but not both (at the same time).
at+sapbr=3,1,”apn”,”wholesale” Until you set your access point name, you might not be able to do things like geolocate (based on cell towers) or do data stuff. TING’s APN is “wholesale” – im sure it is different for every provider.
at+sapbr=1,1 Open up your bearer. . . sounds good, but im not entirely sure what that means.
at+cipgsmloc=1,1 This gives you your lat/lon. Note that on my old T-Mobile sim card, which had no data plan, i got nothing back for some reason; apparently you have to have a data plan to get this info?

For SMS (which i used only briefly) I found the following useful:

at+cmgl=”all” I didn’t realize this, but all text messages really are stored somewhere in the providers network, at least until you do something with them (makes sense) – i just found it interesting that I transplated my SIM card from my tmobile device to my FONA, and could see text messages from years ago.
at+cmgs=”180188xyzwl” This is of course how you send a text. You press enter, after typing the phone number (“1801..” – notice the leading “1”, since im in the USA; not sure if this is needed, but it works with it). When you are done you must hit ; if you hit it cancels the message!
at+cmgr= This is your way of reading a text message

For GPS I used the following (note I have v2 of the FONA808 – the commands are different for v1)

AT+CGNSPWR=1 The device starts with the GPS off. So i turn it on, cause i want it.
AT+CGNSINF Gives you a crudload of gps-related info, including lat-lon, altitude, utc, etc. (see table 2-2 in the “SIM800 GNSS Application Note”)
DIY Cell Phone

DIY Phone

Leave a comment

I was recently granted an adafruit shopping spree by my wife. I chose a FONA808 and some Trinket microcontrollers (literally they are about the size of my thumbnail!).

Im still learning the ins and outs of the FONA, but it is amazing how much momentum you can gain by simply following the instructions listed in the adafruit tutorial. Since I spent a couple hours due to my failure to read I thought I would pass on for some other soul, like myself, the importance of simply doing what lady ada says. Namely: 1) DO connect your power source (battery, usb +5, whatever) to VIO, and 2) if you don’t see the blue light shining hold down the little button on the module for two seconds (or just tie KEY to ground). These two things will work wonders.

Oh, and for those using OS X, I found much success by using the USB serial cable available on adafruit. I just downloaded the drivers from the product page, downloaded coolterm, and away i went. The autobaud on the fona works marvellously, as witnessed by it echoing back after I typed “at” a few times.

Backups

If you had to do DVD backups from windows

2 Comments

Say you want to let your grandmother backup her DVDs using her clunky old windows laptop. You want a windows solution that 1) rips the DVDs, 2) copies them to a backup, 3) makes them available for viewing from wherever (e.g. accessible over the internet).

The solution I came up with is as follows:

  • Download cygwin and install rsync. I hadn’t used cygwin for a long time, but basically you download either the 32 or 64 bit version here, then filter for rsync and select it. Cygwin seems to do a good job of only pulling what is necessary.
  • Download HandbrakeCLI. You dont need the full Handbrake installer, just the CLI. You should be able to google for it, but i found it here
  • libdvdcss. This is all over the internet. Just google and ye shall find. You need this to be able to decrypt most DVDs. Just place the DVD dll inside your handbrake folder to enable decss capability.
  • Download VLC. I felt like ripping my DVDs with chapter markers. The best format for supporting this kind of thing seems to be matroska (mkv). VLC natively supports mkv, which is handy if youre viewing these DVDs locally

I then automated the whole thing with the following script:

@echo off
SETLOCAL ENABLEDELAYEDEXPANSION
set /p dvdname= Enter DVD Name (no spaces):
set basepath=<localScratchDir>%dvdname%.mkv
set remotedir=<remoteBackupPathDir>/%dvdname%.mkv
FOR /F "tokens=* USEBACKQ" %%F IN (`c:\cygwin64\bin\cygpath %basepath%`) DO (
SET basepathunix=%%F
)
echo Ripping to %basepath%
HandBrakeCLI --main-feature -m -i d: -o %basepath% -e x264 -q 20 -B 160
echo Copying %basepathunix% to backup
c:\cygwin64\bin\rsync.exe -e "/usr/bin/ssh -p <remotePort>" -auvhP %basepathunix%  <remoteUser>@<remoteSite>:%remotedir%
echo.
echo.

if "%errorlevel%"=="0" (
        echo Completed successfully!
)
if not "%errorlevel%"=="0" (
        echo DVD backup did NOT complete successfully!
)
echo Removing %basepath%
del %basepath%

set /p dummy=Press enter to finish

Note some assumptions about this script:

  • You are running 64bit cygwin. If not change from c:\cygwin64
  • You want to push to somewhere offsite. You can just skip the rsync step if you want to go local.
  • Note that I used a custom port in this example; you can remove the -e switch and its argument if you’re using a standard port.
  • This example assumes you have set up ssh keys properly. Meaning you ssh-keygen on your windows box (from your cygwin shell!) then copy your key to the dest machine (perhaps using ssh-copy-id). I don’t allow password authentication, so somtimes its easiest to just ssh into the remote host somewhere that has access, then teamviewer/rdesktop/whatever into your windows box to copy the ~/.ssh/id_rsa.pub file into the ~/.ssh/authorized_keys file.
  • I don’t care about extra content, so im only ripping the main feature.

As for legally accessing your DVDs remotely: you can use one of any methods. I have found PLEX to work well.

Frequent Flyer

Airlines / Frequent Flyer Programs: The good, the so-so, and the ugly

Leave a comment

An earlier blog post detailed heroic feats of airline reward redemption: leaping across multiple rewards programs, bounding over unaccompanied minor restrictions, and otherwise gaining cross-country access for peanuts!

And then everything was shattered when I had to change my tickets.  This let me see how well each of the rewards programs faired in handling booking changes.  This leads me to my list of good, so-so, and ugly:

Let’s start with the ugly: Delta.  I think its kinda neat that I booked $70 flights from LAX to SLC.  Then when I wanted to change the flights they were willing to do so for $200 per ticket.  I’d rather pay a bit more for Southwest and get better customer service.

Now the so-so: american airlines earns my so-so category.  They didnt really provide any way to change my existing flight, much like delta. However they have otherwise great service, and some of the best reward travel coverage available; I can always find an AA flight, even through avios.

Which brings me to my winners, the good guys: Southwest and Avios!  I must first say southwest is my all around favorite: TWO free checked bags, NO fees when changing flights.  Awesome.

And Avios deserves a good apology from this author.  I was able to use only 13500 points to book a one-way flight for three of my family members from LAX to SLC.  That same thing would have taken over 30k points on something like southwest or aa.  It turns out that if you aren’t bounding from coast to coast avios is a powerful tool to have in your points chest.  Its fantastic for those short-haul trips.

There ye have it.

Uncategorized

A Tasty Apple Pie Recipe

Leave a comment

Having a bushel or two of apples on hand, I decided to make an apple pie the whole family could enjoy.  I borrowed two recipes.  For posterity i have included the actual recipe in addition to the links.

For the apple pie guts I used Apple Pie by Grandma Ople:

1 recipe pastry for a 9 inch double crust pie
1/2 cup unsalted butter
3 tablespoons all-purpose flour
1/4 cup water

1/2 cup white sugar
1/2 cup packed brown sugar
8 Granny Smith apples – peeled, cored and sliced
Directions
Preheat oven to 425 degrees F (220 degrees C). Melt the butter in a saucepan. Stir in flour to form a paste. Add water, white sugar and brown sugar, and bring to a boil. Reduce temperature and let simmer.
Place the bottom crust in your pan. Fill with apples, mounded slightly. Cover with a lattice work crust. Gently pour the sugar and butter liquid over the crust. Pour slowly so that it does not run off.
Bake 15 minutes in the preheated oven. Reduce the temperature to 350 degrees F (175 degrees C). Continue baking for 35 to 45 minutes, until apples are soft.

and for the crust I used French Pastry Recipe:

Ingredients
3 cups all-purpose flour
1 1/2 teaspoons salt
3 tablespoons white sugar
1 cup shortening

1 egg
1 teaspoon distilled white vinegar
5 tablespoons water
Directions
In a large mixing bowl, combine flour, salt, and sugar. Mix well, then cut in shortening until mixture resembles coarse meal.
In a small bowl, combine egg, vinegar, and 4 tablespoons of water. Whisk together, then add gradually to flour mixture, stirring with a fork. Mix until dough forms a ball. Add one more tablespoon of water if necessary.
Allow dough to rest in refrigerator 10 minutes before rolling out.

The results we’re fantabulous:

P1060846

Computer Vision

A few ways to quickly and automatically binarize an image

Leave a comment

For my wife’s Spell To Write and Read (SWR) homeschooling we have a bunch of scanned worksheets.  A sample of the scanned image is shown below:

10letter_aandc

As you can see its entirely readable and fine for our purposes.  However it is not gentle on our laser printer toner budget.  What we really want is the background to be white, and the foreground to be black – nothing inbetween. This process is called binarization – and scanner software often has a feature that lets you do this during scantime.

We didn’t use that feature (or maybe our software didnt support it) at scantime. As such we need to resort to postprocessing. I have a Master’s in computer graphics and vision, and everytime I use something I learned the value of that degree goes up. It therefore behoves me to use it every chance I get.

As a good computer vision student, when I think binarization my mind jumps straight to Otsu!  He came up with a great way of automatically determining a good threshold value (meaning, when we look at each pixel in the image, everything below a value turns black, all else turns white).

My first thought is to check for an easy button somewhere. In gimp, for example, I found you can load the image, click on “Image -> Mode -> Indexed” then select “Use black and white (1 bit)”. Looks ok!

Now how to automate this, given I have 60+ images? Turns out there is a threshold option in imagemagick. I could go through each image in the directory and manually threshold, but I might get the threshold wrong, and I don’t really want to train my wife on picking a threshold value. Plus I know Otsu is better!

Turns out some guy named Fred has a bunch of ImageMagick scripts, including an Otsu one. I downloaded his script and ran it, yielding the following image:

10letter_aandc

Pretty nice – just black and white.  Thanks Fred… sorry I cannot call him “Fast Freddy” since it took around 18 seconds per image.  I know we can do better! Time to dust off those computer vision skills of Master.

Here’s what I came up with using python/opencv:


#!/usr/bin/python
import cv2
import sys
img=cv2.imread(sys.argv[1],0)
ret,imgThresh=cv2.threshold(img, 0, 255, cv2.THRESH_OTSU)
cv2.imwrite(sys.argv[2], imgThresh)

Short and sweet! And performance is way better: about 3 seconds per image.  But it looks like most of the program runtime is spent loading cv2.  Based on that assumption I decided to add a bulk processing mode:

import cv2
import sys
if len(sys.argv) == 1 or "-h" in sys.argv:
    print "Usage: %s [-inplace] image1 [image2 [image 3 ...]]"
    print " %s inImage outImage"
    sys.exit(0)
if "-inplace" == sys.argv[1]:
    inOut = [ (arg, arg) for arg in sys.argv[2:] ]
else:
    inOut = [ (sys.argv[1], sys.argv[2]) ]
for inImage, outImage in inOut:
    print "Converting %s to %s" % (inImage, outImage)
    img=cv2.imread(inImage,0)
    ret,imgThresh=cv2.threshold(img, 0, 255, cv2.THRESH_OTSU)
    cv2.imwrite(outImage, imgThresh)

When I run this script on the whole directory it takes an average of 2 seconds per image. Better, but longer than needed. What gives? It turns out I have all my data on a QNAP and opening, reading, and writing lots of files is not its forte. When I copy the data to my local SSD on the MAC, the cost per image is now 140ms. Much better.

Since, as often happens, I have found my assumptions totally flawed, can I vindicate Freddy? After rerunning the test it appears he is still a “steady Freddy” at about 2.7 seconds when running straight on the hard drive. Sorry Fred; opencv just beat the pants off you.

Uncategorized

Using raspberry pi for two-way video/audio streaming

2 Comments

I am currently writing custom software to create a variety of distributed media solutions.  One of these involves a raspberry-pi security camera.  Right now my software isn’t finished, but I wanted to prove the concept out using existing software.  System requirements:

Raspberry pi:

  • Broadcast audio (over the LAN) to a server
  • Broadcast video (over the LAN) to a server
  • Receive (and play over the speaker) audio

Laptop:

  • Receive audio/video from the raspberry pi (and display it)
  • Send audio (from the laptop’s own microphone) to the pi.

I bought a raspberry pi b+, cheapo USB microphone, and IR camera (with built-in LEDs), and some speakers.  The whole package was around $100, which is just plain awesome.  Im sure my grandkids will marvel, 40 years down the road, that for the cost of an Obama-approved school lunch, I could buy so much.

Here’s a schematic of the setup:

design

This can be accomplished with a few scripts, which i based heavily on those found here: http://blog.tkjelectronics.dk/2013/06/how-to-stream-video-and-audio-from-a-raspberry-pi-with-no-latency/

Run the following on the pi:

function on_signal (){
    launch "kill -9 $piVideoPid $piAudioPid $osxAudioPid"
}
trap 'on_signal' EXIT

# Receive os x audio
gst-launch-1.0 -v udpsrc port=9102 caps=\"application/x-rtp\" ! queue ! rtppcmudepay ! mulawdec ! audioconvert ! alsasink device=plughw:ALSA sync=false &amp;
osxAudioPid=$!

# Send pi audio
gst-launch-1.0 -v alsasrc device=plughw:Device  ! mulawenc ! rtppcmupay ! udpsink host=$broadcastIp port=$piAudioPort &amp;
piAudioPid=$!

# Send pi video
raspivid -t 999999 -w 1080 -h 720 -fps 25 -hf -b 2000000 -o - |  gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96  ! gdppay ! tcpserversink host=$localIp port=$piVideoPort

Run the following on the laptop (a macbook in my case).  Note if this were a linux box you would probably use “autoaudiosrc”.

function on_signal (){
    launch "kill -9 $piVideoPid $piAudioPid $osxStreamPid"
}
trap 'on_signal' EXIT

piIp=&lt;raspberryPiIp&gt;

# Stream OSX microphone (note the capsfilter, apparently needed due to os x bug)
launch "gst-launch-1.0 -v osxaudiosrc ! capsfilter caps=audio/x-raw,rate=44100 ! audioconvert ! audioresample ! mulawenc ! rtppcmupay ! udpsink host=$broadcastIp port=$macAudioPort" "ASYNC"
osxStreamPid=$!

# Receive audio
launch "gst-launch-1.0 -v udpsrc port=$piAudioPort caps="application/x-rtp" ! queue ! rtppcmudepay ! mulawdec ! audioconvert ! osxaudiosink" "ASYNC"
piAudioPid=$!

# Receive video
launch "gst-launch-1.0 -v tcpclientsrc host=$piIp  port=$piVideoPort ! gdpdepay ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false" "ASYNC"
piVideoPid=$!

echo "Waiting on $piVideoPid $piAudioPid $osxStreamPid"
wait

You should be able to see and talk to whoever is on the other end of the pi. Biggest problem is audio feedback, if the laptop and pi are close to one another.  I havent even begin to think about this, but the coolest things to note are:

  • You get a silky-smooth h264 video
  • The audio is acceptable
  • It works almost entirely using gstreamer.  The cool thing about that is gtreamer has a very easy programmatic API.  So… that makes my job easy when I get to that part of my project.

Enjoy!

Backups

FenBackup Through the Years

Leave a comment

This is more of a tour of the past six years or so of various iterations of Fenimore backup solutions.

 

  • Solution 1.  Soon after I got married I found that between my wife and I we had around 1TB of movies, images, and documents from school.  We were also accruing around 5-10GB a week on family videos and pictures (hey, 1080p video cameras are readily available, and generate fairly big, hi-res videos).   Around Oct 2010 the price of 1.5TB drives was pretty low – around $80 each (this was right before the huge price spike a year or so later due to some disaster in the Pacific).  So I built a 3TB software-RAID 6 array with linux and a large tower.  I also had two 3TB RAID 5 arrays that plugged into hot-swap bays.  I periodically rotated these arrays to an offsite location.  By “periodically,” I mean it was every week or so for a month, then exponentially decayed to about once or twice a year towards the end.  backup_schemes
  • Solution 2. Enter the QNAP.  By this point I had upgraded to 3TB drives in my consolidated linux solution.  I took those drives out of my linux box and moved them into a 4-bay QNAP (TS-453 Pro).  I found that all of these 3TB WD Green drives started dying.  As I write this I am actually not sure what the heck happened, but soon I found myself buying some 5TB WD RED (NAS-grade) drives to replace the WD Greens.  These have worked much better.  This solution has also been nice because anytime I need to do work on my linux box or on the NAS it doesn’t take down my whole network.  It was also nice because I ended up making all my files accessible (in theory) via QNAPs cloud software.  The only flaw is that I tried to continue my offsite backup scheme wherein a drive was cycled to an offsite location; this actually was worse with QNAP, because QNAP doesn’t like the drives to be removed. backup_schemes2
  • Solution 3: Redundant QNAPs over VPN.  This is basically an upgrade to my QNAP solution where I bought a cheaper 2-bay QNAP.  The second QNAP fulfills my offsite backup goals. It sits on the other side of the country and is available 24/7 for my main qnap to push a copy of my data to it.backup_schemes3

Overall I’d say this solution is just about flawless.  It at least seems secure, works well, etc.  I found, for example, I could get to my files when I am not home.  I was even able to stream video from my qnap via PLEX to my laptop in my hotel; this turned out to be useful, for example, when my daughter wanted to watch tangled, which worked fairly well.

 

 

Encryption

Key-based OpenVPN for OSX/Linux/QNAP Howto

Leave a comment

I’ve had excellent luck setting up a key-based authentication VPN for my home network, which is distributed across the country and consists of Linux, Mac (OS X), and QNAP nodes.  What, you ask, does “key-based authentication” refer to?  It means that I do not allow people to enter passwords.  Instead, clients must possess the proper key, which is exchanged automatically by openvpn.   I think key-based authentication can be more secure in some cases, however I mainly wanted to do it “just because.”

Generally speaking the OpenVPN HOWTO is excellent and should suffice for setting up your VPN.  Ill briefly parrot this guide here.  To set things up you:

  • Pick a vpn network.  E.g. something that won’t conflict with any of the LAN networks of the nodes on your distributed network.  For example, one of my nodes sits on a 192.168.x.x/16 network, another sits on a 10.246.75.x/24 network, etc.  Pretty much anything, other than 10.246.75.x/24, from the 10.x.x.x/8 space can be used.
  • Generate keys:
    • From /etc/openvpn/easy-rsa run . ./vars
    • ./clean-all
    • ./build-ca  – basically accept everything as is, but remember the common-name for the server (as <servername>)
    • ./build-key-server <servername>
    • For each of my clients (i have 5 or 6): ./build-key <clientName>
    • Build the diffie-hellman stuff: ./build-dh
  • I chose to enable client-client communication, so I added the following in my server config:
    • client-to-client (to let clients talk to eachother)
    • push “redirect-gateway def1” (to redirect all traffic through my vpn)
    • client-config-dir /etc/openvpn/clients  (to let me specify static ips for all my clients).  For each client (identified by the common name, during the setup above, I create a like-named file with a single line like this:
      • ifconfig-push 10.9.1.22 10.9.1.23 (meaning this client will be accessible within the vpn via the 10.9.1.22 address)

On each client I scp’d over the ca, crt, key files, and made up a quick config script (based on the sample provided with openvpn).  I really only changed the following lines:

  • remote <hostname> <port> (this line has the hostname/port of your server)
  • ca /abs/path/to/vpn/directory/ca.crt
  • cert /abs/path/to/vpn/directory/fenimac.crt
  • key /abs/path/to/vpn/directory/fenimac.key

With that both the server and client should connect; all connected clients should have statically-assigned ips, and they should be able to talk to eachother.

What makes this a bit tricky is QNAP.  They only provide password-based authentication.  Thankfully you can just make a crontab to create the vpn connection, if its not up.  I made a script that looks something like:

VPNLOG=/mnt/HDA_ROOT/vpndir/vpn.log
if [[ `ps ax | grep openvpn | grep vpndir | grep -v grep` ]]; then
    echo "`date` Already running" | tee -a $VPNLOG
    ping -c 1 &lt;server'sVPNIp&gt;
    PINGRET=$?
    if [[ "0" = "$PINGRET" ]]; then
        echo "`date` Link is good!" | tee -a $VPNLOG
        exit 0
    else
        echo "`date` Link is down (ping gave $PINGRET) - killing current vpn" | tee -a $VPNLOG
        ps ax | grep vpndir | tr -s " " | cut -f 1 -d ' ' | xargs kill -9
    fi
fi
echo "`date` Run this thing!" | tee -a $VPNLOG
/usr/sbin/openvpn  --config /mnt/HDA_ROOT/vpndir/qnap.conf | tee -a $VPNLOG

A few important notes:

  • I run this script on the QNAP every couple minutes in a crontab.  Just editing crontab (“crontab -e”) doesn’t do the trick – qnap wipes those entries away upon reboot. To make this work, even across reboots, I followed a guide i found somewhere on the internet.  Basically you do the following: 
    1. Edit /etc/config/crontab and add your custom entry.
2. Run 'crontab /etc/config/crontab' to load the changes.
3. Restart cron, i.e. '/etc/init.d/crond.sh restart'
  • I couldn’t get the vpn to stay up unless I turned on the vpn server inside the qnap.  It wasn’t my favorite thing to just leave that on, but for now im ok with it.