Gogo and SSL Certificates Part 2

Back on another Delta flight, with Gogo wireless, and it looks like they've changed their tune a bit. Now things work fine.

Gogo and SSL Certificates

In a funny bit of coincidence, I'm currently on a Delta flight with Gogo wireless as I see this story about Gogo and SSL certificates break. The allegation is that Gogo is intercepting SSL traffic, decrypting, doing who-knows-what, then re-encrypting before delivering it to you. It's a pretty common practice in most companies where they want to monitor employee behavior, but it typically requires that someone installs a special root certificate on your machine. Otherwise, most modern browsers will throw all kinds of warnings and blocks about the site being untrusted.

So, I loaded up "SSL Detective" (a free iOS app) on my iPhone and went to work. My findings are interesting, but a bit weird.

Basically, all the usual stuff (Twitter, Facebook, Google, Gmail) was just fine. But video sites (YouTube, Vimeo, Crackle) all show up signed by this one strange cert.

It's a pretty strange thing to be analyzing, as there's probably no personal data going there (at least in comparison to decrypting Facebook or gmail). My guess is that it's related to their desire to block streaming video sites.

From what I know, most Flash and JavaScript will die if they attempt to pull data and the certificate doesn't check out. So this is a mostly effective, albeit odd, way to block it. I don't know why they don't just rewrite the DNS entries somewhere with a warning banner ("Don't do this, be kind to your network using neighbors") or just block the IP.

But I thought it interesting enough to share my findings.

Plotting Data in GDB

Using GDB to debug programs is great and all, but if you work with Scientific data then one really critical function is the ability to plot a long array or vector, to see how the solution is evolving at various stages.

After a bit of tinkering, I was able to take this code snippet and modify it for my needs. It had just a few problems:
The syntax they used for labeling things fell apart when I was using Eigen vectors, the -> became shell redirects.
It didn’t link very long (100+) entry arrays.

So I modified it like so:

# plot1d.gdb

# Copyright (C) 2008 Florian Lorenzen
# - Taken from https://sourceware.org/gdb/wiki/PlottingFromGDB
# Modified by Randall Hand to actually make it work.
# original version seemed to badly handle variable names
# like esf->data[0]@1024 (turning the > into a pipe redirect)
# Plot an expression that expands to {x1, x2, ..., xN}, i. e.
# N numbers using gnuplot.
# This file is for the GNU debugger 6.x.
# It writes temporary files named __plot1d.dump, __plot1d.dat, __plot1d.gp, so
# you should not have files of the same name in the working directory.
# It requires sed, awk, and gnuplot available in the $PATH.

# plot1d_opt_range   
# Plot the points of  passing  as plot options using
#  in the set yrange command.
define plot1d_opt_range
shell rm -f /tmp/__plot1d.dump /tmp/__plot1d.dat /tmp/__plot1d.gp
set logging file /tmp/__plot1d.dump
set logging on
set logging redirect on
set height 0
set print elements unlimited
output $arg0
set logging off
set logging redirect off
shell awk '{printf("%s", $0)}' < /tmp/__plot1d.dump | \
sed 's/^{\(.*\)}$/\1/;s/, */\n/g' > /tmp/__plot1d.dat
shell echo 'plot "/tmp/__plot1d.dat" title "$arg0"; pause -1 "Press enter to continue"' > /tmp/__plot1d.gp
shell gnuplot /tmp/__plot1d.gp
# shell rm -f /tmp/__plot1d.dump /tmp/__plot1d.dat /tmp/__plot1d.gp

# plot1d 
# Just plot the points of .
define plot1d
plot1d_opt_range $arg0 "" "[*:*]"

# plot1d_opt  
# Plot the points of  passing  to the
# plot command after the datafile. So, one can pass
# "with lines" here.
define plot1d_opt
plot1d_opt_range $arg0 $arg1 "[*:*]"


With this file placed somewhere and executed, you can then simply do plot1d data[0]@128 to get a nice plot of the first 128 values of data.

And if you like it so much you want to keep it, simply add source plot1d.gdb (which a path to wherever you put it) to your ~/.gdbinit

The Cloud Fumbles

For the last several years, "ransomware" has been one of the bigger problems on the internet. Half computer virus, half phishing attempt, it's the frustrating encryption or theft or personal data that can only be recovered by paying the attacker some nominal sum to get things restored.. It's incredibly annoying, but good antivirus and security measures can generally protect people. Personally, I've never dealt with it but it seems to pop up in the news every few months.
However, now it seems hackers have taken Ransomware to a whole new level. Earlier today, SVN hosting webservice "Code Spaces" was shut down when an attacker demanded money for return of their Amazon EC2 control panel, and when they tried to stop him (by doing exactly what I would have done: Changing passwords and removing access) he irreversibly erased all of their custom data and repos! Similarly the same thing almost happened to Feedly but they were more successful in averting disaster. Evernote got hit with a DDOS as well, but (as far as they are reporting) there was no extortion attempt.
Whether or not these incidents are related, it's a scary time.. For the last several years, lots of people have migrated data to the cloud. Documents and communications exist in Google services, backups exist in Carbonite & Crashplan, personal memories exist in Flickr and Instagram, bits of data exist in services like GitHub and Evernote. When suddenly the cloud is no safer than your personal computer, maybe it's time to start moving more towards "personal clouds" ?

Neat Bash Tricks

I’ve been writing lots of bash scripts lately, so I thought it would be a good idea to document a few of the little tricks I’ve been using.

First off, to make a script that has to be executed in bash:


if [ ! "$BASH_VERSION" ] ; then
  echo "Please do not use sh to run this script ($0), just execute it directly" 1>&amp;2
  exit 1

This was needed for a script that makes uses of Bash syntax for for loop and some basic math, and I had a user reporting errors when they tried to do sh script.sh. sh ignores the opening #!/bin/bash line and tries to execute the script directly, and fails when it hits those lines.

Then, to make a script that has to be sourced in the global environment (to set environment variables like PATH and LD_LIBRARY_PATH for the user).


if [[ "$(basename "$0")" == "script.sh" ]]; then
  echo "ERROR: Don't run $0, source it" >&amp;2
  exit 1

replace script.sh with the name of the actual file.

Then, to URL Encode a parameter in bash (I used this to encode a user provided git password into the url), use this function:

rawurlencode() {

  local string="${1}"
  local strlen=${#string}
  local encoded=""

  for (( pos=0 ; pos
    case "$c" in
      [-_.~a-zA-Z0-9] ) o="${c}" ;;
      * )               printf -v o '%%%02x' "'$c"
  echo "${encoded}"    # You can either set a return variable (FASTER)
  REPLY="${encoded}"   #+or echo the result (EASIER)... or both... :p

Then you can use it like so:

echo Enter your GitHub username:


echo Enter your GitHub Password:

GIT_AUTH=$( rawurlencode "${GIT_USER}" ):$( rawurlencode "${GIT_PASSWORD}" )

git clone http://${GIT_AUTH}@github.com/user/repo

yeah yeah, I know.. SSH keys are better… tell it to the users, not me

Finally, using getopt. It’s a big tool, but the basics of using it:


while getopts "aj:s" o; do
    case "${o}" in
shift $((OPTIND-1))

Important bits:

  • the getopts line: Include each character recognized as a flag, and following it with a : if it requires an argument.
  • Then include a case for each one in the big switch.. use $(OPTARG) to get the argument, if it’s needed.

Makes argument parsing simple in bash scripts simple… Also define a usage function to spew out the supported options, and you’re golden.

Sublime Text, Xeno, and Build Systems

So I'm still loving the use of xeno.io for editing files on remote machines. The only problem I've had so far, and it's not really a problem as much as an annoyance, is that I can't use Sublime Text's built-in Build features. I'ld not really noticed how much I missed it until I wound up working back on a machine locally and I could hammer F7 to build, then F4 to jump to the first error. Install the Build Next plugin and it really becomes a nice way to do development for C/C++ (Still missing the fabulous SublimeCLang pluin tho.)

So, I spent a bit of time and build a custom build-system that works with it. It works with a shell script (below) to find the remote host and directory that the code really resides in, then ssh'es over to run the build. Then it takes any filenames in the output (for error messages or warnings) and converts them back to the local path equivalents, so that F4 works for taking you to the source of errors.

It works pretty well. I also got it to force a xeno sync prior to a build, eliminating one of my major annoyances. It works with either make or ninja (another favorite tool of mine), and works amazingly well.

To get this to work, first you'll need the following shell script, which I call xeno-build.sh:

if dirname $2 | grep .xeno
    echo This looks like a Xeno project.
    SOURCEDIR=`dirname $2`
    SYNCID=`git config xeno.syncprocessid`
    echo Syncing...
    /usr/local/bin/xeno-sync $SYNCID
    HOST=`git remote -v | grep origin | grep push | cut -d ' ' -f 1 | cut -f 2 | cut -d '/' -f 3`
    RDIR=`git config xeno.remotepath`
    BUILDDIR=`dirname $RDIR`/build
    echo Host = $HOST
    echo ssh $HOST "cd $BUILDDIR &amp;&amp; $1" \| sed s=$RDIR=$SOURCEDIR=g
    ssh $HOST "cd $BUILDDIR &amp;&amp; env TERM=screen256color &amp;&amp; $1" | /usr/local/opt/gnu-sed/libexec/gnubin/sed -u s=$RDIR=$SOURCEDIR=g

You might need to edit the BUILDDIR line to match how you do builds . I use CMake and do out-of-source builds, so I always have a root/src and root/build directory.

Then, in your Sublime Text 3 Packages folder, you'll want a build systems file like this:

            "make -j9",
    "file_regex": "^(..[^:]*):([0-9]+):?([0-9]+)?:? (.*)$",
            "name": "Make Tests",
                "make test",
            "name": "Make -j4",
                "make -j4",
            "name": "Make Single",
            "name": "Clean",
                "make clean",

Simply replace "make" with "ninja" here and you can do that too. The basic usage of the script is xeno-build.sh command sourcefile. The command can be anything really (make, ninja, rm, or whatever build system you like), and the sourcefile is any file in the source directory (that really doesn't even have to exist).

So, hopefully someone out there will find this useful. I've made it a standard part of my builds.

A Better Way to fix OSX Calendar & Google Hangouts

So, ever since my post a while back when I was working on OSX Calendar and Google Hangout, I’ve been annoyed by the constant need to drag appointments onto the little automator robot each time. I grew to loathe that little robot, sucking precious seconds of my time prior to each meeting.. The, taking almost 2 minutes every time to launch automator, process my calendar, and then update it. The end result was that I went back to Google Calendar.

So, I’ve spent some time over the last few weekends and came up with a much better options. The result is available on github as CalendarHangout. Setup and run the script, and see every calendar event in Calendar suddenly have a nice little URL link at the bottom that you can click on to launch hangout.

Simply download (or clone) that, and edit the script with the name of your calendar in Google (typically your email address). Then run it once by hand, and go through the steps to authenticate it (GData3 API requires OAuth authentication). Then, you can toss it into a cronjob or just run it on occasion to fix everything 2-weeks out!

So what’s it going? It gets a list of all appointments on the 2-week horizon from Google, then uses AppleScript to update each one with the Hangout Link. Due to the obnoxious nature of both companies involved (Google and Apple), this is far more frustrating than it needs to be:

  • Google doesn’t support the RFC standard “URL” field, which would make this trivial.
  • Apple doesn’t offer any robust way to get occurrences of repeating events.
  • Apple Calendar tends to take a long time & lots of CPU cycles to make these changes, and applescript seems to be the only way to talk to them.

So, I use Google Calendar’s API to get events that are happening in the next two weeks, where AppleScript would fail because of their appalling recurrence support. Then, I use AppleScript to update the events on your local calendar (connecting them via UID), since Google doesn’t support the decades-old “URL” field.

And it all works! It’s slow, taking approximately 10 minutes to run on my machine (and maxing out all CPU’s as it does). I know, that’s a stupid computational load but seems to be due to a known bug in OSX Calendar (look for talk of tccd and applescript calendar slow).

So try it out, if you find any glitches or make any improvements post in the comments! (or even better, submit a pull request via GitHub!)

Fishshell: CMake & finding source directories

CMake is a great tool for makefile generation, far better than old arcane configure scripts and such, but it’s great out-of-source build support can lead to a common annoyance of constantly jumping back and forth between build and source directories, or having multiple build directories for a single source checkout. In my case, I frequently find myself forgetting exactly where the correct source tree is when I’m working in a build-tree.

So, here’s a little fish function called prompt_src that you can add to your prompt to let it always show you the source directory for the build tree you’re in, and also show the current git version that you’re working from. The image above shows my OpenCV/build directory (the response is in yellow indicating it’s not a Git source tree), and then my main application directory showing in Red because it’s been modified, but it’s the develop branch.

# src
function prompt_src --description 'Find details on this builds SOURCE dir'
    set -l cdir (pwd)
    while [ $cdir != "/" ]
        if [ -e $cdir/CMakeCache.txt ]
            set -l SourceDir ( cat $cdir/CMakeCache.txt | grep SOURCE_DIR | head -n 1 | cut -d '=' -f 2)
            if [ -d $SourceDir/.git ]
                set -l gitinfo (git --git-dir=$SourceDir/.git --work-tree=$SourceDir status -sb --untracked-files=no)
                set -l branch ( echo $gitinfo[1] | cut -b 4- )
                set -l branch_color (set_color red)\[M\]
                if test (count $gitinfo) -eq 1
                    set branch_color (set_color green)

                echo \* Builds (set_color green)$SourceDir $branch_color \($branch\) (set_color normal)
                echo \* Builds (set_color yellow)$SourceDir (set_color normal)
        set cdir (dirname $cdir)

FishShell: Create & Expand compressed archives

If you spend much time in a terminal, be it on Mac or Linux, one thing you wind up doing often is creating and decompressing tarballs. Be they tgz, tbs, or just plain tar files, they’re the archive format of choice for folks working in *nix environments due to their universal support. Most commonly, the only way to get any real status information is to turn on “verbose” mode which outputs each filename as it goes. That’s not terribly useful for large archives.

If you install the ‘pv’ tool, you can partner it with some commandline-fu to get nice progress bars. But why deal with it, when you can write a fish function to do it for you!

Here's a script (expand.fish) that decompresses a variety of formats:

function expand -d 'Decompress a file' -a filename
    set -l extension ( echo $filename | awk -F . '{print $NF}')
    switch $extension
        case tar
            echo "Un-tar-ing $filename..."
            pv $filename | tar xf -
        case tgz
            echo "Un-tar/gz-ing $filename..."
            pv $filename | tar zxf -
        case tbz
            echo "Un-tar/bz-ing $filename..."
            pv $filename | tar jxf -
        case gz
            echo "Un-gz-ing $filename..."
            pv $filename | gunzip -
        case bz
            echo "Un-bz-ing $filename..."
            pv $filename | bunzip2 -
        case zip
            echo "Un-zipping $filename..."
            unzip $filename
        case '*'
            echo I don\'t know what to do with $extension files.

And here's a matching script for creating tarballs:

function tarball -d "Create a tarball of collected files" -a filename
    echo "Creating a tarball of $filename"
    if [ -e $filename ]
        echo "Sorry, $filename already exists."
    set -l args $argv[2..-1]
    set -l size (du -ck $args | tail -n 1 | cut -f 1)
    set -l extension ( echo $filename | awk -F . '{print $NF}')

    switch $extension
        case tgz
            tar cf - $args | pv -p -s {$size}k | gzip -c > $filename
        case tbz
            tar cf - $args | pv -p -s {$size}k | bzip2 -c > $filename
        case '*'
            echo "I don't know how to make a '$extension' file."
    set -l shrunk (du -sk $filename | cut -f 1)
    set -l ratio ( math "$shrunk * 100.0 / $size")
    echo Reduced {$size}k to {$shrunk}k \({$ratio}%\)