Self Surveillance Dashboard

I am trying to closely monitor my productivity while working. For this reason I have a couple of metrics that I use to show on a dashboard on my iPad.

The metrics I am most interested in are:

  • Rescuetime productivity for the week

  • Top apps today (with time, and productivity type)

  • Rescuetime productivity pulse for today, and for the week

  • Keypresses this week

  • Clicks this week

  • Git commits per day

  • Ratio between keypresses and clicks

What does it look like

This is how my dashboard looked last night. Productivity this week has been high in general. The productivity pulse was relatively high yesterday, however the keypress to click ratio is low enough to tell me, that I have been doing a lot of non-coding work yesterday. (Usually on a very productive day, this metric is between 20 and 30, on the graph it is around 12 on average).

How is it done

There are three parts to this dashboard; collection of data, a place to display them, and the actual widgets themselves.

Metrics

The metrics on the dashboard come from only two places (well actually more, keep reading). Rescuetime and Stathat.

Rescuetime metrics are rather simple to understand. The interessting part is how stathat gets the metrics for keypresses, clicks, ratio and commits.

Keypresses and clicks

For keypresses, clicks and the ratio between them, another service comes into play. I use WhatPulse to monitor my computer usage, and it stores its metrics in a sqlite database. The api of stathat is very simple, so it is really just a bash oneliner to add data to any metric:

<code class="language-bash">curl -d <span class="s2">"stat=[STATNAME]&ezkey=[STATHAT_KEY]&value=`sqlite3 "</span><span class="o">[</span>PATH_TO_DB<span class="o">]</span><span class="s2">" "</span><span class="o">[</span>SELECT_STATEMENT_TO_GET_ONE_VALUE<span class="o">]</span><span class="err">"</span> http://api.stathat.com/ez</code>

I have made a shell script for each of the three metrics that come from WhatPulse, and each of them runs at a regular interval.

Commits

This proved to be just as easy. As above the call to stathat is just as simple (except this is a counter, not an absolute value)

The interesting part is that it is run as a git hook after each commit. I have placed a post-commit hook in my .git_template/hooks directory

<code class="language-bash"><span class="c">#!/usr/local/bin/zsh</span>
curl -d <span class="s2">"stat=commit&ezkey=[STATHAT_KEY]&count=1"</span> http://api.stathat.com/ez</code>

and placed a init template in my git config

<code class="language-bash"><span class="o">[</span>init<span class="o">]</span>
	<span class="nv">templatedir</span> <span class="o">=</span> ~/.git_template</code>

This makes git init (and git clone too) use this template dir for projects. This means that the hooks in this template dir are added to all new projects. (And you can always git init an existing project, to get the hook scripts added there as well - so off course I did a shell script for doing just that, on all projects I allready had on my computer).

Thats about it.

Dashboard

The actual dashboard was the easiest part. I worked for some time on a NodeWebkit application for my desktop which could be my dashboard - very inspired by the Telepath mac app that Nich Winter did and showed off in his video The 120-hour Workweek.

However I never really got it working quite like I wanted, and when I learned about the Status Board App by Panic, I knew it would solve the problem more elegantly.

Widgets

RescueTime allready had plugins setup so that was easy. Stathat also had integrations directly. So only the top apps widget required some work.

I decided to create a little ruby (Sinatra) service to run and show the widget on a webserver. Statusboard can show either different graphs from data formatted in a special way - or it can show a simple html page (complete with css and javascript).

The sinatra app is only 44 lines of code:

<code class="language-ruby"><span class="nb">require</span> <span class="s1">'uri'</span>
<span class="nb">require</span> <span class="s1">'net/https'</span>
<span class="nb">require</span> <span class="s1">'json'</span>

<span class="k">class</span> <span class="nc">Personal</span> <span class="o"><</span> <span class="no">Sinatra</span><span class="o">::</span><span class="no">Base</span>
  <span class="n">set</span> <span class="ss">:root</span><span class="p">,</span> <span class="no">File</span><span class="o">.</span><span class="n">dirname</span><span class="p">(</span><span class="bp">__FILE__</span><span class="p">)</span>
  <span class="n">set</span> <span class="ss">:views</span><span class="p">,</span> <span class="no">Proc</span><span class="o">.</span><span class="n">new</span> <span class="p">{</span> <span class="no">File</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">root</span><span class="p">,</span> <span class="s2">"views"</span><span class="p">)</span> <span class="p">}</span> 

  <span class="k">def</span> <span class="nf">get_data</span>
    <span class="n">uri</span> <span class="o">=</span> <span class="no">URI</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="s2">"https://www.rescuetime.com/anapi/data?format=json&key=</span><span class="si">#{</span><span class="no">ENV</span><span class="o">[</span><span class="s1">'RESCUETIME_KEY'</span><span class="o">]</span><span class="si">}</span><span class="s2">&resolution_time=day&rk=activity&by=interval"</span><span class="p">)</span>
    <span class="n">http</span> <span class="o">=</span> <span class="no">Net</span><span class="o">::</span><span class="no">HTTP</span><span class="o">.</span><span class="n">new</span><span class="p">(</span><span class="n">uri</span><span class="o">.</span><span class="n">host</span><span class="p">,</span> <span class="n">uri</span><span class="o">.</span><span class="n">port</span><span class="p">)</span>
    <span class="n">http</span><span class="o">.</span><span class="n">read_timeout</span> <span class="o">=</span> <span class="mi">30</span>
    <span class="n">http</span><span class="o">.</span><span class="n">use_ssl</span> <span class="o">=</span> <span class="kp">true</span>
    <span class="n">http</span><span class="o">.</span><span class="n">verify_mode</span> <span class="o">=</span> <span class="no">OpenSSL</span><span class="o">::</span><span class="no">SSL</span><span class="o">::</span><span class="no">VERIFY_PEER</span>
    <span class="n">request</span> <span class="o">=</span> <span class="no">Net</span><span class="o">::</span><span class="no">HTTP</span><span class="o">::</span><span class="no">Get</span><span class="o">.</span><span class="n">new</span><span class="p">(</span><span class="n">uri</span><span class="o">.</span><span class="n">request_uri</span><span class="p">)</span>
    <span class="n">response</span> <span class="o">=</span> <span class="n">http</span><span class="o">.</span><span class="n">request</span><span class="p">(</span><span class="n">request</span><span class="p">)</span>
    <span class="no">JSON</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="n">response</span><span class="o">.</span><span class="n">body</span><span class="p">)</span><span class="o">[</span><span class="s2">"rows"</span><span class="o">]</span>
  <span class="k">end</span>

  <span class="k">def</span> <span class="nf">get_class_name</span> <span class="n">productivity_score</span>
    <span class="o">[</span><span class="s2">"very_unproductive"</span><span class="p">,</span> <span class="s2">"unproductive"</span><span class="p">,</span> <span class="s2">"neutral"</span><span class="p">,</span> <span class="s2">"productive"</span><span class="p">,</span> <span class="s2">"very_productive"</span><span class="o">][</span><span class="n">productivity_score</span><span class="o">+</span><span class="mi">2</span><span class="o">]</span>
  <span class="k">end</span>

  <span class="k">def</span> <span class="nf">format_time</span> <span class="n">time</span>
    <span class="n">hours</span> <span class="o">=</span> <span class="n">time</span><span class="o">/</span><span class="mi">3600</span><span class="o">.</span><span class="n">to_i</span>
    <span class="n">minutes</span> <span class="o">=</span> <span class="p">(</span><span class="n">time</span><span class="o">/</span><span class="mi">60</span> <span class="o">-</span> <span class="n">hours</span> <span class="o">*</span> <span class="mi">60</span><span class="p">)</span><span class="o">.</span><span class="n">to_i</span>
    <span class="n">seconds</span> <span class="o">=</span> <span class="p">(</span><span class="n">time</span> <span class="o">-</span> <span class="p">(</span><span class="n">minutes</span> <span class="o">*</span> <span class="mi">60</span> <span class="o">+</span> <span class="n">hours</span> <span class="o">*</span> <span class="mi">3600</span><span class="p">))</span>
    <span class="s2">"%02d:%02d:%02d"</span> <span class="o">%</span> <span class="o">[</span><span class="n">hours</span><span class="p">,</span> <span class="n">minutes</span><span class="p">,</span> <span class="n">seconds</span><span class="o">]</span>
  <span class="k">end</span>

  <span class="n">get</span> <span class="s1">'/widget'</span> <span class="k">do</span>
    <span class="n">erb</span> <span class="ss">:widget</span>
  <span class="k">end</span>

  <span class="n">get</span> <span class="s1">'/content'</span> <span class="k">do</span>
    <span class="o"><</span><span class="n">a</span> <span class="n">href</span><span class="o">=</span><span class="s1">'https://github.com/data'</span> <span class="n">class</span><span class="o">=</span><span class="s1">'user-mention'</span><span class="o">></span><span class="vi">@data</span><span class="o"><</span><span class="sr">/a> = get_data</span>
<span class="sr">    erb :content</span>
<span class="sr">  end</span>
<span class="sr">end</span></code>

The widget view contains both the javascript and css directly, since it is very simple, and is only loaded once a day, and probably never changes.

<code class="language-html"><span class="nt"><html></span>
<span class="nt"><head></span>
<span class="nt"><meta</span> <span class="na">http-equiv=</span><span class="s">"Content-Type"</span> <span class="na">content=</span><span class="s">"text/html; charset=utf8"</span> <span class="nt">/></span>
<span class="nt"><meta</span> <span class="na">http-equiv=</span><span class="s">"Cache-control"</span> <span class="na">content=</span><span class="s">"no-cache"</span> <span class="nt">/></span>
<span class="nt"><style></span>
<span class="nt">table</span> <span class="p">{</span>
  <span class="k">width</span><span class="o">:</span> <span class="m">100%</span><span class="p">;</span>
  <span class="k">font-size</span><span class="o">:</span> <span class="m">12px</span><span class="p">;</span>
<span class="p">}</span>

<span class="nt">tr</span> <span class="p">{</span>
  <span class="k">background-color</span><span class="o">:</span> <span class="m">#ccc</span><span class="p">;</span>
  <span class="k">color</span><span class="o">:</span> <span class="m">#000</span><span class="p">;</span>
  <span class="k">padding</span><span class="o">:</span> <span class="m">1px</span><span class="p">;</span>
<span class="p">}</span>

<span class="nt">tr</span><span class="nc">.very_unproductive</span> <span class="p">{</span>
  <span class="k">background-color</span><span class="o">:</span> <span class="m">#FF0000</span><span class="p">;</span>

<span class="p">}</span>

<span class="nt">tr</span><span class="nc">.unproductive</span> <span class="p">{</span>
  <span class="k">background-color</span><span class="o">:</span> <span class="m">#F78181</span><span class="p">;</span>
<span class="p">}</span>

<span class="nt">tr</span><span class="nc">.neutral</span> <span class="p">{</span>
  <span class="k">background-color</span><span class="o">:</span> <span class="m">#F7F8E0</span><span class="p">;</span>
<span class="p">}</span>


<span class="nt">tr</span><span class="nc">.productive</span> <span class="p">{</span>
  <span class="k">background-color</span><span class="o">:</span> <span class="m">#BCF5A9</span><span class="p">;</span>
<span class="p">}</span>

<span class="nt">tr</span><span class="nc">.very_productive</span> <span class="p">{</span>
  <span class="k">background-color</span><span class="o">:</span> <span class="m">#2EFE64</span><span class="p">;</span>
<span class="p">}</span>

<span class="nt">table</span> <span class="nt">tr</span> <span class="nt">td</span> <span class="p">{</span>
  <span class="k">padding</span><span class="o">:</span> <span class="m">4px</span><span class="p">;</span>
  <span class="k">color</span><span class="o">:</span> <span class="m">#000</span><span class="p">;</span>
<span class="p">}</span>
<span class="nt"></style></span>

<span class="nt"><script </span><span class="na">type=</span><span class="s">"text/javascript"</span><span class="nt">></span>
<span class="kd">function</span> <span class="nx">refresh</span><span class="p">()</span>
<span class="p">{</span>
  <span class="kd">var</span> <span class="nx">req</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">XMLHttpRequest</span><span class="p">();</span>
  <span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">"Refreshing Count..."</span><span class="p">);</span>
  <span class="nx">req</span><span class="p">.</span><span class="nx">onreadystatechange</span><span class="o">=</span><span class="kd">function</span><span class="p">()</span> <span class="p">{</span>
    <span class="k">if</span> <span class="p">(</span><span class="nx">req</span><span class="p">.</span><span class="nx">readyState</span><span class="o">==</span><span class="mi">4</span> <span class="o">&&</span> <span class="nx">req</span><span class="p">.</span><span class="nx">status</span><span class="o">==</span><span class="mi">200</span><span class="p">)</span> <span class="p">{</span>
      <span class="nb">document</span><span class="p">.</span><span class="nx">getElementById</span><span class="p">(</span><span class="s1">'topAppsContainer'</span><span class="p">).</span><span class="nx">innerHTML</span> <span class="o">=</span> <span class="nx">req</span><span class="p">.</span><span class="nx">responseText</span><span class="p">;</span>
    <span class="p">}</span>
  <span class="p">}</span>
  <span class="nx">req</span><span class="p">.</span><span class="nx">open</span><span class="p">(</span><span class="s2">"GET"</span><span class="p">,</span> <span class="s1">'/content'</span><span class="p">,</span> <span class="kc">true</span><span class="p">);</span>
  <span class="nx">req</span><span class="p">.</span><span class="nx">send</span><span class="p">(</span><span class="kc">null</span><span class="p">);</span>
<span class="p">}</span>

<span class="kd">function</span> <span class="nx">init</span><span class="p">()</span> <span class="p">{</span>
 <span class="k">if</span> <span class="p">(</span><span class="nb">document</span><span class="p">.</span><span class="nx">location</span><span class="p">.</span><span class="nx">href</span><span class="p">.</span><span class="nx">indexOf</span><span class="p">(</span><span class="s1">'desktop'</span><span class="p">)</span> <span class="o">></span> <span class="o">-</span><span class="mi">1</span><span class="p">)</span>
  <span class="p">{</span>
    <span class="nb">document</span><span class="p">.</span><span class="nx">getElementById</span><span class="p">(</span><span class="s1">'topAppsContainer'</span><span class="p">).</span><span class="nx">style</span><span class="p">.</span><span class="nx">backgroundColor</span> <span class="o">=</span> <span class="s1">'black'</span><span class="p">;</span>
  <span class="p">}</span>

  <span class="nx">refresh</span><span class="p">()</span>
    <span class="kd">var</span> <span class="kr">int</span><span class="o">=</span><span class="nx">self</span><span class="p">.</span><span class="nx">setInterval</span><span class="p">(</span><span class="kd">function</span><span class="p">(){</span><span class="nx">refresh</span><span class="p">()},</span><span class="mi">60000</span><span class="p">);</span>
<span class="p">}</span>
<span class="nt"></script></span>
<span class="nt"></head></span>
<span class="nt"><body</span> <span class="na">onload=</span><span class="s">"init();"</span><span class="nt">></span>
<span class="nt"><div</span> <span class="na">id=</span><span class="s">"topAppsContainer"</span><span class="nt">></span>
<span class="nt"></div></span>
<span class="nt"></body></span>
<span class="nt"></html></span></code>

And finally the content view contains the actual table rows

<code class="language-html"><span class="nt"><table></span>
  <span class="err"><</span>% @data.take(9).each do |row| %>
    <span class="nt"><tr</span> <span class="na">class=</span><span class="s">"<%= get_class_name(row[5]) %>"</span><span class="nt">></span>
    <span class="nt"><td></span><span class="err"><</span>%= row[3] %><span class="nt"></td></span>
    <span class="nt"><td></span><span class="err"><</span>%= row[4] %><span class="nt"></td></span>
    <span class="nt"><td></span><span class="err"><</span>%=format_time row[1].to_i %><span class="nt"></td></span>
    <span class="nt"></tr></span>
  <span class="err"><</span>% end %>
<span class="nt"></table></span></code>

… why take(9)? Thats what can be showed without overflowing!

Thats it.