Status Report

Greetings wonderful tor community! This month I pretended to be a web development again for kicks and giggles.

Tor People Page

Hiro’s putting together a new website for Tor. As the maintainer of our people page I ported it over. Personally I think the new page looks a lot better.


people page before


people page after

Hrio declined my patch, and that’s fine. Learning Lektor was fun. The new site will launch with Hiro’s version of the page with her as its maintainer.

Personal Website

Beside the Tor website I also put the finishing touches on my own! For comparison my old site is available here.

personal website

Happy January, onion enthusiasts! Made it through another spin of the globe. January I wrapped up a couple projects that were in flight…

Nyx 2.1 Release

My biggest news for users this month is Nyx 2.1! No features, but this release includes a full year’s worth of fixes that smooths Nyx’s rough edges.

If you run into any issues please let me know!

Bandwidth File Support

Bandwidth files are weighting heuristics used by Tor for relay selection. Stem can now read these documents. For example…

import stem.descriptor
import stem.util.str_tools

# Bandwidth files are not yet available, but soon will be...

with open('/home/atagar/Desktop/bandwidth_file_v1.2') as bandwidth_file:
  desc = next(stem.descriptor.parse_file(bandwidth_file, 'bandwidth-file 1.2'))

print('Median relay bandwidth measurements are...\n')

for fingerprint, measurement in desc.measurements.items():
  # Not quite sure what the units are. The spec seems to indicate KB/s?

  bandwidth = '%s/s' % stem.util.str_tools.size_label(1024 * int(measurement.get('bw_mean', '0')))
  print('  * %s (%s) averaged %s' % (measurement.get('nick', ''), fingerprint, bandwidth))
% python
Median relay bandwidth measurements are...

  * PompoRelay (319AC0126125DA6A557A416C1BB643E52E272AD0) averaged 187 MB/s
  * oberon (B4486871EA062BAF2354F070FECC344CF3C20880) averaged 196 MB/s
  * ICSIL1xDEDISx1 (7CBBAB7867F939763D94FC6C2819E129971CB0E4) averaged 54 MB/s
  * pollypurebred (EF37D150F3A851B32D9E843605D0EA8A1B938117) averaged 86 MB/s
  * Dementor (46D79CCB83639718177B4BDB8AED6BE9378B1D0F) averaged 158 MB/s
  * wormhole (8B863F5CE8676EFE5BB070CE3CAF54A76AE74D1A) averaged 90 MB/s

Hope everyone had a fun for the holidays! Like many of us December I glugged eggnog with family so this will be short.

  • Stem 1.7.1 hotfix release. Tor adjusted its bootstrap log format in a way that broke Stem, requiring a hotfix release for continued compatibility.
  • Nyx 2.1 incoming! No sexy features, but many long overdue stability fixes. January will be going toward this.
  • Stem failed to read large payloads from Tor’s ORPorts.
  • Writing a new homepage for myself. My current site dates back to my college days and it looks the part. Still in progress but this month I finished drafting its content, passed W3C validation, and made its github repo.
Old Homepage
old homepage
New Homepage
new homepage

Happy holidays everyone! This November Iain and I worked together on Stem’s descriptor capabilities. Though this won’t be of interest to a general audience, for metrics folks like us these are pretty neat.

Descriptor Digesting

Tor relay information is provided by multiple documents. Signed descriptors transitively validate others by inclusion of their digest. For example, our consensus references server descriptor digest, and server descriptors in turn cite extrainfo digests.

To illustrate, here’s a diagram from Iain…

descriptor digests

Stem can now calculate digests from server, extrainfo, microdescriptor, and consensus documents. For instance, to validate an extrainfo descriptor…

import sys

import stem.descriptor.remote
import stem.util.tor_tools

def download_descriptors(fingerprint):
  Downloads the descriptors we need to validate this relay. Downloads are
  parallelized, providing the caller with a tuple of the form...

    (router_status_entry, server_descriptor, extrainfo_descriptor)

  conensus_query = stem.descriptor.remote.get_consensus()
  server_desc_query = stem.descriptor.remote.get_server_descriptors(fingerprint)
  extrainfo_query = stem.descriptor.remote.get_extrainfo_descriptors(fingerprint)

  router_status_entries = filter(lambda desc: desc.fingerprint == fingerprint,

  if len(router_status_entries) != 1:
    raise IOError("Unable to find relay '%s' in the consensus" % fingerprint)

  return (

if __name__ == '__main__':
  fingerprint = raw_input("What relay fingerprint would you like to validate?\n")
  print('')  # blank line

  if not stem.util.tor_tools.is_valid_fingerprint(fingerprint):
    print("'%s' is not a valid relay fingerprint" % fingerprint)

    router_status_entry, server_desc, extrainfo_desc = download_descriptors(fingerprint)
  except Exception as exc:

  if router_status_entry.digest == server_desc.digest():
    print("Server descriptor digest is correct")
    print("Server descriptor digest invalid, expected %s but is %s" % (router_status_entry.digest, server_desc.digest()))

  if server_desc.extra_info_digest == extrainfo_desc.digest():
    print("Extrainfo descriptor digest is correct")
    print("Extrainfo descriptor digest invalid, expected %s but is %s" % (server_desc.extra_info_digest, extrainfo_desc.digest()))
What relay fingerprint would you like to validate?

Server descriptor digest is correct
Extrainfo descriptor digest is correct

Detached Signatures

Somewhat obscure, detached signatures are exchanged among directory authorites to establish the next hour’s consensus. They reference the agreed upon digest the consensus should have, and are only available five minutes of every hour (minutes 55-60). Stem can now read these documents

import stem.descriptor.remote
detached_sigs = stem.descriptor.remote.get_detached_signatures().run()[0]
for i, sig in enumerate(detached_sigs.signatures):
  print('Signature %i is from %s' % (i + 1, sig.identity))

When available (minutes 55-60 of the hour)

% python
Signature 1 is from 0232AF901C31A04EE9848595AF9BB7620D4C5B2E
Signature 2 is from 14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4
Signature 3 is from 23D15D965BC35114467363C165C4F724B64B4F66
Signature 4 is from 27102BC123E7AF1D4741AE047E160C91ADC76B21
Signature 5 is from 49015F787433103580E3B66A1707A00E60F2D15B
Signature 6 is from D586D18309DED4CD6D57C18FDB97EFA96D330566
Signature 7 is from E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58
Signature 8 is from ED03BB616EB2F60BEC80151114BB25CEF515B226
Signature 9 is from EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97

When unavailable (minutes 0-55 of the hour)

% python
Traceback (most recent call last):
  File "", line 3, in 
    detached_sigs = stem.descriptor.remote.get_detached_signatures().run()[0]
  File "/home/atagar/Desktop/stem/stem/descriptor/", line 476, in run
    return list(self._run(suppress))
  File "/home/atagar/Desktop/stem/stem/descriptor/", line 487, in _run
    raise self.error
urllib2.HTTPError: HTTP Error 404: Not found

Hello fellow onion lovers! Winter is here, and with it gloomy overcast days and seasonal depression. Yay!

Half kidding aside, September and October have been a great couple of months. Saw a great performance of Jane Eyre with Shari, Cirque du Soleil’s Volta with my family, took care of my grandma a few weeks, fun Amazon team outing that was littered in Henry’s art (the same artist that decorated our tor office)…

flatsticks art

… and of course tasty, tasty code.

Stem 1.7

No surprise the big ticket item of these last couple months has been Stem 1.7. In the works for almost a year, this release included numerous features and fixes.

Check it out!

Membership Audit

Tis that time of year again! Per our membership policy and my duties as our membership secretary I’ve sunk several weeks into reviewing what folks have been up to. Lotta neat things!

Nothing too notworthy to announce at the moment. The process is chugging along reasonably smoothly. I’ve attempted to tailor it the best I can as an opportunity to re-integrate fringe members of our community that are drifting away. In some cases folks indicate that work, life, and other hobbies have made it time to move on. In others we’re brainstorming fun projects they might enjoy.

Curious to see how it goes!

Smaller things these last couple months include…

Greetings fellow onion enthusiasts! Like the rest of the summer August has been less tor-ry than usual. This month went toward eviscerating my home in an act of utter apartment seppuku.

apartment mess

Truly it’s amazing how much stuff closets accumulate. Dumped my every worldly possession in a pile and now dealing with the mess. I’ll be at this through September as well but so far so good. Eight full grocery bags of stuff to rid myself of and counting. Fun project!

Stem 1.7 Release Prep

As for the technical front… *sigh*. I told Matt I would release Stem 1.7 when he asked for it but six weeks later and still no release. Dealing with several pre-release issues…

Tor Summer of Privacy

This month Dave concluded his Tor SoP project. Honestly we didn’t get too far. I’ll probably advertise this project again in next year’s GSoC/SoP. That said, Dave sent quite a bit of code at the end so much of August went toward reviewing that.

PS. Already half way through September? Yikes. Between the tardiness of this report and spring cleaning I’m gonna skip my September update. I won’t be in Mexico but hope you all have fun!

Oh oops! Just realized I completely forgot last month’s status report. Probably just as well. Festivals and fun crowd tor out during the summer.

Strawberry Festival, Fremont Fair, Underground Tour, and spring cleaning aside honestly not much to report.

  • Lots of code reviewing. I might not be writing code but Dave is.
  • Membership and website adjustments. Rote work aside, rewrote our ‘welcome’ emails and worked with Tom on adding people page pronouns.
  • Revamped Stem’s equality checks and hashing. Hashes now account for attribute types, are consistent between interpreter runs, and cached by immutable classes.
  • Python 3.5 bundled a buggy lru_cache implementation. Stem now works around this.
  • Python 3.6’s new ‘async’ keyword broke Stem’s testing tools.

Hope everyone else is having a similarly great summer!


Driver trained his whole life for this moment.

Decorated space marine marooned with his fellow starving colonists, the microfusion rifle in his hands hummed with unearthly firepower. Enough to vaporize a mountain. But tonight Driver stalked pray of a different kind. His nemesis. His dinner. A sleeping, injured turtle with dementia.

Just two feet from his quarry Driver raised his weapon and… missed. He missed eighty times in a row.

Ya know, I didn’t feel bad that night when Driver starved. Clearly I’m playing too much Rimworld. module

Courtesy of teor Tor authority and fallback information now reside in their own files. This is great, as it allows Stem to provide fresh directory information through our module


COLUMN_FORMAT = '%-17s%-20s%-10s%-10s'

  authorities =
except IOError as exc:
  print('%s\n' % exc)
  authorities =

print(COLUMN_FORMAT % ('Name', 'Address', 'ORPort', 'DirPort'))

for authority in authorities.values():
  print(COLUMN_FORMAT % (authority.nickname, authority.address, authority.or_port, authority.dir_port))
% python 
Name             Address             ORPort    DirPort   
maatuska        80        443       
tor26           443       80        
Bifroest      443       80        
longclaw       443       80        
dizum       443       80        
bastet       443       80        
gabelmoo      443       80        
moria1          9101      9131      
dannenberg      443       80        
Faravahar      443       80        

May’s been quite a juggling act. Smaller things this month include…

"Mission accomplished!" I said. No aircraft carrier, but none the less you’d think I would learn from transnational snafus.

Much of March was still out of commission, but I’ve felt better a few weeks now so maybe this time it’ll hold? We didn’t get to a root cause so likely that whatever nailed me will crop back up someday. But we’ll cross that bridge when we come to it. After all, there’s only so many orifices a doctor can probe… right?   (he says hopefully)

What could be funner than that, you ask? Taxes! And dentists! And paperwork! Kidding aside, last couple months had some great tor stuff too.

ORPort Descriptor Downloads

Stem can now download descriptors through ORPorts just like Tor!

reply = stem.descriptor.remote.get_server_descriptors(
  endpoint = stem.ORPort('', 90),

This came from a desire to health check my relay, but dovetails nicely with the ORPort capabilities that will be the big feature of Stem’s 1.7 release.

Stay tuned!

Descriptor Compression

Recently Tor added ZSTD and LZMA support and Stem now supports them too. ZSTD and LZMA’s higher compression ratio comes at a CPU cost and are less available, so when using these to save bandwidth you should provide a fallback…

reply = stem.descriptor.remote.get_server_descriptors(
  compression = (
    Compression.LZMA,  # higher compression but often unavailable
    Compression.GZIP,  # decent compression and always available

Summer of Privacy

A very, very warm welcome to Dave Rolek! Dave will be expanding Stem’s ORPort and hidden service v3 capabilities this summer as part of Tor Summer of Privacy.

Congrats, Dave!

Smaller things these last couple months include…

  • Expanded Nyx’s FAQ to clarify how to get started and explain why apt-get often doesn’t work.
  • Further expunged Nyx’s old name. Thanks to Homebrew for deprecating its old port. One of these days I should reach out to the Wikipedia editors as well…
  • Stem now supplies a user agent when downloading descriptors, so relays can distinguish Tor and Stem clients from others.
  • Formalized the procedure we use when checking membership activity to better avoid surprises and encourage past volunteers to get back involved.
  • Finally got around to swapping this blog to https. Yeah, yeah, I know…

Hi all! Nasty stomach bug left me sleeping and running between doctors most of the month so sadly not much to report. That said, feeling much better now so picking steam back up on tasty, tasty code.

On the fun-things front this month I wrote my second build for Path of Exile, but less than usual for Tor.


Tor Summer of Privacy

This year Google declined our GSoC application to make room for budding open source projects. Personally I think asking us to take a break every so often is applaudable (small projects need the boost!), but this left us asking: what next?

In 2015 when this happened we ran a smaller program of our own called Tor Summer of Privacy. Colin is now organizing a similar program for this year (thanks Colin!).

Stay tuned on Tor’s blog for the official announcement. Tim and I wrote a joint proposal, so if you wanna get involved with Tor this summer then check it out!

Tor Membership

This February the only other tidbit that comes to mind are a couple initiatives as our membership secretary. Kicked off the selection process for our next Community Council and began our bi-annual checkup for membership activity.