Damn I love old architecture. Amsterdam’s venue was my favorite yet. Thanks Jon for setting it up!


Tor Developer Meeting

Beautiful city, neat hotel, among the best venues yet. On the tor front focused on community governance. Tricky topics are best discussed in person, and for me highlights were chats with Jens and Pepijn. Both expressed concerns regarding our community council in the past so this was a great opportunity to pick their brains. Jens especially had some great ideas for making the process even better.

Meeting aside saw the canals…

… Van Gogh museum…

… sat inside a giant clog…

… and of course found a large condom store next to snortable coca powder.

Amsterdam left me very confused.


Community Council

Proposal passed! Thanks to Alison and Tim for all their hard work. You can peruse our ratified community council policy here.


Ed25519 Certificate Support

In collaboration with Patrick O’Doherty added Stem support for parsing Ed25519 certificates.

When PyNaCl is present these certificates are used to check the integrity of server descriptors if you parse with validation.

Jury duty this month, but that was more than made up for with tasty, tasty pizza – thank you Shari! February had lots of great stuff.


Google Summer of Code

We have been accepted!

Got a project to pitch? It’s not too late! Students are already rolling into irc so if you want to mentor this summer the sooner you let me know the better.


Community Council

Tag teaming with Alison on a formal proposal for our community council. Initial council attempt hit some potholes but I’m glad for it since it’s given us a much better idea of what we want the council to become.

Fingers crossed that it’s successful!


Besides this just a handful of Stem changes…

  • Patrick migrated our cryptographic descriptor validation from pycrypto (which is unmaintained) to cryptography.
  • Further sped up our tests. Unit and integ tests now run 15% faster, and @only_run_once is fixed so the ‘RUN_ALL’ target is dramatically faster.
  • Corrected unesacaping of controller responses. Thansk to this authentication cookies with non-English characters in their path now work.

See ya all in Amsterdam!

Oh for the love of… I’m having horrible luck this winter. Third bug to knock me out. From the fever guessing this one was the flu. Ok virulent antigens, you’ve had your fun with Damian. Time to leave him alone now…

None the less, this was a good month.


Tor Internal Bylaws

We finally have a ratified voting policy!

Thanks everyone! Alison and I will put together some initial proposals but first gonna take a little breather.


Stem Test Speed

During my morning commute I’ve been poking at Stem’s test performance. Faster tests mean quicker development cycles. Always a good investment.

This took some unexpected turns resulting in…

  • 22% faster integration tests.
  • 25% faster descriptor parsing when validating.
  • Our stem.util.test_tools module now provides a TimedTestRunner class that gives individual test runtimes. Sadly this is something Python’s built-in unittest module doesn’t have.

    You can run our tests with the ‘–verbose’ argument to see this information…

    verbose test output


Other neat things this month include…

Happy end to 2016! Shockingly Christmas holidays weren’t the most productive for me. Family aside a particularly juicy cold knocked me flat for a week. That said, here’s the tor things I was up to in December…


Descriptor Protocol Support

Added Stem support for new tor additions, biggest being tor’s new protocol fields. While doing this work ran into some issues where tor’s actual behavior conflicts with the spec. Now just waiting for another pair of eyes.


Tor Internal Bylaws

Progress! Taking this slow to do my best to make sure we get it right but think we’re narrowing in on a voting procedure folks are comfortable with.

Hi all! Small Nyx/Stem changes aside highlights for this month are surprisingly non-coding focused…


Stem 1.5 Release

Damn long overdue. This is a big release encompassing seventeen months of work. Check it out!

Highlights include vastly improved python 3.x performance, manual information, and fallback directory data. But this is just the tip of the iceberg.


Tor Internal Bylaws

Tried my best to ratify Alison’s community membership doc but seems that’s not to be. Pity. With our internal community I’m not optimistic we’ll ever be able to agree on a large overarching policy like it. As such trying my hand at more narrowly focused bylaws instead.

Presently working on our voting procedure. Assuming we can agree on that I’ll follow it up with a proposal to again try to fix tor-internal@. Fingers crossed it goes better than last time!

Damn this was long overdue. I’m delighted to announce Stem 1.5.2, the accumulation of seventeen months of improvements.

What is Stem, you ask? For those who aren’t familiar with it Stem is a Python library for interacting with Tor. With it you can script against your relay, descriptor data, or even write applications similar to Nyx and Vidalia.

https://stem.torproject.org/

So what’s new in this release? Short answer: a lot.


Improved Python 3.x Performance

Reading from tor’s control port 800x faster, Python 3.x users will find this release a dramatic improvement. By ‘dramatic’ I mean multiple orders of magnitude.

python3 performance difference


Tor Manual Information

Stem’s new stem.manual module provides programmatic access for Tor manual information. For example, say we want a little script that told us what our torrc options do…

from stem.manual import Manual
from stem.util import term

try:
  print("Downloading tor's manual information, please wait...")
  manual = Manual.from_remote()
  print("  done\n")
except IOError as exc:
  print("  unsuccessful (%s), using information provided with stem\n" % exc)
  manual = Manual.from_cache()  # fall back to our bundled manual information

print('Which tor configuration would you like to learn about?  (press ctrl+c to quit)\n')

try:
  while True:
    requested_option = raw_input('> ').strip()

    if requested_option:
      if requested_option in manual.config_options:
        option = manual.config_options[requested_option]
        print(term.format('%s %s' % (option.name, option.usage), term.Color.GREEN, term.Attr.BOLD))
        print(term.format(option.summary, term.Color.GREEN))  # brief description provided by stem

        print(term.format('\nFull Description:\n', term.Color.GREEN, term.Attr.BOLD))
        print(term.format(option.description + '\n', term.Color.GREEN))
      else:
        print(term.format("Sorry, we don't have any information about %s. Are you sure it's an option?" % requested_option, term.Color.RED))
except KeyboardInterrupt:
  pass  # user pressed ctrl+c

manual info demo


Fallback Directory Information

Relieving load fromt the directory authority, Stem can retrieve information about and use tor’s fallback directory mirrors

import time
from stem.descriptor.remote import DescriptorDownloader, FallbackDirectory

downloader = DescriptorDownloader()

for fallback_directory in FallbackDirectory.from_cache().values():
  start = time.time()
  downloader.get_consensus(endpoints = [(fallback_directory.address, fallback_directory.dir_port)]).run()
  print('Downloading the consensus took %0.2f from %s' % (time.time() - start, fallback_directory.fingerprint))

fallback directory fetching demo


As always this is just the tip of the iceberg. For a full rundown on the myriad of improvements and fixes in this release see our changelog.

Hi all! For the first time since I started writing these in 2010 missed a couple. Health issues knocked me out of commission but finally all sorted out. Here’s the funner stuff I was up to in August through October.


Nyx Rewrite

Done! Er… well, done-ish.

Finished rewriting the last of Nyx’s legacy codebase (menu and controller modules). For users this means we now have the tested, python 3.x compatible codebase that will be Nyx 2.0.0.

So why no release? Couple reasons…

  1. Stability. Slick as the new codebase is it introduced regressions causing non-deterministic terminal glitches. Unfortunately these are gonna be a real pita to track down…
  2. New website. This is actually pretty far along. Content is there, work left is mostly styling. My web-dev fu isn’t too strong so this may take a while.

As some of you may have noticed Stem hasn’t received a release in over a year. This isn’t due to idleness. In fact, Stem has enough changes to pack several releases.

Rather, I’ve kept delaying Stem’s 1.5.0 release over and over to coincide with Nyx’s. ‘Just one more month!’ I told myself. Ooooh, such cute optimism.

So Stem will get its release. Probably this week.


Few other noteworthy things…

  • Tor dev meeting. Thanks Jon for making it a success!
  • Finished running this year’s GSoC. Program went great, though little sad I wasn’t up for the mentor summit.
  • Stem can now download microdescriptor consensuses.
  • Helped Roger investigate relays failing to serve current consensus data.
  • Moved DocTor from Tonga to Bifroest among other monitoring updates.
  • Neat non-tor stuff. Dota Internationals, Midsummer Renaissance Festival, and vacation up to Bellingham for the Sparks Museum and Boeing Factory Tour.

Hi all! Getting in lots of family time as I sort out ongoing health issues, but none the less got some neat things to report this month!


Interpreter Panel

Part of Sambuddha’s GSoC project, half my month has gone toward reintroducing nyx’s interpreter and I’m pleased to say it’s turned out great!

Built upon Stem’s tor-prompt this expands upon the capabilities of the last arm release, providing an interactive python interpreter along with updated tor capabilities.


Curses Test Coverage

Concluding a four month overhaul, nyx now has a high degree of test coverage for its curses display capabilities which is good cuz… well, that’s kinda what nyx does. To answer the obvious question no, still no release date but this is a major milestone. Remaining work I have planned includes…

  • Refactor nyx’s menu and controller modules.
  • Put together a new site for nyx.
  • Run an open beta to solicit testing and ideas from our community.

Stem was in the works for two years before its first release. Nyx too will be ready when it’s ready.

  “Always do things right. This will gratify some people and astonish the rest.” -Mark Twain

Summer is here! Family, festivals, and other totally-not-tor things are occupying much of my time so this is gonna be short…

  • Expanded Nyx tests to include the graph and log panels. With this we have coverage of 66% of the curses components.
  • Stem support for new tor additions, highlights of this month being shared randomness and ADD_ONION basic auth.
  • GSoC midterm evaluations – everyone passed!

That’s all – now off to enjoy the sun!

Eeek I’m late! Wanted to get the PyCon trip report out first but… ok, maybe I went a tad overboard there. Oh well, better late than never. Between summer festivals and health issues tor took the back seat again this month but still some neat stuff…


PyCon

Probably wrote way too much on this already so I’ll spare ya. If you haven’t skimmed the report yet then check it out – PyCon was a neat event!


Nyx Event Selection Dialog

Remember arm’s bizarre and clumsy event selection dialog? Remember how confusing it was?

Yeah, it sucked. Sambuddha and I have been tossing pull requests back and forth all month as part of his GSoC project. This is requiring a lot of my direct involvement but oh well, the new dialog has turned out nicely!


Few other noteworthy things…