Status Report

Hi all! Small Nyx/Stem changes aside highlights for this month are surprisingly non-coding focused…

Stem 1.5 Release

Damn long overdue. This is a big release encompassing seventeen months of work. Check it out!

Highlights include vastly improved python 3.x performance, manual information, and fallback directory data. But this is just the tip of the iceberg.

Tor Internal Bylaws

Tried my best to ratify Alison’s community membership doc but seems that’s not to be. Pity. With our internal community I’m not optimistic we’ll ever be able to agree on a large overarching policy like it. As such trying my hand at more narrowly focused bylaws instead.

Presently working on our voting procedure. Assuming we can agree on that I’ll follow it up with a proposal to again try to fix tor-internal@. Fingers crossed it goes better than last time!

Hi all! For the first time since I started writing these in 2010 missed a couple. Health issues knocked me out of commission but finally all sorted out. Here’s the funner stuff I was up to in August through October.

Nyx Rewrite

Done! Er… well, done-ish.

Finished rewriting the last of Nyx’s legacy codebase (menu and controller modules). For users this means we now have the tested, python 3.x compatible codebase that will be Nyx 2.0.0.

So why no release? Couple reasons…

  1. Stability. Slick as the new codebase is it introduced regressions causing non-deterministic terminal glitches. Unfortunately these are gonna be a real pita to track down…
  2. New website. This is actually pretty far along. Content is there, work left is mostly styling. My web-dev fu isn’t too strong so this may take a while.

As some of you may have noticed Stem hasn’t received a release in over a year. This isn’t due to idleness. In fact, Stem has enough changes to pack several releases.

Rather, I’ve kept delaying Stem’s 1.5.0 release over and over to coincide with Nyx’s. ‘Just one more month!’ I told myself. Ooooh, such cute optimism.

So Stem will get its release. Probably this week.

Few other noteworthy things…

  • Tor dev meeting. Thanks Jon for making it a success!
  • Finished running this year’s GSoC. Program went great, though little sad I wasn’t up for the mentor summit.
  • Stem can now download microdescriptor consensuses.
  • Helped Roger investigate relays failing to serve current consensus data.
  • Moved DocTor from Tonga to Bifroest among other monitoring updates.
  • Neat non-tor stuff. Dota Internationals, Midsummer Renaissance Festival, and vacation up to Bellingham for the Sparks Museum and Boeing Factory Tour.

Hi all! Getting in lots of family time as I sort out ongoing health issues, but none the less got some neat things to report this month!

Interpreter Panel

Part of Sambuddha’s GSoC project, half my month has gone toward reintroducing nyx’s interpreter and I’m pleased to say it’s turned out great!

Built upon Stem’s tor-prompt this expands upon the capabilities of the last arm release, providing an interactive python interpreter along with updated tor capabilities.

Curses Test Coverage

Concluding a four month overhaul, nyx now has a high degree of test coverage for its curses display capabilities which is good cuz… well, that’s kinda what nyx does. To answer the obvious question no, still no release date but this is a major milestone. Remaining work I have planned includes…

  • Refactor nyx’s menu and controller modules.
  • Put together a new site for nyx.
  • Run an open beta to solicit testing and ideas from our community.

Stem was in the works for two years before its first release. Nyx too will be ready when it’s ready.

  “Always do things right. This will gratify some people and astonish the rest.” -Mark Twain

Summer is here! Family, festivals, and other totally-not-tor things are occupying much of my time so this is gonna be short…

  • Expanded Nyx tests to include the graph and log panels. With this we have coverage of 66% of the curses components.
  • Stem support for new tor additions, highlights of this month being shared randomness and ADD_ONION basic auth.
  • GSoC midterm evaluations – everyone passed!

That’s all – now off to enjoy the sun!

Eeek I’m late! Wanted to get the PyCon trip report out first but… ok, maybe I went a tad overboard there. Oh well, better late than never. Between summer festivals and health issues tor took the back seat again this month but still some neat stuff…


Probably wrote way too much on this already so I’ll spare ya. If you haven’t skimmed the report yet then check it out – PyCon was a neat event!

Nyx Event Selection Dialog

Remember arm’s bizarre and clumsy event selection dialog? Remember how confusing it was?

Yeah, it sucked. Sambuddha and I have been tossing pull requests back and forth all month as part of his GSoC project. This is requiring a lot of my direct involvement but oh well, the new dialog has turned out nicely!

Few other noteworthy things…

Hi all, health issues nailed me for much of April but ick aside here’s what I was up to last month.

Google Summer of Code

GSoC planning is done and we’re delighted to announce seven great projects for the summer!

In particular I’ll be mentoring Sambuddha who’s making several Nyx additions that’ll make our upcoming release even better!

Nyx Curses Testing

User interfaces are tough to test. This goes for websites, GUIs, as well as terminal UIs. Often there’s libraries for doing so but honestly they’re never exactly what I’d call ‘elegant’.

As the rarest of the three terminal UIs are particularly under-served when it comes to testing, which has left Nyx a black hole when it comes to code coverage. But that all changed this month when I had a bit of an eurika moment.

So… we can now test Nyx’s UI! This month I added test coverage for our popups and header panel, with ongoing work for the rest of the interface. This is slow going and will no doubt gobble my May but I’m really delighted with the direction it’s going.

Few other noteworthy things…

  • Replaced tor-assistants@ with a new list to cut down on spam we all need to deal with.
  • Been working on a new website for Nyx during my commutes. Still a ways off but coming together bit by bit.

Hi all, March has been a busy month! My fingers are in quite a few pies but they’re all still baking so this is gonna be a short one…

Google Summer of Code

Sweet, delicious chaos! Forty-five applications rolled in this year which needless to say is keeping us busy. Best of luck to everyone that applied!

Selection announcement is April 22nd.

Nyx Curses Usage

Still a work in progress but I’m abstracting direct Curses usage out of Nyx. Short term this provides better thread safety, and long term could allow direct Windows support through PDCurses. Work on this will continue all through April.

Overhaul aside, brainstormed project ideas with GSoC applicants including…

Also cleaned up our ticket queue and made Nyx a bugtracker page.

Few other noteworthy things…

  • Stem now provides shorthand aliases for fetching descriptors. This makes scripting as simple as…

    import stem.descriptor.remote
    print 'Current exits are...\n'
    for desc in stem.descriptor.remote.get_consensus():
      if desc.exit_policy.is_exiting_allowed():
        print(" * %s: %s" % (desc.nickname, desc.exit_policy))

  • Better descriptor validation of non-ascii content.
  • Worked with Sebastian, who’s writing a guide for using Stem to better test Tor. He found some interesting testing issues on OSX..

Titan II, V2, SR-71 Blackbird, not to mention the Spruce Goose. If you’re ever in Oregon I’d highly suggest the Evergreen Aerospace Museum. It’s truly impressive. Bonneville Dam was fun too, but I’m kinda a sucker for big feats of engineering.

Fun trip aside February was a productive month!

Google Summer of Code

We’re in!

Org administration is taking a big chunk of my time and will continue to do so through March, but things are coming together nicely. Student applications are due March 25th. Many thanks to Isabela, Roger, and Sebastian for helping with the proposal!

Nyx Torrc Panel

Last panel’s done! In doing so we’re dropping torrc validation which has been the #1 point of confusion for arm users on irc. As always the rewrite came with deletion of what we in the biz call ‘a crap ton of code‘. Still lot more work before we release, but getting closer!

Few other noteworthy things…

This was a great start to the year! Highlight was lunch with David White, author of Battle for Wesnoth who also gave me a tour of Valve. But January had lots of neat Tor stuff too…

Nyx Connections Despite DisableDebuggerAttachment

For years Tor’s DisableDebuggerAttachment has been the bane of Nyx. The feature wasn’t intended to effect us, but screws with proc permissions breaking every connection resolver we have.

Resolvers read /proc//fd to get connection inodes, then use that to determine what from /proc/net/tcp belongs to our process. Tor’s DisableDebuggerAttachment breaks that by making /proc//fd only readable by root. However, even without knowing the inodes we can identify Tor related connections by if they go to a relay or our Tor ports. This is exactly what Nyx already does to identify a connection’s type.

TL;DR. Connection resolution now works all the time. Only drawbacks are…

  • This will blend connections when running multiple Tor processes.
  • Connection resolution can’t work until we have consensus information. This can take a few seconds when starting up.
  • We can’t show client or exit connections. Nyx already scrubbed these so no big loss, but means we now can’t even show that they exist.

Small drawbacks to have the connection panel work by default once again. User can still set ‘DisableDebuggerAttachment 0’ in their torrc for more reliable connection resolution.

IPv6 Connection Resolvers

Thanks to toralf Stem can now retrieve IPv6 connection information. More important for users, this means Nyx’s connection panel now works for IPv6 relays!

Few other noteworthy things…

Happy holidays everyone! Between family and tenacious colds December is always a slow month, but I’m pleased to announce another milestone toward Nyx’s next release!

Simplified from a 611 line monstrosity to trivial 346 line panel, Nyx’s configuration editor is the latest part to get some love. Simpler codebase aside, this overhaul greatly improves startup time. In its last release arm required multiple seconds the first time it was run to bootstrap this panel. This is no longer the case.

And really… that’s it. Faster, simpler, and otherwise a drop-in replacement. Next up is the last (and simplest) panel which presents the torrc. Honestly we’re getting kinda close to being ready to release which is exciting. First time arm users have gotten a new toy since 2012!