Status Report

Hi all! Getting in lots of family time as I sort out ongoing health issues, but none the less got some neat things to report this month!


Interpreter Panel

Part of Sambuddha’s GSoC project, half my month has gone toward reintroducing nyx’s interpreter and I’m pleased to say it’s turned out great!

Built upon Stem’s tor-prompt this expands upon the capabilities of the last arm release, providing an interactive python interpreter along with updated tor capabilities.


Curses Test Coverage

Concluding a four month overhaul, nyx now has a high degree of test coverage for its curses display capabilities which is good cuz… well, that’s kinda what nyx does. To answer the obvious question no, still no release date but this is a major milestone. Remaining work I have planned includes…

  • Refactor nyx’s menu and controller modules.
  • Put together a new site for nyx.
  • Run an open beta to solicit testing and ideas from our community.

Stem was in the works for two years before its first release. Nyx too will be ready when it’s ready.

  “Always do things right. This will gratify some people and astonish the rest.” -Mark Twain

Summer is here! Family, festivals, and other totally-not-tor things are occupying much of my time so this is gonna be short…

  • Expanded Nyx tests to include the graph and log panels. With this we have coverage of 66% of the curses components.
  • Stem support for new tor additions, highlights of this month being shared randomness and ADD_ONION basic auth.
  • GSoC midterm evaluations – everyone passed!

That’s all – now off to enjoy the sun!

Eeek I’m late! Wanted to get the PyCon trip report out first but… ok, maybe I went a tad overboard there. Oh well, better late than never. Between summer festivals and health issues tor took the back seat again this month but still some neat stuff…


PyCon

Probably wrote way too much on this already so I’ll spare ya. If you haven’t skimmed the report yet then check it out – PyCon was a neat event!


Nyx Event Selection Dialog

Remember arm’s bizarre and clumsy event selection dialog? Remember how confusing it was?

Yeah, it sucked. Sambuddha and I have been tossing pull requests back and forth all month as part of his GSoC project. This is requiring a lot of my direct involvement but oh well, the new dialog has turned out nicely!


Few other noteworthy things…

Hi all, health issues nailed me for much of April but ick aside here’s what I was up to last month.


Google Summer of Code

GSoC planning is done and we’re delighted to announce seven great projects for the summer!

In particular I’ll be mentoring Sambuddha who’s making several Nyx additions that’ll make our upcoming release even better!


Nyx Curses Testing

User interfaces are tough to test. This goes for websites, GUIs, as well as terminal UIs. Often there’s libraries for doing so but honestly they’re never exactly what I’d call ‘elegant’.

As the rarest of the three terminal UIs are particularly under-served when it comes to testing, which has left Nyx a black hole when it comes to code coverage. But that all changed this month when I had a bit of an eurika moment.

So… we can now test Nyx’s UI! This month I added test coverage for our popups and header panel, with ongoing work for the rest of the interface. This is slow going and will no doubt gobble my May but I’m really delighted with the direction it’s going.


Few other noteworthy things…

  • Replaced tor-assistants@ with a new list to cut down on spam we all need to deal with.
  • Been working on a new website for Nyx during my commutes. Still a ways off but coming together bit by bit.

Hi all, March has been a busy month! My fingers are in quite a few pies but they’re all still baking so this is gonna be a short one…


Google Summer of Code

Sweet, delicious chaos! Forty-five applications rolled in this year which needless to say is keeping us busy. Best of luck to everyone that applied!

Selection announcement is April 22nd.


Nyx Curses Usage

Still a work in progress but I’m abstracting direct Curses usage out of Nyx. Short term this provides better thread safety, and long term could allow direct Windows support through PDCurses. Work on this will continue all through April.

Overhaul aside, brainstormed project ideas with GSoC applicants including…

Also cleaned up our ticket queue and made Nyx a bugtracker page.


Few other noteworthy things…

  • Stem now provides shorthand aliases for fetching descriptors. This makes scripting as simple as…

    import stem.descriptor.remote
    
    print 'Current exits are...\n'
    
    for desc in stem.descriptor.remote.get_consensus():
      if desc.exit_policy.is_exiting_allowed():
        print(" * %s: %s" % (desc.nickname, desc.exit_policy))
    

  • Better descriptor validation of non-ascii content.
  • Worked with Sebastian, who’s writing a guide for using Stem to better test Tor. He found some interesting testing issues on OSX..

Titan II, V2, SR-71 Blackbird, not to mention the Spruce Goose. If you’re ever in Oregon I’d highly suggest the Evergreen Aerospace Museum. It’s truly impressive. Bonneville Dam was fun too, but I’m kinda a sucker for big feats of engineering.

Fun trip aside February was a productive month!


Google Summer of Code

We’re in!

Org administration is taking a big chunk of my time and will continue to do so through March, but things are coming together nicely. Student applications are due March 25th. Many thanks to Isabela, Roger, and Sebastian for helping with the proposal!


Nyx Torrc Panel

Last panel’s done! In doing so we’re dropping torrc validation which has been the #1 point of confusion for arm users on irc. As always the rewrite came with deletion of what we in the biz call ‘a crap ton of code‘. Still lot more work before we release, but getting closer!


Few other noteworthy things…

This was a great start to the year! Highlight was lunch with David White, author of Battle for Wesnoth who also gave me a tour of Valve. But January had lots of neat Tor stuff too…


Nyx Connections Despite DisableDebuggerAttachment

For years Tor’s DisableDebuggerAttachment has been the bane of Nyx. The feature wasn’t intended to effect us, but screws with proc permissions breaking every connection resolver we have.

Resolvers read /proc//fd to get connection inodes, then use that to determine what from /proc/net/tcp belongs to our process. Tor’s DisableDebuggerAttachment breaks that by making /proc//fd only readable by root. However, even without knowing the inodes we can identify Tor related connections by if they go to a relay or our Tor ports. This is exactly what Nyx already does to identify a connection’s type.

TL;DR. Connection resolution now works all the time. Only drawbacks are…

  • This will blend connections when running multiple Tor processes.
  • Connection resolution can’t work until we have consensus information. This can take a few seconds when starting up.
  • We can’t show client or exit connections. Nyx already scrubbed these so no big loss, but means we now can’t even show that they exist.

Small drawbacks to have the connection panel work by default once again. User can still set ‘DisableDebuggerAttachment 0’ in their torrc for more reliable connection resolution.


IPv6 Connection Resolvers

Thanks to toralf Stem can now retrieve IPv6 connection information. More important for users, this means Nyx’s connection panel now works for IPv6 relays!


Few other noteworthy things…

Happy holidays everyone! Between family and tenacious colds December is always a slow month, but I’m pleased to announce another milestone toward Nyx’s next release!

Simplified from a 611 line monstrosity to trivial 346 line panel, Nyx’s configuration editor is the latest part to get some love. Simpler codebase aside, this overhaul greatly improves startup time. In its last release arm required multiple seconds the first time it was run to bootstrap this panel. This is no longer the case.

And really… that’s it. Faster, simpler, and otherwise a drop-in replacement. Next up is the last (and simplest) panel which presents the torrc. Honestly we’re getting kinda close to being ready to release which is exciting. First time arm users have gotten a new toy since 2012!

Hi all! For years Nyx (aka arm) has done a neat trick where we describe what torrc options do and how they’re used. To do this Nyx had its own cobbled together parser for tor’s man page. Clearly a hack, but it worked.

That was all well and good, but we could clearly do better and now we have!

Stem manual information module

Besides filling Nyx’s needs the shiny new stem.manual module provides…

  • Tor test coverage. This adds several integration tests to confirm tor can properly build a valid man page.

  • Provides all Stem users with three methods for getting tor manual information…

    1. from_cache() – Retrieves information bundled with Stem. This is only as up to date as Stem itself, but the fastest and most reliable method.

    2. from_man() – Parses information from the local system by running ‘man tor’. Still fast, but obviously requires tor’s man page to be present.

    3. from_remote() – Retrieves the latest manual information from tor’s git repository. This is slow and shouldn’t be used without a fallback, but provides the most up-to-date manual information.

  • Along with tor’s manual information we provide brief, more user-friendly descriptions of all tor’s configuration options.

  • Parser is much improved over Nyx’s. In particular the stem.manual module has vastly improved performance, test coverage, and updated summary information.

Hi all! I’m heading home this next weekend for my favorite holiday (happy pumpkin day!), so sending this a tad early. Personally my favorite thing about dev meetings are the collaborative projects they inspire afterward. This month set aside Nyx in favor of working with the rest of ya.


Metrics Collaboration with Karsten

As part of our devious schemes to benchmark our various descriptor libraries I wrote a comparison of their capabilities.

Been a while since I used Eclipse. Got metrics-lib set up and sent a minor patch to note its dependencies. Next need to script up the benchmark scripts we discussed. Also discussed our CollecTor index and thanks to Karsten it’s now a thing!


List Administration

During our dev meeting we had several discussions around our email lists. This month set up tor-project@ to both…

  1. Improve transparency a tad by providing a place for our internal community to discuss things that don’t need to be so secret.
  2. Lessen the need to add everyone and their dog to tor-internal@!

Jury’s still out on if this’ll work, but I hope it will. Also followed up on the thread about trimming tor-internal@. The community seems pretty split on this so doubt anything will come of it.


DocTor and Directory Authorities

  • DocTor check for network latency to help surface issues like Faravahar’s ongoing sluggishness.
  • Took a quick stab at an IPv6 ping for authorities. Presently stumped on IPv6 support on cappadocicum.
  • Sending the fingerprint change notifications to bad-relays@ and bumped its threshold to cut down on false alarms.
  • Made a DirAuth wiki to help coordinate tickets. Was worth a shot but personally doubt this is gonna go anywhere.

  • Few other noteworthy things were…

    • Tor SoP concluded this month with everyone passing! Worked with Cristobal to settle on a license for Erebus and now in the process of moving it to tpo infrastructure.
    • Attended TA3M and SeaGL but sadly didn’t stick around late enough to hear Richard Stallman.