Trip Report

I’ve been to quite a few conferences. LinuxFest Northwest, SeaGL, PETS, Toorcamp, Defcon (prior trip report), but PyCon was particularly impressive. At over three thousand attendees with five parallel tracks of talks the word ‘busy’ hardly seems to do the conference justice.

Top TL;DR highlights for me were new capabilities in the Python 3.x series and HTTP 2.0. In particular…

  • Python 3.6 releases on Christmas, finally adding string interpolation!
    >>> name, job = 'Damian', 'software engineer'
    >>> print f'{name} is a {job}'
    Damian is a software engineer
  • Python 2.x support will be completely discontinued in 2020.
  • New async/await keywords in Python 3.5 provide built-in support for Twisted-style async IO.
  • Gradual type syntax in Python 3.5 makes code even more self-documenting and supportive of static analysis.
  • First major protocol update since 1999, HTTP 2.0 is now supported by all modern browsers and 60% of users in the wild. Connection multiplexing allows all site assets to be retrieved over a single connection, improving latency on the order of 50%. The new protocol also negates any need for the clever performance hacks we’ve developed over the years like asset minimization and sprite maps!

PyCon 2017 will be in Portland one more time before moving on to another venue, so if the following sounds interesting then check it out!



Serendipity is delightful. My first time taking the train, I strongly suggest Amtrak (particularly the Coast Starlight) if heading down to Portland. Comfortable, scenic, and by happy coincidence sat with Sarah Leivers: PyCon speaker with roots in the UK deaf community.

Sarah made the interesting point that even for deaf communities in English speaking countries English is often a second language. Signing is their native tongue, putting them at a disadvantage when it comes to involvement in our communities. Part of the larger ESL puzzle, our discussion was a nice reminder of why it’s important to keep documentation as linguistically simple and accessible as we can.

In the observation car the Parks Department described sights we passed, my favorite being the Centralia train station. Completed right around the time these newfangled ‘airplane’ things were taking off, to celebrate they decided to christen the building with champaign. Three bottles were loaded onto a plain and dropped. The first couple bottles missed but the third hit dead on, puncturing right through the roof.

Spoiler alert: this was the last building they christened in such a way.

Go to a conference without exploring the area and you’re doing it wrong. My train left me a few hours to explore the city, starting with the Portland Saturday Market. Easily comparable to Pike Place, the market is four city blocks jam packed with all the essentials of life: hand-carved bark houses, tie die, and of course fancy hats!

Next hit the Lan Su Chinese Garden, beautiful gem nestled into the heart of downtown…

Of course visited Ground Kontrol just a block away. Classic arcade that successfully reminded me just how much I suck at Marble Madness. In my defense haven’t played since my good old Amiga 2000…

Finally, hidden below my hotel lurked a black light pirate themed putt-putt course. So… seems that’s a thing!

With that out of the way, on to the conference!

File Descriptors, Unix Sockets and other POSIX wizardy

First talk of the first day, Christian Heimes gave a crash course on *nix file descriptors. In python descriptors are fetched with f.fileno() and Christian demoed interacting with them directly to open his cd tray.

Christian’s talk focused on file descriptor basics (which honestly I’m rustier on than I should be)…

  • Descriptors 0-2 are reserved for stdin/stdout/stderr with -1 for errors.
  • Fork clones the current process while pointing to the same global entry.
  • Exec replaces the current program, inheriting the prior descriptors (which is why pipes continue to work).
  • Descriptors can be delegated. This is useful in sandboxing situations like seccomp, allowing a broker to open files/sockets on a sandboxed process’ behalf.

Lastly Christian walked through a little strace example that illustrates how descriptors are used in a basic scenario…

% cat
with open('/home/atagar/Desktop/') as my_file:
% strace python
open("/home/atagar/Desktop/", O_RDONLY|O_LARGEFILE) = 3
read(3, "with open('/home/atagar/Desktop/"..., 4096) = 80
read(3, "", 4096)                       = 0
close(3)                                = 0
write(1, "with open('/home/atagar/Desktop/"..., 81) = 81

Refactoring Python: Why and how to restructure your code

Nice presentation by Brett Slatkin, the author of Effective Python on how and when to make code more maintainable. As developers we optimize for making things work in our first pass, and for many of us that’s where the story ends. To make code that’s truly easy to follow requires time and patience to take follow-up passes that optimize for maintainability. Something most developers don’t do.

To illustrate this Brett asked: how much of your coding time goes toward implementation? 90%? 75%? The few developers he knows that write easy to follow code only do so because they spend fully half their time refactoring anything they write. Maintainability isn’t cheap, and when faced with deadlines it’s often the first thing to go.

Brett’s other main takeaway was that without tests you’re DOA. Refactoring requires a willingness to make mistakes, and without high coverage any major overhaul of production systems is in practice impossible.

This dovetailed nicely with the following talk, Code Unto Others, which gave a few tips…

  • When it comes to maintainability remember that you don’t scale. Any rough code you write is something you’ll need to explain over and over to engineers that touches it. That’s not really how you want to spend your time, is it?
  • Commonly people can track 5-9 things at a time which is why phone numbers are seven digits. Subdivide modules to take advantage of this. As a counter-example they used Mercurial’s Repository class, a 17,000 line headache for newcomers.
  • Be wary when describing your module uses the word ‘and’ (“it does this and that”). If you need that word you’re probably doing it wrong. After reading the first half of a class you should be able to take an educated guess at what you’ll see in the second.

Finding closure with closures

Peek under the hood at how Python implements closures…

>>> def print_greeting(first_name):
...   def msg(last_name):
...     platform = os.uname()[0]
...     return "Hi %s %s, you're running %s" % (first_name, last_name, platform)
...   print(msg('Johnson'))
...   print("co_varnames: %s" % ', '.join(msg.__code__.co_varnames))
...   print("co_names: %s" % ', '.join(msg.__code__.co_names))
...   print("co_freevars: %s" % ', '.join(msg.__code__.co_freevars))
>>> print_greeting('Damian')
Hi Damian Johnson, you're running Linux
co_varnames: last_name, platform
co_names: os, uname
co_freevars: first_name

varnames are local variables while freevars are variables we’re closing over from the outer scope. A gotcha that’s probably bitten every python dev is that assignment to a closed over variable overwrites it with a local…

>>> def get_score():
...   total = 0
...   def add_points():
...     total += random.randint(0, 5)
...   for i in range(3):
...     add_points()
...   return total
>>> get_score()
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 6, in get_score
  File "", line 4, in add_points
UnboundLocalError: local variable 'total' referenced before assignment

Python 3.x adds a new ‘nonlocal’ keyword for re-binding closures but for those of us stuck in the past our best option is to use the mutable hack. Gross, but it works.

>>> def get_score():
...   total = [0]
...   def add_points():
...     total[0] = total[0] + random.randint(0, 5)
...   for i in range(3):
...     add_points()
...   return total[0]
>>> get_score()

What is and what can be: an exploration from ‘type’ to Metaclasses

Owww, my head. This and another talk the previous day by Mike Graham introduced audiences to the wonderful world of python metaclasses…

    “The subject of metaclasses in Python has caused hairs to raise and even brains to explode.” -Guido

Method for redefining the fundamental behavior of objects and in doing so tear the fabric of reality, metaclasses are what you invoke each time you extend object. Dustin demonstrated this by defining his own metaclass that transparently causes method invocations to be accompanied by a bark…

from functools import wraps
from inspect import isfunction

def bark(f):
  def wrapper(*args, **kwargs):
    return f(*args, **kwargs)

  return wrapper

class MetaDog(type):
  def __new__(meta, name, bases, attrs):
    for name, attr in attrs.items():
      if isfunction(attr):
        attrs[name] = bark(attr)

    return type.__new__(meta, name, bases, attrs)

class Dog(metaclass = MetaDog):
  def sit(self):

  def stay(self):

d = Dog()

So why will you use this? Well… hopefully you won’t. Besides the obvious unforgivability of this sin upon your coworkers, this is the kind of black magic Ruby folks do all the time but Python devs know better. Like redefining builtins, just don’t.

That aside, it was interesting to learn a little more about the abstract base class module and how python works under the hood.

Building protocol libraries the right way

Cory Benfield, author of Requests, urllib3, and other core I/O libraries discussed a common pitfall that inflicts protocol libraries: mixture of I/O with parsing.

Python has as many HTTP parsers as there are I/O libraries. Urllib variants, aiohttp, Twisted, Tornado, and friends all reinvent this wheel. Code re-use is particularly great when you have a well defined problem with a single correct solution. Arithmetic, compression, and parsing are all examples of this, so why don’t they all share a unified parser?

The problem is that we tangle network I/O with parsing of the messages we read. As such all these projects trip over the same obscure edge cases and re-implement the same optimizations.

Cory’s message was simple: keep parsing separate. Besides code reuse this greatly improves testability because you don’t need to invoke your I/O stack for coverage.

Personally I found this talk interesting because this is exactly something I ran into with Stem. To work our I/O handler needs enough understanding of the control-spec to delimit message boundaries, but beyond that parsing is a completely separate module. This has been a great boon for testing…

250 OK"""

def test_single_getinfo_response(self):
  Parses a GETINFO reply response for a single parameter.

  control_message = stem.response.ControlMessage.from_str(TEST_MESSAGE, msg_type = 'GETINFO')
  self.assertEqual({'version': b''}, control_message.entries)

HTTP can do that?!

Whimsical look at lesser known bits of the HTTP specification…

  • Need just metadata of a GET request? Use HEAD instead for a far lighter response.
  • Calling OPTIONS will tell you the HTTP operations a resource supports.
  • Besides normal CRUD operations (GET, POST, PUT, DELETE) the HTTP spec has PATCH to update just part of a resource.
  • The specification also has TRACE, LINK, and UNLINK methods. Nobody uses them but hey, they’re there.
  • Few interesting headers include ETag for versioning resources, If-Modified-Since to only solicit a response if the resource has changed, and Cache-Control to define cacheability. Actually, the specification even has a From header in case you want to tell everybody in the world your email address…
  • Few standard but infrequently used response codes are…
    • 410 – That resource used to be here but now it’s gone.
    • 304 – You asked to get this resource if it’s been modified but it hasn’t.
    • 451 – Unavailable for legal reasons. Mostly comes up with censorship firewalls.
  • Unsurprisingly you can make up your own status codes and reason strings. Sumana had several amusing ones she’s found in the wild.

Playing with Python bytecode

Amusing demonstration of executing raw bytecodes in python, including runtime manipulation to switch a functor’s addition operation to multiplication. Interesting in a ‘oh god, you can do that?’ sense but even the presenters said ‘kids, don’t do this at home’. Few (if any?) practical applications, and opcodes change even between minor Python interpretor version bumps making any such hacks a maintenance nightmare.

SQLite: Gotchas and Gimmes

Tips by Dave Sawyer for SQLite, mostly focusing on the advantages over pickles (performance, safety, etc), common pitfalls, and locking strategies…

  • Deferred – Multiple readers/writers.
  • Immediate – Multiple readers/single writer
  • Exclusive – Single reader/writier.

WAL (Write Ahead Locking) is an alternative where readers are unlocked with the writer appending deltas. Upon checkpoints SQLite halts all reads/writes to apply the deltas as a batch.

See Python, See Python Go, Go Python Go

Last talk I attended and the one I wanted to see most. Imagine a world where performance critical code could be written in Go rather than C. No more memory leaks. No compilers. Sounds great, right? Well, keep dreaming.

Both Python and Go can drop to C and Andrey gave a demo of doing so as a bridge between them, and in the process explained why this is a terrible idea. The CPython Extension interface requires a bit of boilerplate but can work with no dependencies while CFFI requires some magic but provides a more portable solution. But in either case crossing both the Go-to-C and C-to-Python boundaries drop you to the least common denominator. This means no Go interfaces or routines, and no Python classes or generators.

GC, GIL, and JIT all add their own headaches but worse, you need to implement your own memory management. Sharing between Go and Python risks release of memory the other side still references. Andrey got around this by passing his own dereferenceable pointers but… ick.

In the end Andrey’s demo worked and in fact was just as performant as a direct Go implementation, but made it clear there be dragons. Frustratingly, it’s still better to just call os.system().



This being my first PyCon I focused on talks rather than the hallway track but none the less had some nice finds…

  • Seattle is home to quite a few technical meetups. Hardware hacking, TA3M, Ruby Brigade, you name it and there’s probably a group for it. SeaPig has been a fun local python group but sadly its gone dormant in recent years. Among the booths however I ran into members of PuPPy, another local python group that seems to be quite alive and well!
  • Didn’t realize in advance but AWS networking ran a booth during the job fair. Fun chats with Shawn – he has a great approach for exciting folks to apply.
  • Crossed paths with meejah several times. Together we whipped up a recipe combining our libraries so users can read stem-parsed event objects from txtorcon. Neat stuff!

Simply a great conference, I look forward to hitting PyCon again next year!

As per conferencing tradition Friday was spent on travel and meeting the other attendees. Some of the highlights for me were…

  • David from KDE

    Besides demoing some KDE eye candy we discussed their project infrastructure. KDE is a federation of smaller projects and had over sixty students this year (ten times the number mentored by Tor).

    Their project’s scale has led to some unusual infrastructure decisions. For instance, they have a partly decentralized git infrastructure where pushes go to a single master host and pulls are from any of several mirrors. The config they use to do this leads to some… odd behavior. For instance a ‘git pull’ updates your tracking branch but not the origin branch reference. The result is that to do a pull for realz you need to call *both* pull and fetch. No doubt they also get fun behavior from mirroring delays…

    We also talked a bit about post-review and defaults they could set to better support their setup. KDE has the largest public ReviewBoard instance, but the above git setup makes it a bit confusing to use.

  • Sukhbir from Debian

    In 2011 Sukhbir applied to us for GSoC to work on TorBirdy. We loved his proposal, but due to prior commitments he ended up working with Debian instead. Since then he has become a GSoC mentor for Debian and involved with the Tor by implementing his earlier proposal for TorBirdy.

    Sukhbir’s interested in getting even more involved with Tor so we discussed other projects that might interest him, and ways that we could better publicize TorBirdy on our site.

  • Arc from Python

    When I found out that Python had a mentor at the summit I made a mental note to hunt him down and ask about packaging best practices. After an unexpected discussion about rugby I found out that it’s actually easy to support both python’s 2.x and 3.x series by including a 2to3 conversion at build time. This can be done via either distutils or distribute.

    I also asked him to look into his crystal ball for when python 3 would take over the world and he said ‘Next year. Ubuntu and Fedora are ready and willing to make the switch. The last main holdout is Gnome. They tried to migrate but work there isn’t finished yet.’

Saturday was the first day of the unconference. After an amusingly confused attempt to have each of the couple hundred attendees shake each other’s hand there were sessions. Some were a little interesting, but I spent more time on the hallway track since that’s the real benefit of the summit. The only useful tidbits I got from the talks were…

  • Do outreach early. The successful GSoC students who stick around tend to be the ones that get involved before the application phase. We should try harder to recruit college students to hack on tor, with the carrot that this’ll give them a leg up when applying for the program. OpenHatch might be something to look into for this. This would be a nice task for a community manager if we get one…
  • Google Code In is a program somewhat similar to GSoC where highschool students become involved with open source. Last year they had 18 organizations and this year they’re narrowing it down to 10. I was already highly tentative about having us apply and now that I’ve heard more I’m sure we don’t have enough bandwidth for the hand-holding this would require.

As for the hallway track…

  • Adriano and Luis from Umit

    Last year Adriano showed me Open Monitor, a censorship detector written in python. Sounds familiar? I thought so too, and tried a few times to get them to talk with Ooni Probe and vice versa without success. My impression is that they’re UI developers (a skillset we sorely lack in the tor project) with a rather unscalable backend, while Ooni Probe’s backend is far more mature but lacks any sort of UI for rendering real time censorship information.

    I made another stab at getting the two projects to talk, after which the meeting took a weird turn with Adriano arguing that ‘some censorship is good’. Evidently they decided that Open Monitor won’t look for censorship concerning ‘porn or terrorism’. I argued that this was a slippery slope and that censorship monitoring shouldn’t try to pass a moral judgment on the content being censored, but after a time it was clear that we were talking past each other.

    I still think that we should leverage their UI expertise, but that’s up to the Ooni Probe devs.

  • Open Source Lab

    Met with a couple administrators from the OSU’s Open Source Labs. They provide hosting for several of the largest open source projects including Apache and the Linux Foundation. Mostly we talked about amusing legal threats they get for hosting the phpBB project. Evidently lawyers are quite skilled at clicking the ‘this is a phpBB forum’ link followed by ‘hosted by the OSL’ before sending their angry emails. We also talked a bit about setting up non-exit relays. They might be pretty receptive to this if we want to follow up.

  • Sumana and Rob from Wikimedia

    Unsurprisingly Wikipedia occasionally has issues with spammers using Tor. We talked about some possible options, such as requiring accounts for Tor users to edit with a sort of proof of work in account creation to make ban evasion more of a pita.

  • Terri from Python

    Mailman 3 is coming, and with it an interface that *doesn’t* look like it came from the 1980s! Most importantly for us, the new version of Mailman provides a forum interface, letting email and forum users communicate by whichever method they prefer. This would be a good answer to our forums ticket. She estimates that it’ll be ready in six months or so.

Flying out on Sunday cut my day short, but there was one session that I thought was interesting. Gnome and Wikimedia are launching a program similar to GSoC to encourage more women to get involved with open source. It runs later this winter. One gotcha is that Google’s not involved so mentoring orgs need to cover the $5k stipend.

I like the idea. Is this something we want to take part in? If so then I’d be happy to administer the non-financial parts of it.

Hi all. When tor developers go on trips we commonly write a trip report afterward. My recent trip to Defcon didn’t contain anything sekrit so sending the notes I took here. If any of these presentations sound interesting to you then videos should be available later on the Defcon site.

Cheers! -Damian

PS. I was new to many of the things being presented so my notes quite likely have some inaccuracies. Don’t quote me on anything – watch it for yourself if interested.

tl;dr Only presentations related to the privacy field were…

  • Can You Track Me Now?

    Panel discussion on mobile GPS tracking with lots of interesting stats.

  • Crypto and the Cops

    EFF presentation on the 5th amendment protections and how it applies to compulsory decryption.

Day 1

Las Vegas is… an unusual city. Like most people I knew I wasn’t in Kansas any more when I saw slot machines in the airport. By day the city’s not much to look at. Mostly just a bunch of dusty buildings baking in the 110 degree desert heat. But by night the whole city illuminates in neon lights making it quite a spectacle.

My first night I explored the strip and several of the larger casinos including Bally’s, Bellagio, Caesar’s Palace, and the Rio where the conference was taking place. They were… enormous. Caesar’s in particular took me almost a half hour simply to walk the length of it. Probably my favorite sight was the Bellagio fountains which had a magnificent water show choreographed to ‘Singing in the Rain’. It was very classy.

Day 2

The second day was the first of the conference and mostly consisted of two things: standing in line a couple hours for registration and introductory talks. The talks were just a single track and included…

  • Wireless Security: Breaking Wireless Encryption Keys

    Handshake process for WPA, WEP, and WEP2 among other things. I was still sorting out the material that I got during registration so I didn’t pay very close attention which was a pity since I could use a refresher.

  • Intro to Forensics: Tools & Tactics

    A very brief introduction to nmap, tcpdump, netcat, ntop, and Metasploit. Nothing new to me here though it was a nice reminder that I know less about them than I’d like. I haven’t played with them since grad school and should try them again when I get home…

  • Cerebral Source Code

    Presentation on social engineering. Sadly missed a large part of this one since I still hadn’t had breakfast. The QA portion that I saw was fun, mostly questions about how he’d extricate himself from various sticky situations. His answers were clever, as well as entertaining. My favorite…

    Q: What would it take to have more trust in this community?
    A: LSD?

  • DC101: Defcon Survival

    This was a panel discussion with general advice for enjoying Defcon. Mostly it boiled down to ‘you get what you put in’ which caught me a little off guard. I came here expecting to watch some talks, see Vegas, and schmooze with other conference attendees. Defcon, however, aims to be a lot more interactive than that and the organizers put a lot of effort into events that weave throughout the conference.

  • HF Skiddies suck, don’t be one. Learn some basic Python.

    This one made me twitch. The speaker both gave a very bad tutorial for getting started with python (who the hell fires up *Eclipse* every time they want to write python?!?) and seemed to be incoherently drunk for most of it. Mostly I spent this time reading the program.

  • Hacking the Hackers: How Firm is Your Foundation

    This was a long checklist of various topics the speaker thought the Defcon audience should know about, largely focused on hardware hacking since that was his specialty. Interestingly ‘TOR’ was the first thing he listed. I always wondered how that spelling persists despite our efforts to stamp it out. Considering that this conference had roughly 20,000 attendees, each of whom will pass it along I guess now I know…

  • Fun with the Lockpicking Village

    A few years back I read the MIT guide to lockpicking and spent a couple days practicing against my door. Since then I’ve fallen out of practice and this was a nice reminder about both the basics and a few things that I didn’t know (how bump keys work, defeating deadlatches, how to use a can of compressed air to defeat magnetic locks that allow egress traffic, etc).

The hallway track wasn’t anything too notable. Met three developers with Metasploit and someone from the security group I attended at WSU. But that was about it.

Day 3

Second day of the conference I ran into someone from Amazon Infosec. A talk I wanted to attend concerning the attack surface of near field communication (NFC) devices was full, but several of the others were interesting…

  • Can You Track Me Now?

    This was my favorite presentation of the whole conference. A panel of four presenters, including Chris Soghoian, discussed geolocational privacy of mobile devices including local threats, historical information, and requests for real time tracking.

    Why does this matter, you ask? Because when it takes a team of agents to track your every step surveillance is costly. But when a single agent can track you and your every acquaintance from the comfort of his desk we get a whole lot closer to a 1984 style surveillance state real fast.

    Sprint now has a department of 200 people to service wiretapping requests. They also provide a self-service portal to law enforcement that has been used six to seven hundred thousand times a year, for 8 million GPS coordinates. This includes your every move going back 6 months to 2 years depending on your provider. 7 years in the case of AT&T.

    And no, none of this requires probable cause or a warrant. In other words, beware if your ex is a cop.

    Another eye opening bit of the talk was phone encryption. Apple has a master skeleton key for all iPhone devices so, when requested, they can send law enforcement a DVD with all the contents of your phone. Android is a little different in that they remotely reset your password to something that they can give the law enforcement agent.

    Beside three letter agencies, ad networks and apps are also a concern. 47 of the top 100 mobile applications send locational information to 3rd parties. For instance, ‘Best Alarm Clock’ sends locational information in the clear to 3rd parties whenever the app is used (somehow I doubt they need to do that for the alarm clock to function…).

    On a policy front one slight glimmer of hope comes from a recent case which ruled that placing a GPS tracking device on your car *does* constitute a search and hence requires a warrant (which means they kinda sorta need evidence of wrongdoing, *gasp*!). Don’t read too much into this, though. The majority opinion ruled this way because it technically constituted trespassing on your property.

    Just because you’re paranoid doesn’t mean they’re *not* out to get you…

  • Crypto and the Cops

    Marcia Hofmann from the EFF discussed 5th amendment protections and how they apply to being compelled to reveal your passphrase for an encrypted device. There have been five cases in this area so far, some that ruled each way in terms of compulsory decryption. Though not intuitive, the main determining factor in those cases constituted a gap in knowledge. If law enforcement knows exactly what they’re looking for and where it is then there’s less chance of the amendment applying than if the search is a fishing expedition. Moral of the presentation: never volunteer information, even if you’re innocent. Ask for a lawyer first.

  • Making Sense of Static

    Walkthrough of GPS signal acquisition, mostly focusing on how receivers account for Doppler shift and determine the code phase. The speakers also presented libswiftnav, an open source implementation of a GPS receiver system.

  • Life Inside the Skinner Box

    Cautionary presentation about the automation of law enforcement and where it might someday lead. This was pretty high level, discussing the societal implications.

  • Anti-Forensics and Anti-Anti-Forensics

    Discussion by a forensic examiner of several tricks that can draw out the length of an investigation (and by extension cost, increasing the chance of settlement). He also covered how they can be mitigated. Some of them included…

    • using a non-standard RAID configuration
    • randomizing file crated/modified timestamps
    • altering file header types
    • using restricted windows filenames (CON, PRN, AUX, etc)
    • owning lots ‘o media
    • modifying files provided with your system so the checksums no longer match what a default install would include (and hence adds to the haystack of things to sift through)

Before heading back for dinner I also caught part of a Skytalk. I didn’t find the presentation to be too interesting (brainstorming about mass data aggregation), but I got a chuckle from the sign on the door: “Absolutely no cameras allowed. Violaters will be violated violently.”

Day 4

I had less luck on the third day of the conference, hopping between several talks that weren’t particularly interesting. The best I found were…

  • World War 3.0 – Chaos, Control, the Battle of the Net

    Policy talk about Dubai’s upcoming WCIT negotiations (aka, the thing that’s gonna politicize the Internet). Most of the presentation was surprisingly fluffy, the only interesting bit coming from Dan Kaminsky who made the point that there’s generally four interests involved…

    • privacy
    • security
    • anti-piracy
    • sovereignty

    … with ‘reliability’ being the golden goose that no one wants to cook. His point was simply that these four camps war and forge alliances, privacy and security for instance sometimes at odds over anonymity but thick as thieves on other obvious fronts. He also said something like “We have made a system optimized for moving pictures of cats. And it’s *really* good at it. Though we’d like it to do more.” (likely horribly misquoted)

  • Exploit Archeology: Raiders of the Lost Payphone

    Step by step process of reverse engineering and reprogramming an Exetel payphone. Twenty year old software from a company that imploded without a trace long, long ago. Ye gods each step turned out to be a trek to Mt. Doom and back…

  • Off Grid Communication with Android

    Mesh network by introducing a transparent proxy after going through the Java networking interface. He wrote something based on the ‘Wi-Fi Tether for Root Users’ app to put his phones into ad-hoc mode then tried both the OLSR and BATMAN proactive routing protocols. He also tried a reactive protocol that broadcasts messages, then left routing decisions to the receiver which can trivially pick the shortest path though this had issues scaling above 200 devices or so.

  • Back Ops

    My favorite talk of the day, given by Dan Kaminsky. It was originally slated for two hours, but got crunched at the last minute into a one hour slot so he went pretty fast over five different topics including…

    • Timing Attacks

      Attacks can distinguish 15-100 ms deltas over the Internet and 100 nanoseconds over a LAN. Making everything take an identical amount of time is generally unrealistic since it forces everything to worst case performance. His view was not to let the perfect be the enemy of the good, and simply introduce random delay into requests.

    • Bad Random Number Generation

      First a rant that for twenty years security engineers have worried about entropy sources, yet we still haven’t gotten it right. The common sources of entropy are: hardware RNG, keyboards, mouse, and disk rotation. However… most servers don’t have any of these, especially with the move to virtual storage devices. Roughly 1/200 keys on the Internet are poorly generated, and he proposed using idiosyncrasies in clock timing as an alternative source of randomness.

    • Security Aspects of Languages

      Starting with the question ‘why is PHP so damn popular?’, he discussed why secure coding practices like prepared statements often aren’t followed. His conclusion: SDEs are in charge and they just want their service to work. The more of a pita security makes itself the more it’ll be viewed as an obstruction and hence circumvented. He then proposed more usable security patterns, for instance making the languages smart enough to translate queries like “SELECT name FROM users WHERE username=$id” into the proper prepared statement counterpart.

    • Censorship Detection

      After a brief mention of OONI-Probe he discussed a censorship detection method of his via the use of favicons. I saw this in an earlier presentation of his and he mostly skimmed through it.

    • Vulnerability Scanning

      Trying to answer the question of ‘how much of the Internet is vulnerable to X’, he used *stateless* TCP connections to scan hosts far quicker than he usually could. The trick was to skip retaining any information about outbound connections, making fire-and-forget TCP handshakes and leaving it to the other end to remember the state of the connection.

Day 5

I didn’t expect much from the lineup on the last day, but some of them were surprisingly good. I also ran into a couple more people from Amazon and one from Tor.

  • SIGINT and Traffic Analysis for the Rest of Us

    Very good talk on APCO Project 25 (P25), two way radio communication used by police, fire departments, federal enforcement, the secret service, and the DoD. A drop-in replacement for analog FM, this was first introduced in the early 1990s. It can optionally be encrypted, with federal/DoD users commonly enabling it and police/fire leaving it off. If you see an earpiece or bulky walki-talky then this is what it’s probably using.

    The speaker discussed several vulnerabilities in P25. Firstly, it uses a forward correcting protocol that is easy to jam. Usually a jammer needs to be stronger than the signal it’s jamming, but with P25 it’s sufficient to selectively block a message’s header to cause the rest of the message to be ignored. How hard is this, you ask? To cheaply jam a P25 signal you just need a $15 GirlTech IMME toy (it cost less to buy the toy than the transmitter that comes in it). As an added bonus it comes in pink or purple and comes with ponies on the box!

    Second, P25 has no notion of authentication. Anyone can transmit and user ids can be trivially spoofed. The user id of transmitters are sent in the clear, even when encryption is enabled.

    Third, any P25 device that receives a malformed signal will reply with the user id (again, in cleartext). This is invisible to the user and means that you can ping all P25 devices and triangulate their positions. Think of it as the “Marauder’s Map” for the secret service.

    Fourth, P25 is a narrow band radio broadcast (12.5 Khz) which is fairly easy to intercept via a scanner (such as an Icom R-2500). This makes encryption a must for private communications. However, the encryption of P25 devices has several usability issues…

    • Encryption is enabled or disabled via a switch on the handset. The only indication of if encryption is enabled or not is a “1” or “(/)” symbol on the handset. Which means ‘encrypted’ and which means ‘cleartext’ is evidently a common point of confusion.
    • The beep that the device makes to indicate that it’s in encrypted mode is the same that’s used to indicate a low battery among other things.
    • Encryption is transparent to the receiver, so if an individual in a group is broadcasting in cleartext then it’s impossible for anyone else in the group to know and correct them.

    • Rekeyed P25 devices become incompatible while being rekeyed, forcing users to drop to cleartext if any of the devices haven’t yet been updated.

    As a result, about 30 minutes per day of federal communications are sent in the clear. In a multi-city study on what this included researchers were able to gather the names of informants, license places of vehicles being tailed, etc. These snippets were also enough to reasonably figure out what investigations were going on at a given time.

    When brought up with the federal field offices encryption rates improved for a week or so, then dropped back to their prior ratio of cleartext. The researchers also discussed these with the manufacturers, who insisted that it wasn’t their fault if their users couldn’t properly use their devices.

    Btw, the only federal service to have a spotless record in terms of encrypting their communications? The US Postal Service.

  • SCADA Strangelove or: How I Learned to Start Worrying and Love the Nuclear Plants

    This was a talk on the security of SCADA HMI software (or lack thereof), which is used throughout several minor things like nuclear power plants. Originally to prevent plant operators from installing pong on the kiosks, HMI applications hasn’t evolved much since those humble origins. This talk was funny as hell, going over issues with several widely deployed HMI systems…

    • Microsoft Bob: Early attempt at a friendly, ‘helpful’ interface. If you made three incorrect password attempts you got a dialog saying “I see you forgot your password. Please enter a new one here…”. It also stores passwords in plaintext.
    • InvisiLink: Stored passwords are an xor with a static key. What is that key, you ask? “0123456789”
    • KingView: Passwords simply have their last byte subtracted from 0xff.
    • Iconies Genesis32 9.2.2: The login dialog includes a ‘challenge code’ that can be used as an alternative method of getting local admin access. To use this code the user is supposed to call customer service, who then provides the password corresponding with the code. Alternatively, you can take the first eight characters of the challenge code’s md4 hash to do the same.

There were also a couple talks on ‘firesheep inspired’ frontends for various exploits. The first did NTLM relaying against Windows corporate networks to allow for a variety of not-so-nice things like reading through another person’s Exchange inbox.

The second was a UI to automatically ARP poison a network then use sslstrip to snag credentials. I felt a little embarrassed for the speakers in this talk since they first tried to sensationalize what sslstrip could do. After that they claimed to have a stealthy MITM, then described the noisiest attack I can think of. Not only did they ARP poison the network and do SSL downgrading, but they did an nmap scan of every host on the network and, just for added stealthyness, blocked all tunneling protocols (ssh/vpn). The later was on the theory that “they’ll just use the local network instead”. Uggg.

So more annoying push-button solutions for script kiddies. Yay.

Ending Conclusions

Fun and interesting conference, but *damn* vegas is pricey. The trip was 1.5k and only $200 of that was the conference registration. I enjoyed it and it was a good thing to do at least once, but way too costly to do again.