Web API Lifecycles and Hypecycles

I've long been fascinated by the explosion of APIs over the past years, captured by the excellent ProgrammableWeb site.

Curious about how categories were evolving over time, I mined ProgrammableWeb's index for interesting patterns. I focused primarily on categories with at least 50 APIs, dividing them up into semesters from the second half 2005 to the first half of 2010. One important detail to be aware of: the PW index includes the last modified date of the API, not its creation date. So think of these graphs as a measure of activity in a particular category. For example an API may have been created in 2006 but if it was updated in 2010 it will count towards that last bar on the graph.

So what's hot? Social APIs, unsurprisingly, show a feverish activity: every site is busy creating or expanding their offerings in this space.

Enterprise APIs too are seeing a lot of movement.

Encouragingly, so is Shopping. A harbinger of an economic turnaround, or just wishful thinking? :-)

What about up-and-coming API categories to watch? Of the ones with over 30 APIs Travel and Utility have seen the most movement over the last year and a half.

Here are the remaining 13 categories with 50 or more APIs. Other strong performers include GovernmentTelephony, and Tools. Categories in relative decline? Reference and Video

DTerm: Useful Omnipresent Command Line for OS X

Found this sweet free utility called DTerm recently. It enables you to pull a context-aware pop-up that you can use to run command line utilities from whatever program you're currently using. What do I mean by "context-aware"? DTerm will automatically change directories to the one your program is currently in. Moreover, for those of us using multiple spaces, any programs you run from DTerm will open their windows in your current space.

Here's an example. Say I want to package up a bunch of images. Simple: hit Shift-Cmd-Return to invoke DTerm, its window overlay on the Finder's, and I can then run a "tar" command. That's it. I could even stay in DTerm and copy pics.tgz to a different drive, or scp it to another server.

This is a very useful tool. Here are a few other things you can do with it:
  • Quick calendar: "cal" will display this month's calendar, hit Shift-Cmd-C and you'll have it in your clipboard (cal 2010 will give you this year's calendar)
  • Starting TextMate: Typing "mate ." from a Finder window will open TextMate in project mode in the current directory
  • Comparing files: Select 2 files in the Finder, run DTerm, type "cmp" or "diff" then Shift-Cmd-V to paste the names of the files you selected into DTerm
  • MD5 checksum: Select the files you want to sum and run "md5" + Shif-Cmd-V
  • Info on all files in the current directory, including hidden ones: "ls -al"
  • Create a series of folders in the current directory: much faster to type "mkdir foo bar foo/bar" than to use the Finder
  • Quick lookup info on a domain: "dig google.com"
  • Want your mac to read you something? Select some text, copy it, invoke DTerm and type "say" followed by pasting the text surrounded by quotes
  • Byte, word, line counts: "wc" and the file(s) you're interested in

Not all these examples require DTerm's features but having a terminal window at your fingertips, without needing to switch context, is very useful.

And it's another reason to make better use of all those command line utilities!

Hat tip to @azaaza for the pointer.

Defcon Day Two Highlights

If there was a theme to the presentations I saw on Saturday, it's that as a technology is increasingly closed, its security decreases exponentially. The solution is sunlight: bring the products and their vulnerabilities out in the open. Yes, it does mean running the risk of vulnerabilities becoming known. But it's the only solution we've found that actually produces fixes. An obscure, insecure product helps only the black hats.

Insecurity Engineering of Physical Security Systems: Locks, Lies, and Videotape by Marc Weber Tobias, Tobias Bluzmanis, Matt Fiddler
A good example of this was a talk by three locksmithing experts. Though their preamble was too long, the main part of presentation was fascinating. They showed how to break five different types of locks: from a re-keyable mechanical lock to a fingerprint reading lock. All were defeated with simple attacks, some so simple that they beggared belief. The fingerprint reader, for example, has a standard bypass lock in case the battery runs out of the reader... With the insertion of the paperclip in the bypass lock, it opened like a charm. Wired has a great writeup, including videos.

Extreme-range RFID Tracking and Practical Cellphone Spying by Chris Paget
Chris gave two great presentations. The first showing how to read RFIDs at ranges of a couple hundred feet. The second focused on how to build your own GSM base station. Both talks were full of technical information and Chris did a good job at clearly walking us through the steps he'd taken. The GSM talk was fascinating. In essence, it is surprisingly easy not just to create your own base station (cost ~$3,000) but it's also trivial to spoof an existing carrier such as AT&T. When audience cellphones connected, Chris' fake tower would instruct them to drop encryption (a fact that handsets don't advertise to their users BTW) enabling the capture of phone conversations. While this currently only worked for outbound calls, it was still an impressive demonstration. One solution? Switch to 3G, it's a lot more secure than 2G.

We Don't Need No Stinkin' Badges: Hacking Electronic Door Access Controllers by Shawn Merdinger
This pres was a good example of the evils of security by obscurity. Electronic door access control is ubiquitous throughout the business world, yet these systems are usually run by building management. These folks may know a lot about physical security, but not information security. The result? Vendors supplying shockingly insecure systems that are never patched. Shawn focused on a product by S2 Security but claimed many competitors also had flaws such as insecure default configurations, full access to nightly database backups, an unprotected URL to reset the device to factory defaults, leveraging vulnerable software components, etc. etc. etc. Basically, if your company's door access controller is on an (internal hopefully!) network, you had best isolate it as much as possible. To my knowledge Shawn hasn't uploaded his pres anywhere here are the four S2 CVEs he submitted.

You're Stealing It Wrong! 30 Years of Inter-Pirate Battles by Jason Scott
A lighter look at the history of pirate groups and much much more. Scott, a computer historian and Defcon regular, gave a highly entertaining presentation and provided a wonderful trip down memory lane for many an audience member (myself included!). We gave him a standing ovation at the end of his speech (something I've rarely seen at Defcon). Jason, make sure you come back next year. Oh, and if you, dear reader, have old computer stuff you want to get rid of... Don't! Send them to Jason instead.

Malware Freak Show 2: The Client-Side Boogaloo by Nicholas J. Percoco and Jibran Ilyas
These two gents from Trustwave demo'ed four examples of malware found at client sites over the past year. Five years ago, they said, attackers focused on "smash and grab": find a vulnerability, exploit it, get as much info as you can, get out. Nowadays attackers are writing custom targeted malware that stays under the radar, allowing them to slowly infiltrate their victims' networks. Not sure what their sample size was but they claimed that on average malware infiltrates a site for 156 days before being detected. That's a long time.

Jackpotting Automated Teller Machines Redux by Barnaby Jack
Arguably the most talked about presentation at Black Hat and Defcon, Jack blew the doors wide open on ATM security. There are a lot of articles about his talk on the net, so I won't repeat it here. Jack basically found a number of vulnerabilities in these Windows CE devices (yes, Windows CE), including a remote exploit allowing him to reprogram the ATM. One of the most dramatic moments of his pres came when, in a matter of seconds, he popped open an ATM (cabinet master keys are apparently trivial to obtain), inserted an SD card with his own code, and power cycled the machine. Once the ATM booted you can see what appeared on the screen below and watch the video to see what happened next!

Defcon Day One Highlights

While a few of Friday's talks contained little new, original, or useful information (disappointingly the former Facebook CSO's talk was particularly inane), the majority of the presentations were interesting. A few were eye-opening. Here are some short summaries of my favorites.

Crawling Bittorrent DHTs for Fun and Profit by Scott Wolchok
Scott presented his research on creating a very comprehensive database of Bittorrent Distributed Hash Tables. Suffice it to say that his approach and findings will unfortunately prove very useful to record companies if they aren't already using these techniques. File sharers beware!

The Law of Laptop Search and Seizure by the EFF legal team
This talk focused on what law enforcement can and can't do (but may still try to get away with!) when seizing your laptop. There were a lot of details presented... orally. EFF, why no presentation? A few key points from my notes (oh, and in case you hadn't realized: IANAL!)
  • In general law enforcement can't just take your laptop and search it, your rights are protected by the fourth amendment
  • If law enforcement does want to search your laptop they need a warrant or you need to fall in a exception category such as: you have a public share on your computer, you're sharing via P2P, you've given consent, there's immediate danger that you might destroy the info, etc.
  • You can revoke consent at any time (i.e. if you first let law enforcement look at your laptop, you can change your mind)
  • If there are multiple users of a computer, any one of them could give consent, though courts have recognized that this consent only goes so far as the authorizing user has access (though the forensic tools they use make no such distinctions... Beware!)
  • All searches that occur at a border are considered reasonable. No suspicion is needed for any searches to occur, nor is a warrant needed (in other words: your rights go out the window!)
  • You cannot be forced to give over your encryption keys, courts have found that this is a fifth amendment right, and the gov't hasn't appealed this decision
  • Remote Computing Services, e.g. online backup or file sharing (like the very useful Dropbox). It is very easy for the gov't to get this data. They just need a subpoena, sometimes not even. Probably cause isn't required, since searching these cloud-based files often is how the gov't shows probable cause. They're not required to notify you within a reasonable time frame
  • Electronic Communication Services, e.g. online mail services like gmail. Your data is only protected for the first 180 days. After that the gov't doesn't need a warrant to get access to this info. However the gov't doesn't think this law applies to emails you've read, drafted, and sent. This is being appealed and the DoJ is fighting it. The EFF, ISPs, and others are trying to get a better law passed, maybe next year (the sooner the better!)
  • The EFF's advice: POP your mail, don't leave it in the cloud, and avoid online backups if possible

Lord of the Bing: Taking Back Search Engine Hacking from Google and Bing by Rob Ragan and Francis Brown
The most interesting talk of the day. These guys have taken google search engine hacking to a whole new level. Very creative. Sadly I haven't found their presentation online but the tools they wrote are. One of my favorite sections focused on combining google hacking with custom searches into a massive RSS feed for real time updates of vulnerable sites crawled by google. I'm sure we haven't heard the last of this...

Weaponizing Lady GaGa, Psychosonic Attacks by Brad Smith
Brad is an excellent speaker and by far the most entertaining of the day. He discussed the uses and misuses of psychosonics: the generation of (generally undetectable) sound patterns designed to alter a target's state of mind. One of the funniest parts of his speech came when he listed the top 10 sonic torture songs... :-)

Hacking the Defcon 18 Badge

Since its 14th edition, Defcon badges have gone electronic. Hardware wizard Joe Grand (he and I both worked at @stake a long time ago, though in different offices) creates these masterpieces and unleashes them on the thousands of people who descend upon Las Vegas every year for this oldest of the US hacker conferences, now in its 18th incarnation.

Befitting this conference, the badges have all sorts of hidden capabilities, easter eggs, etc. One of Defcon's many challenges is to find these backdoors. This year's badge is no exception. Sporting an LCD panel for the first time ever, pressing the badge's buttons causes all sorts of cryptic (and some not so cryptic) behavior.

One of the badge's challenges is to crack "Ninja mode" which you have to enable by picking an electronic lock consisting of fifteen tumblers, each one with three states (for a total of over 14million combinations).

I had fun with this one. I was making slow, steady progress until I thought of exploring the Defcon CD... Bingo! Joe was thoughtful enough to include a full development environment for the card, as well as the source code to the firmware! From that point "hacking" became a simple exercise in reverse engineering the code. I won't give the key away but I will say that Wolfram|Alpha proved very useful for quick conversions between binary, trinary, and hexadecimal.

In retrospect I should have looked at that CD much earlier :-)

27" iMac Electricity Consumption Stats

I pulled our handy little Killawatt out from its resting place this week and used it to track our 27" iMac's (quad core i7 processor) electricity usage. The Killawatt plugs in between the wall socket and the device you want to monitor. It will calculate cumulative power consumption (Kwh), watts, amps, etc. The device is particularly useful in figuring out how much an electric appliance, or computer, really costs to run.

27” Apple iMac
Kwh / Day
Cost / Day
On, screen dark
On, light usage
On, max usage

"Max usage" means all CPUs were chugging away and a DVD was playing. Cost / day is based on my current cost of just under $0.12/Kwh.

Overall that doesn't feel too bad, though it can add up over a year. If you were compressing videos 24x7 non-stop for a whole year it would cost you over $210 (and would probably seriously reduce the lifespan of your iMac to boot :-)