Pop quiz hot shot...

It's a chaotic Thursday afternoon. A series of bombs have gone off across your city paralyzing local transportation and infrastructure. You and co workers are trying to get information about loved ones and friends but your Internet link seems to keep going down. Local law enforcement has just entered your operations center saying that they're evacuating the building due to a suspicious van that's sitting unattended across the street...

...and you just got word that your website was defaced.

What do you do?

Sadly, it can't be as simple as "Call Keanu Reeves."

Tabletop exercises are incredibly useful from the perspective of Information Security folks: Drilling incident response so that it comes second nature can make the times when a crisis happens so much easier. However, all to often we focus on everything within our little silo: "A firewall just got down", "A server was compromised", "A worm is spreading across the LAN", etc. Bringing in a fresh set of eyes to look at exercises and suggest additional variables to toss into the equation can keep teams on their toes and start to think about problems beyond their area of control.

Recently, I was in a discussion with some peers regarding a recent exercise one of them had where there was a fire drill during the middle of it. While they were lamenting this, I asked "Well, what happens if there was a fire drill during an incident? Or worse, a fire?" seizing upon this dicussion followed and scenarios like the one described above started to form. They all boiled down to one question: What happens when the real world intrudes on your incident handling process?

While most businesses have some kind of Continuity of Operations plan, how many folks know how it applies to them? More importantly, how does it fit within your own procedures? Most IT pros know how to handle a virus outbreak on an internal network, but it's a completely different monster when halfway through the entire staff has to be evacuated due to a natural or man made disaster.

Some questions to consider (and discuss with your staff (and bosses)):

• What if something happens and your operations center is no longer able to be staffed during an incident?
• What's your chain of command? Who's in charge if there is a event similar to a Case Orange scenario with regular staff being incapacitated or unable to communicate?
• How do you contact people when regular lines of communications are down?
• Is security a stakeholder in the DR process?
• If all company sites are inaccessible and information is unable to reach your staff, where is your rallying point?

A lot of security professionals know that electronic networks are fragile and vulnerable, still, how many of us assume that telephones, cellular networks, and power will be up during an attack? Exercises should start preparing staff for a large scale disaster that may impact these services and designing processes to work around them.

For those of you who haven't been following the project, WebLabyrinth has gone through some changes and has had some new reporting and alerting features added to it. Recently, it was featured on a Technical Segment on PaulDotCom Security Weekly Episode 240 which was posted to YouTube your viewing pleasure.

WebLabyrinth is also featured prominently in Paul's "Offensive Countermeasures: Bringing Sexy Back" presentation that he recently gave at SOURCEBoston. If you're interested in learning more about WebLabyrinth and other "Offensive Countermreasures" Paul and John are teaching a class this Summer at Blackhat. If it's anything like his presentation, I highly recommend signing up.

Debian Squeeze was released last month and one of the new features was the faster dependancy based booting. However, if you attempt to upgrade your 5.0/lenny installation and you are running OSSEC, the bitchin free host-based intrusion detection sytem, the upgrade process doesn't go without a hitch: During the upgrade, the process to upgrade the System V init scripts barfs:

error: Unable to migrate to dependency based boot sequencing.
error: Problems detected: 
insserv: warning: script 'K20ossec' missing LSB tags and overrides, 
insserv: warning: script 'ossec' missing LSB tags and overrides

Fantastic. Squeeze and OSSEC have decided to be the two kids that don't get along on the local playground.

After some Googling, the main link that came up was someone asking on the OSSEC list asking "OSSEC Y U NO UPGRADE CLEANLY?" with no reply. Further research showed that the problem is that the stock OSSEC script doesn't have the Linux Standard Base init headers. Thankfully, Debian has a wiki page on how to LSBize your init script, which gives a great overview on how to make LSB compliant init scripts. 

So after taking a look at that along with the stock init script, whipping up a LSB header is fairly simple.

# OSSEC         Controls OSSEC HIDS
# Author:       Daniel B. Cid <dcid@ossec.net>
# Modified for slackware by Jack S. Lai
# Modified for Debian Squeeze by Ben Jackson <bbj@mayhemiclabs.com>
### BEGIN INIT INFO
# Provides:          ossec
# Required-Start:    $local_fs $remote_fs $network $syslog $named
# Required-Stop:     $local_fs $remote_fs $network $syslog $named
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# X-Interactive:     true
# Short-Description: Start/stop OSSEC Host Intrusion Detection System
### END INIT INFO

Simply replacing the exiting header with this This script does err on the side of caution for the dependencies: Does OSSEC need a DNS service to start? Who knows? But it solves the problem, the update script runs cleanly, and the system boots without a hitch.

Now, as the previous post stated, I love me some honeypots. So I got introduced to a fun tool on PaulDotCom Security Weekly Episode 225. During the show's technical segment, John profiled a simple python script called SpiderTrap. John is a big fan of honeypots and what he calls "offensive countermeasures", rather than simply blocking attackers that probe your network, John enjoys messing with them and keeping them busy until they give up and frustrated. SpiderTrap, written by his intern than Robish, reflects that methodology, it is a small web server that serves up a never ending maze of random links. When a web scanner encouters this, it attempts to map out the entire web application, which, theoretically, will never happen with SpiderTrap.

SpiderTrap is cool, however, I instantly thought of some shortcomings: First, it needs a dedicated host to run on, and second, it doesn't help your existing web applications. The gears started turning and it occured to me that the script would be fairly trivial to implement in PHP and that Apache's mod_rewrite could handle spoofing a never ending maze links. Thankfully, I had a few free nights coming up during the next weekend, so I coded something simple up.

What came out of my coding sessions was WebLabyrinth. WebLabyrinth, like SpiderTrap, is designed to create a maze of bogus web pages to confuse web scanners. However, unlike SpiderTrap, it can be run on any Apache server with mod_rewrite and PHP. This allows users to embed it within existing websites. Also, I added a feature which allows the web pages to consist of more then just a few links: It uses a Dissociated Press function to randomly generate text (courtesy of David Pascoe-Deslauriers' BSD licensed code). Included in the code is the first chapter of "Alice's Adventures in Wonderland" (which just seemed so right to use for this purpose), but any text can be used.

So, a few words of warning, while v0.2.0 does try to detect Google's spider and block it, it current detects no other search engines. Also, if an overly agressive scanner gets trapped, it may be a drain on system resources. I will try to address both these issues in future versions.

Now, partially for the above reasons and partially because I don't want to pay for some idiot scanning my server, there is no demo site available. However, Mr. Strand was very nice and created a video for PaulDotComTV showcasing the tool

It's a fairly simple idea and there isn't much to do from here, however, I am mulling some additional ideas for future versions. I am always open to suggestions, so please feel free to contact me.

While catching up on my podcasts, I was listening to PaulDotCom Security Weekly Episode 220. During the news stories they talked about an article in Network World regarding the effectiveness of honeypots in the Enterprise and how they were the best thing since sliced bread. Everyone agreed with the article and sung the praises of honeypots and how they almost always were great indicators of issues on the network and almost always caught malware propagating across their networks.

Honestly, this left me scratching my head. I have been running honeypots in an enterprise environment for over two years and they have been little more then novelties.

Now, don’t get me wrong: I love me some honeypots. When I first found out about honeyd back around 2002 or so I fell in love. I coded up an entire suite of scripts to mimic default installs of various Linux distributions. I used it in my university’s CTF competition to light up the /22 they gave us with over 700 decoy computers. I bought an extra IP for my DSL connection at my home solely for running it. I presented on it at The Last HOPE. It’s an amazing tool. When I got the green light at my work to set it up on the LAN for monitoring I was pumped! When running it at home I received an amazing amount of attacks! Surely the network, which was still thought to resemble the wild west would give us an amazing picture to how bad things are. The office had informal bets to see how long the box would last before it was exploited.

Well, that was over two years ago. We’re still waiting.

Honeypots are a double edged sword detection wise. When positioning them, a lot of folks place them in a DMZ open to the wild of the Internet. While this allows you to see exactly what is being flung at your network, most people don’t realize the amount of shit that gets rained down on your servers. In an average day one can see attacks from upwards of 20-30 hosts. This is wonderful, but correlating these to attacks on your infrastructure can be insanely time consuming. For each attack you need to ask:

• What was the attack?
• Do I have hosts vulnerable to that particular attack?
• Did this malicious host launch that attack against those servers?
• Was that attack successful?

While important questions, these alerts pile up. It’s very easy to end up chasing your tail and quickly become overwhelmed.

The flip side of this is positioning them inside your network. While you (hopefully) won't be overwhelmed with attacks, you'll run start running into the opposite where you'll be wondering if you'll be seeing any events. If you don't have some kind of automating alerting system, you may start not notice if and when an attack happens. Whoops.

Honeypots provide great information for network based attacks. For the scenarios described in the Network World article and above they function admirably. However, I feel comfortable in saying that over 95% of attacks coming into most networks are web based attacks that come in via one path: Users’ web browsers. Trojans like Zeus, Gozi, Sasfis, Koobface, and Conficker thrive in most modern enterprise environments. Recently there was a great post on the RSA weblog by Uri Rivner comparing the amount of data stolen by the Zeus Trojan and Wikileaks (Zeus wins. Handily). What do these crimeware kits have in common? They don’t probe other computers, they don’t scan, they just phone home to their C2 infrastructure. Honeypots are as useful in detecting and preventing these situations as a bowl of Tapioca pudding would be.

Like all security devices, honeypots have their place. Having nodes reporting into existing alerting structure (SIEMs) as tripwires and “patient zeros” are a good idea. However, with some of the price tags on these pieces of software one has to do a cost benefit analysis regarding their existing infrastructure. Would you be better served by a honeypot or hardware for an IDS system, another firewall, a SIEM, a log server, vulnerability scanner, or some other piece of security infrastructure? While they can and will catch certain attacks, however, they are one small piece in what you should be deploying and they should be one of the last pieces of your puzzle rather then your first.

As some of you may have noticed, ICanStalkU was down for a while, and came back with a new look and a few less features. For those of wondering what happened, lets set the wayback machine to last Wednesday, October 20th:

e-Mail: "This is a notice that your OAuth token for ICanStalkU has been suspended from interacting with the Twitter API."

Ben: "Oh, for %#$& sake..."

We were dead in the water. Our Twitter account was suspended, which while annoying, wasn't a big deal in the grant scheme of things. What killed us was the fact that we could not longer access the search API and therefore couldn't find photos. This left us with three options:

1. Shake our fists at the sky, pack our saddle bags, and ride ICanStalkU into the sunset.
2. Shake our fists at the sky, thumb our nose at the suspension, set up another account for API access, and try to keep ahead of any further suspensions.
3. Shake our fists at the sky, contest the suspension, and remain dead in the water while Twitter does whatever they do.

We decided that we should cast the die with option #3 and see what happens. After a couple days we got a response. Turns out, Twitter caught wind of our little party and decided to set fire to our tree fort. Apparently you can't @-reply people on an automated basis even if you do it only once an hour (despite other accounts that do the same thing) and the ICanStalkU website violated the "API Terms of Service by appearing to modify the content of tweets from their original text to text that the user did not explicitly generate"

We disagreed with their findings, but since Twitter is master of their own domain, complaining would likely get us nowhere. So, we went the other route and attempted to find an amicable solution. What we came up with was two major changes:

1. The account could no longer @-reply people.
2. The website needed to be changed so that it no longer appeared to be a tweet stream (no avatars, no text appearing to come from a user, etc)

So, we went ahead and made the changes, took advantage of the downtime to make some changes on the back end, and threw the switch. We would like to give a shout out to Brian, who, once we had our account reviewed, worked with us quickly to resolve the situation.

So, we're back up, stalking folks, and hopefully suspension free for the foreseeable future.

I was recently was asked to do a presentation regarding Information Security for his alma mater's IEEE. I believe that death by PowerPoint is one of the most painful ways to go out, so I tried to keep it light while staying on the subject manner.

• Malware, Big Hacks, and Stalking: Information Security at a Glance (PPT, 4.8MB)
• Malware, Big Hacks, and Stalking: Information Security at a Glance (PDF, 3.0MB)

Share and Enjoy!

It's been a whirlwind few weeks at the labs: First off, we have received plenty of media coverage regarding ICanStalkU lately: New Scientist, ABCNews.com, CNET, Forbes.com, and the BBC.

Next, ICanStalkU was presented at The Next HOPE.You can download the slides here:

• Locational Privacy and Wholesale Surveillance via Photo Services (PPT, 7.8MB)
• Locational Privacy and Wholesale Surveillance via Photo Services (PDF, 5.1MB)

The tools we released are available on Google Code.

One thing that you may notice in the presentation is that we talk about releasing a dataset of URLs of geo-tagged photos. This brings us to a slight change in our plans. After talking to people after the presentation, we've reconsidered releasing the dataset.

First off, while we still are committed that the idea behind ICanStalkU is a good one and public outreach is important, we were asked what purpose does the dataset serve and we couldn't come up with a good answer. It is a very interesting collection of information, but does it serve any purpose beyond publicly exposing user's locations? We're not sure. Also after some discussion, we think that releasing this information may change the conversation from "People are posting this information online so that anyone can get it!" to "Hackers are recording your location!", which, while technically true, gets away from the fact that anyone can do this and it hardly takes a degree in rocket science to do so. If a couple of security geeks, some open source tools, and a dirt cheap virtual server can amass a sizable dataset in a few months on a shoestring budget, imagine what could be possible if someone, say a corporation or a government decided to toss some money and time at similar project?

Also, as we are part of the "If you don't need it, why log it?" camp, we are also going to start regularly purging our database of any content older then 6 hours, so that if something happens and the data does get out into the open, the amount of people exposed will be fairly limited.

In summary, we think that the dataset crosses a line without having a tangible benefit. We apologize if you were looking forward to analyze it.

We recently changed our @ICanStalkU Twitter account to provide more than just statistics regarding the amount of photos we have analyzed. In addition to the statistics, we also started @-ing people who posted geo-tagged photos. In order to not be totally spammy and unrelenting we rate limited it to one unlucky person per hour and the person @-ed is the last person we analyzed. The choice is more-or-less luck of the draw, as we have no input on who is chosen.

The reaction to the replies has been mixed, but leaning toward positive. A lot of folks are not realizing what they are posting online and are learning about the risks thereof. The non positive reactions are mostly folks thinking that we are creepy people (which, while correct, it's not because of this) and a few folks who knowingly put the information online.

One such person who did this and was caught up by the trawler was Nik Butler (@loudmouthman) of the Social Media White Noise podcast, in which he mentions us in Episode 41. We believe there are two sides to every discussion, and Nik brings up some interesting points regarding the posting of such data, so we suggest you give it a listen. We would also ask you to ignore the bits where he calls us names. That made us sad :(

Nik brings up so good points, but we feel that some of them are wrong. His main contention is that all of this data is publically available elsewhere so that there is no need to censor such information posting online. He also makes statements that while we know where an individual photo was taken, we cannot make the statement “Oh, he’s at his house” all we can say is that “Oh, this phone was at Latitude X and Longitude Y at time Z” and that we should be focusing more on liberties rather than privacy.

First, let us state for the record that we fully support Nik and anyone else who chooses to put geo-tagged photos online. Nik is a perfect example of someone who has consciously made the decision to accept the risks of putting this information online and we have no problem with that. He is correct that everyone should have the liberty to do such things and that forcing folks to disable geo-tagging is wrong. Where we seem to disagree is we feel that that liberty and privacy walk very much hand in hand. Liberty to do things requires a public that knows the pros and cons of doing it. Our viewpoint of not “the government needs to intervene” on the subject of geo-tagging, we have always been in the “people need to know they’re doing this” and “people need to make an informed decision of whether or not they want to put this information online” camp. We have personally talked to around twenty people and helped them disable this on their phones who did not know they were publishing that information and were happy that we helped them fix it. We, personally, feel that this makes the project a success. We also feel that such features should be “opt-in” rather than “opt-out” and that operating systems should make features off by default.

Next, Nik feels that each photo is a meaningless bit of information. He is both right and wrong on this. Each piece of information is like a small puzzle piece: meaningless in and of itself, but if combined with other photographic "pieces", a larger picture is formed. While one can’t say with certainty with one photo “His house is located here”, if someone takes his entire photo stream and extracts geo-tags from it, patterns begin to emerge: a tight cluster of 10-20 photo locations from a certain house over a long period of time would really increase the probability of that being his house. Smaller clusters over long periods nearby may indicate friends’ houses. Another cluster located in a business park may indicate a workplace or client. A cluster of photos over a week in a remote location may indicate a vacation spot. This is a perfect example of what is referred to in intelligence circles as OSINT. We have enough information in the public, Nik described some in his show (Although, the address in our WHOIS records was from a long time ago), however, we do draw the line at geo-tagging our images.

Speaking of information in public, Nik makes the statement that we can find plenty of information about him from other sources. This is another statement that we won’t disagree with, but we feel that this does not apply to most people. A lot of Twitter accounts are what could be considered “anonymized” by the user: a first name, no website, and hardly any other kind of information. With geo-tagged photos, we can now find a lot more information about then the user probably wanted to “get out.” In one case, we were able to trace an “anonymous” account (no name, no URLs, no tweets detailing any personal information) posting information that, presumably, the user did not want to be linked with back to the him, to the user himself, complete with his home address, work address, public web presence, Facebook, and Twitter. The information posted, let’s call them “revealing” pictures, could be damaging to his reputation and ripe for blackmail. While, like Nik, this is an example that doesn’t apply to most users, it is a good example to what can happen when you geo-tag images.

Finally, on a more personal note, we are offended that Nik jumps to conclusions about our reasoning for this project. Suggesting that we created ICanStalkU and are spamming people to drum up business for some kind of consulting business is flat out 100% wrong. We are all gainfully employed in companies and do this kind of stuff strictly as a hobby. We do not whore ourselves out via this project and Ben has dropped close to $100 on this project out of his own pocket for bandwidth and hosting without asking. If Nik wants to try and find so much as a donation jar on the website, he’s welcome to, but it will be a fool’s errand. We doubt that he looked at the site for very long as he only got one out of two of the creator’s names right, unless this “Jason Bentram” is a double-secret-partner that even we didn’t know about! The stated purpose of ICanStalkU is “Raising awareness about inadvertent information sharing” and that’s all it does.

In closing, we respectfully disagree with some of Nik’s statements, but he is exactly what we want everyone posting geo-tagged photos online to be like: Aware of the risks, evaluated the issues, and made a conscious decision to post them. Kudos, Nik.

We would like to announce that Mayhemic Labs has two upcoming presentations at The Next HOPE July 16-18, 2010 at The Hotel Pennsylvania in New York City:

With the plethora of third party services that allow folks to post photos to their Twitter account, how hard would it be for someone to stalk a person's location via the GPS metadata tagged in their images? Mayhemic Labs did the research and it turns out the answer is "not very." Over the past few months, Mayhemic Labs has amassed a sizable database of people using these services - and what geographic information has been encoded on their publicly available photos. This presentation will cover the basics of how and why this research was done, why sharing such information is bad, why privacy is hard to get right, attempts at public outreach at ICanStalkU.com, how you can replicate such a system, and various instances of privacy fail. Also, tools will be released that will allow you to test your own (or other people's) photo streams.

Why You Should Be an Amateur (Saturday, July 17th, 6:00PM EDT in the Bell Room)

HOPE (Hackers On Planet Earth) is a conference series sponsored by the hacker magazine 2600: The Hacker Quarterly. There have been seven conferences to date: HOPE, Beyond HOPE, H2K, H2K2, The Fifth HOPE, HOPE Number Six, and The Last HOPE. More information is availabe at HOPE.net