I started the first day with 1o57′s talk on the new DefCon badge. This year’s badges were non-electronic (for the first time in several years) — they were antiqued titanium discs with the Eye of Ra and various codes inscribed in them with a water knife. Apparently making the 10,000 DefCon badges actually used the entire supply of sheet titanium in the United States at the time. Bright side of them being non-electronic: they actually had them before the con started! There has been a history of the badges getting hung up in customs on the way from China, but the non-electronic badges were produced in the USA. 1o57 designed an elaborate puzzle contest around the badges, but I can’t say much about it as I didn’t participate this year. There was, however, a very nice-looking code wheel on the floor of the Rio convention center rotunda that was key to the game and gave the room a nice DefCon look, so it was appreciated even by non-participants.

I spent the next couple of hours exploring the non-talk aspects of DefCon (none of the sessions in those slots were particularly interesting to me) and bought up some DefCon shirts and a couple of 2600 Hacker Calendars. I also donated $170 to the Electronic Frontier Foundation in my name and my wife’s, though I didn’t actually end up going to the party to which that entitled me admission (the donation and not the party was the primary purpose anyway.)

I dropped into Mark Weber Tobias’s physical security talk, called Insecurity: An Analysis Of Current Commercial And Government Security Lock Designs, which involved some hilarious attacks on “high-security” physical locks. You know those locks with 5 vertically-arranged pushbuttons you see in every airport or government building? They pop right open if you stick a neodymium-iron-boron magnet on the side. A keycard/keypad electronic lock with a USB port on the bottom for reprogramming is impervious to electronic attacks… but opens if you shove a paperclip to the back of the USB port. This sort of attack was ubiquitous — simple modifications that made sophisticated electronic locks open in purely mechanical ways. The overall point is that to get through a door, you do not have to open the lock — you have to actuate the mechanism that the lock actuates. Sometimes this is really easy.

The next talk was entitled Why Airport Security Can’t Be Done FAST, about the TSA’s Future Attribute Screening Technology. This project intends to detect malicious intent, based on biometrics and facial cues, kind of like an electronic Cal Lightman. The problem, in short, is the standard Bayesian statistical issues that always come up when trying to detect something vanishingly rare like terrorism. The top 10 airlines in the world carry a billion passengers per year — the top 5 US carriers alone carry 500 million per year. How many of these are terrorists who actually intend to blow up a plane that flight? Let’s be very conservative and pretend 100 people try to board an American plane with the intent to blow it up every year (probably an enormous overestimate.) Now let’s imagine my FAST system is 99.9% accurate at detecting terrorists — sounds great, doesn’t it? Let’s get that into our airports immediately! But wait… 99.9% accurate means it will probably catch all 100 terrorists. It’ll also catch 500,000 innocent people — 0.1% of the 500 million passengers. So if FAST points you out as a terrorist, there’s a 0.0002% chance it’s right! Due to the base rate fallacy, a 99.9% accurate terrorist detector’s alarms are false positives 99.9998% of the time. Oops.

What do you bet the real FAST isn’t 99.9% accurate, either?

I next attended the EFF Year in Civil Liberties panel for a summary of legal issues in information security, privacy, and free speech. This was followed by the Hackerspace Panel, about hackerspaces and DefCon groups around the country and what they do to encourage innovation and bring hackers, makers, and other interested people together. Both panels went very well, especially given that the Q&A nature of panels often makes them hit-or-miss.

Friday night at DefCon is surprisingly free of events — about all that’s going on is the Black Ball and the DefCon Pool Party. I met up with the DC206 group again, had some dinner, and mostly hung out at the pool party for the evening and discussed the day’s events and other topics in hackerdom. Frankly, talking about interesting topics (in a hot tub outside with DJs spinning techno in the background, no less) beats most parties anyway.

On the other hand, search is not exactly something people have been clamoring for SSL on.  Implementing SSL for large amounts of web traffic is not cheap (done right it’s not terribly expensive, either, but it’s an engineering effort at least,) so normally it’s only done in response to either regulation or customer demand.

I think Google has an ulterior motive here — possibly two of them.  Current web browsers, as a privacy feature, will not pass extra headers from an SSL site to a non-SSL site or vice-versa.  This means that if I click a link on the SSL Google site, the web site I clicked on will not receive a Referrer: header indicating what I had searched for on Google.

(Incidentally, yes, this does mean that right now every time you click a link or ad on Google, the site you click through to gets to see what you searched for.  It’s always been this way, most people just don’t know it.)

There’s a big business in website analytics.  People run various statistics packages on their website to find out what searches lead to them, what sites link to them, etc.  It’s critical for optimizing marketing or advertising strategies.  There are also several analytics services that will do this for you, including Google’s own product Google Analytics.  If everyone started using SSL for searches, all of these would be broken… well, except Google’s of course, because Google Analytics doesn’t need to rely on the Referrer: header — it has the inside scoop from Google Search itself.

In addition to this, in the pay-per-click advertising world, conversion tracking is very important.  One advertiser may pay for thousands of keywords and run dozens or hundreds of ads.  They track each click all the way through to sales — in other words, they look not just at which ads people click on, but which ads buyers click on, vs. ads that only attract browsers who don’t follow through and purchase.  Once again, these usually work via the Referrer: header, which SSL takes away.  And once again, Google offers its own conversion tracking system, which will no doubt still work when all the others are broken.  This one can be worked around — you can make a third-party PPC conversion-tracking system that doesn’t use Referrer:, it’s just a little more work — but not everyone will work around it.

Both of these results would mean, in a world where many searches were over SSL, rather than just a tiny fraction as it is today, that advertisers & webmasters would have the choice of either operating “blind” or giving all their data over to Google.  And they have a very good reason not to want to do this — if you’re an ad buyer, and Google is the supplier you buy from, do you want Google to know exactly what keywords & placements are most profitable to you?  Clearly Google can use this inside knowledge of their customers’ businesses to maximize prices on the most effective advertising spots.

This is the sort of thing that can lead to an antitrust lawsuit.  So far Google has managed to spin it as a consumer-friendly privacy feature, but we’ll see if that lasts.

Thus, from a security perspective, this just isn’t a good idea.  There’s a large tradeoff in inconvenience, delay, and civil liberties violation for a miniscule increase in security.  However, it does get me thinking about an interesting problem — how does one hide data from people inclined to search your electronic devices for it?

A legal search is a totally different kind of threat from a hacker attack.  With a hacker attack, you simply have to keep them out of the data — with a legal attack, you have to hide the existence of the data, as the legal system has at their disposal an additional channel for getting the data — they can subpoena it and demand you disable any protective measures and hand over the data.  Thus, encryption — the primary defense against data disclosure to hackers — is of limited use against a legal attack.  (And note that a “legal attack” doesn’t just mean law enforcement or other rightful authorities — it also means attack via lawsuit.  Abuse of the legal system is not limited to the political administration — competitors and other adversaries can and do use the legal system to get at things they shouldn’t have.  In other words, this information isn’t of value only to criminals — there are a lot of perfectly legitimate reasons to hide data.)

The EFF points out a few possible ways of avoiding scrutiny from customs:

• Create multiple accounts on the machine, and just log in with an account with nothing sensitive in it when asked to log in.  This is basically taking advantage of the lack of technical expertise on the part of the searcher.
• Take only the data you need on the trip — just minimize what there is to find.  This is a good idea anyway, but probably unsatisfactory if you are carrying, say, diplomatic communications.
• Bring no data at all, and when you arrive at your destination, retrieve the information via VPN.  Before flying back, VPN the data back and delete it.
• For sensitive business communications, have the data encrypted by someone else who provides the key only when you arrive at your destination.  This would work to protect the data, but it also means that, being unable to comply with an order to reveal the data, you may just have to miss your flight.

I have two more that they didn’t mention:

• Encrypt the data onto something that is not an “electronic device” subject to search, like a CD-ROM, USB key, or whatever.  It no longer falls under the search provision.  Obviously it could be searched if you were actually arrested or sued, but it gets around this particular issue.
• Use TrueCrypt Hidden Volumes.  Merely hiding an encrypted file on a disk will not hide it from a skilled attacker, because cryptographic data is distinctive.  Statistically, it has a uniform distribution, which makes it look unlike any other kind of data except white noise (random numbers.)  Essentially, it looks so bland and generic that it stands out — because no real data is that essentially devoid of information.  Since nobody keeps a hard disk full of random noise files, if one exists, it must be encrypted data — which means you can be subpoenaed for the key.  TrueCrypt’s hidden volume feature gets around this in a novel way, which I’ll discuss below.

Hidden volumes take advantage of the similarity between random noise & encrypted files.  A section of disk is reserved for an encrypted virtual disk.  When this is created, it is filled with random noise, which is replaced by encrypted data as needed.  The trick is that you can create another encrypted virtual disk inside the first one.  So long as some data is in the “outer” volume (as no one would have a huge encrypted file on their hard drive with nothing in it — it’s not plausible), there is no evidence that the “inner” volume even exists unless you have the key.  The inner volume’s encrypted data blends into the outer volume’s white noise.  Thus, you put slightly-secret data in the outer volume, and really-secret data in the inner volume.  When asked to reveal the key, you reveal the key to the outer volume only, and have plausible deniability of the inner volume’s existence.

As with any countermeasure, though, there are limits.  If you’re hiding from the NSA or some foreign government’s equivalent, just putting a couple TrueCrypt volumes on your laptop’s hard disk will not do the job.  The problem is that the operating system and the applications you use may leave traces that reveal the existence of the inner volume (e.g. Word’s file history notes that you opened a file on Drive F:, when your laptop doesn’t have an F:…)  For extremely sensitive data, it would be necessary to not only put it in a hidden inner volume, but also to only ever access that inner volume from an ephemeral operating system (e.g. a LiveCD, or an OS you boot off a USB key and load into a RAMdisk.)  If the OS you use never makes any changes to the disk outside the encrypted volume, evidence of the volume remains hidden.  You would of course want a normal OS and outer volume to be present and used, for plausible deniability to be present (as, once again, it’s not reasonable to have a laptop with only random noise on the hard drive.)  You would also want to access the outer volume with the laptop’s native OS after any session in which you accessed the inner volume (as otherwise the access date on the encrypted file could be newer than the last boot date on the OS, once again leaving a breadcrumb trail.)

And all this makes me wonder once again what the government plans to get out of casually searching the data on laptop hard disks.  The only people whose data will be discovered are those with nothing to hide.

RFID devices are becoming more and more common, and this trend will continue — they’re too convenient for many purposes for the security risks around them to stop them. You may not want every consumer good you buy to be tagged with an ID that lets people watch your shopping from 100 yards away, but the scenario of being able to check out at the grocery store by instantaneously scanning every item in your cart simultaneously is too compelling for people to resist.

Bruce Schneier has a post on the ineffectiveness of security cameras, but while calling them ineffective it does note that criminals moved their crimes to somewhere the cameras couldn’t see. This may be “ineffective” for a government camera system designed to deter crime, but it’s precisely what privately-owned security cameras are meant to do — make a target unappealing so criminals go elsewhere. This actually shows that cameras do deter crime… but only where they can see it.

However, both of these technologies can have pernicious effects, too. The HexView article points out that you could use the RFID tire monitors to commit murder — set a bomb with a radio trigger that goes off when the “right” car drives over it. It would also be just as useful to private investigators spying on citizens as it is to law enforcement chasing down criminals. And speaking of law enforcement, these cameras create a dangerous imbalance in their favor — the camera evidence is all under their control, and thus can come up when needed to prove a perpetrator’s guilt yet be conveniently lost in cases of police brutality, abuse of power, corruption etc.

This is an interesting time for surveillance — police and government surveillance ability is skyrocketing (London is practically blanketed in cameras at this point, as the British seem much less uncomfortable with them than Americans are) but it is still largely in the hands of authority figures. This is dangerous because of how fast the change is coming — our criminal laws and sentencing structures are based on the principle that most criminals get away with it. A $75 fine for speeding seems pretty reasonable, but what if that fine were levied every time a car hit 1 mph over the speed limit? Most of us would get fined a dozen times a day, every day, despite not even meaning to speed, because our behaviors are based on the idea that we probably won’t get caught and that even if we are police are unlikely to punish us for very minor transgressions. If people were caught for speeding every time, and fined every time, a $75 fine would be absurd — the fine could probably be under $1 and still bring in a few hundred dollars a month from every citizen. What is the right legal structure here? I can see two possibilities:

• Raise the speed limits to the speeds we really think no one should exceed, and continue to fine every time.  Maybe you should get charged every time you exceed, say, 85 on a highway or 55 on a city street.  Set them high enough that there’s no leeway required.
• Leave the speed limits where they are but set the fine really low, say a $0.25 per minute of speeding.  This makes speeding discretionary — you can obey the law, or not, but if you choose not to you pay a penalty.  This is a fundamental change in the whole idea of crime and punishment, and itself has some pernicious consequences — it means that a certain income level can render you “above the law,” which is not a good thing.  Obviously some crimes (such as murder) should not be treated as discretionary, but for traffic violations it could make sense.

It’s not just traffic laws that are like this; consider the War on Drugs.  If every person who ever smoked marijuana went to prison, we would have a nation of felons — there’d be few people left who could vote, get security clearances, hold most jobs, etc.  The RIAA lawsuits against file-sharers are a good example of what happens when technology that catches everyone gets used to enforce laws designed under the assumption that only the worst and most flagrant criminals will be caught — people being hit by millions of dollars in fines for using technology to do something that wouldn’t even raise an eyelash if done by old, physical means (e.g. posting a song on BitTorrent vs. handing it to a friend on a cassette tape.)

A surveillance society needs a different kind of jurisprudence — one that sets punishments that fit the crime even if applied every time.  On the bright side, actually doing this would lower crime rates tremendously due to the psychology of criminals.  Escalating punishments does little to deter crime because criminals are risk-seekers — they do not expect to get caught.   Even a small punishment can be a strong deterrent if applied every time — if criminals are usually caught, such that all criminals have some first-hand experience with being caught and punished, it would break this idea.  On the not so bright side, a surveillance society must have very liberal laws to avoid being a police state — our current legal system, applied to everyone every time, would result in tyranny.  We all break 10 laws a day, it’s only sloppy enforcement that allows us to live our lives.  Unfortunately, the technology for ubiquitous enforcement will come well before the legal system changes to make it livable do.

What’s interesting to me is what will happen when surveillance becomes even more common: that is, when it is no longer monopolized by authority.  This has already started with cellular phones.   Almost everyone carries around a device which, while primarily for communication, contains a camera and often a voice recorder and videocamera as well.  Everyone is equipped to carry out impromptu surveillance at any time.  Devices like these glasses from ThinkGeek (found via BoingBoing) coupled with the rapidly falling cost of storage capacity will change this to everyone actually carrying out impromptu surveillance all the time.  This will have a chilling effect on human behavior at first — would you act differently if you knew everyone around you was videotaping everything you did?  Everything you say will, indeed, be able to be used against you, and not just in a court of law.  However, look at what young people put on MySpace and Facebook these days — the next generation does not have the assumption of privacy.  They’ve grown up in a world where they know everything goes on a permanent record, and have simply accepted it.  Sure, they’ll be occasionally shocked by it (e.g. the first time their party photos on MySpace disqualify them from a job), but the knowledge of permanence has not stopped them from sharing themselves, and eventually the rest of us will adjust, too.

Consider what the democratization of surveillance does to government power.  When we’re all recording, someone is watching the watchers.  Corruption, abuse of power, etc. all rely on the fact that authority figures can get away with crimes because they are more reliable witnesses in court than their victims are.  When everything is on the record — and not just the official record, but everyone’s record — police and government officials become compelled to act within the law.  While this may not be much of an impediment in truly totalitarian societies like China where the courts are as corrupt as everyone else, it’s a very strong bulwark of freedom in any society with an independent judiciary and a liberal tradition like the Untied States and Europe.  This is the next generation of surveillance — everyone sucking in light and sound from their glasses, or lapel pens, or even contact lenses, recording every moment of their lives on multi-terabyte devices that fit in their pockets.  It’s probably only 5-7 years away, and it washes away the current problems of a surveillance society and replaces them with new ones.

I think this cycle will continue for some time.  After all, once we’re past the era of democratized surveillance, computer graphics and artificial intelligence technology will improve to the point that ordinary people can modify their recordings to create perfect video of events that never happened, indistinguishable from the real thing.  What happens to recordings in law courts then, when they cease to be reliable evidence and become hearsay?  Tapes will become the new eyewitnesses, known to be unreliable and requiring corroboration from others.  When it becomes truly easy to make forged video, perhaps we will have emerged from the surveillance society from the other side — why bother to record anything when there’s no way to tell if it’s real?  Sometimes the only way out is through.

They make a big deal out of their privacy practices.  They do not maintain histories on browsers the way Google does — they just replace ads on pages based on the page’s content, kind of like Google AdSense but for image and rich-content ads as well.   Customers, unsurprisingly, don’t really care either way about this service — what’s it matter if I get CNN’s own banner ads on their pages or my ISP’s banner ads?  They’re still ads, and nobody likes them, but whose ads they are isn’t high on a consumer’s priority list.

However, products like this (generically called “ad replacers”) are going to be extremely important to the future of the Internet.  The linked article talks about how ISPs’ profit margins are narrow given their customers’ increasing appetites for bandwidth, and how this advertising revenue will help them recover.  What it doesn’t mention, though, is where this revenue comes from – it’s the ad revenue that would otherwise be given to the sites you browse.

In other words, ubiquitous use of ad replacers would boost ISP revenue while destroying ad revenue paid to web sites.  This is a tremendous threat to Google as it eliminates their sole revenue stream!  For that matter, if an ad replacer can substitute ads, why not substitute the first page of Google search results?  Google won’t sell you #1 placement in organic search… but with an ad replacer, Comcast (for example) could sell you #1 placement on Google for Comcast users.  In addition, all the small niche websites that currently pay their hosting bill (and their owners’ salaries) off of advertising revenue may find themselves unable to do so.  People hate advertising, but what happens to the Internet without it?  The free, ad-supported Internet goes away, replaced with paid, subscription-based walled gardens.  Nobody wants that, but that’s the world ad replacers lead to — and ironically, it’s a world that has no room for them, as they would then have no ads to replace.  This is difficult to fight economically, though — an ad replacer can be a tremendous source of revenue so long as there aren’t many of them.  There’s lots of incentive to make them, even though in the long run they kill the ecosystem.

What this will lead to is a new security arms race.  Publishers will have to start finding ways to “hide” ads in their pages, so that ad replacers do not recognize that they’re ads and replace them.  This will be particularly hard for the large ad networks like Google’s where the ads must be embedded in thousands of dissimilar web pages.  As the publishers come up with better ways to hide ads, the ad replacers will be updated to find them.  The result is likely to be quite a mess, and result in neither the ISPs nor the publishers getting as much revenue as they’d like.  In addition, while Phorm may promise not to build up profiles of private information on you, an ISP who did engage in Google-like privacy invasion would be able to do it far better than Google can — after all, they have all your billing info since you’re a paying customer.  Unlike Google, they really do know who you are, personally, and not just by your browsing habits.

In the long run, international backbone providers could even start replacing ads in order to avoid local legislation, though this would lead to the ridiculous situation of the same ad on a page possibly being replaced several times on its way to the user.  I don’t see any solution to this other than legislation — the same sort of “net neutrality” laws  that forbid content-based traffic shaping or Comcast-like protocol tampering could also forbid ad replacers.  Unfortunately, economic incentives aren’t likely to have much effect, since the actual end users won’t change ISPs to go to one that promises not to run ad replacers — as only the publishers, not the end users, care whose ads are seen.

First, for those not working in the information security industry: something being classified as personally identifiable information (PII) is a big deal. If data is PII, you are liable for damages if the data is ever released, and you are required by statute to take significant and often expensive measures to protect it. If you’re a public corporation, Sarbanes-Oxley requires you to do all sorts of things to protect the data (e.g. encryption.) If your company takes credit card payments, the Payment Card Industry Data Security Standard requires you to do even more (e.g. physical protection of the hardware the data sits on, specific firewall/router configurations, etc.) Most large companies have their own standards for how PII must be protected that combine or even go beyond the regulatory and industry requirements. Overall, the required protections around PII are onerous enough that companies strive to minimize how much PII they have at all — it’s often cheaper and easier to just delete the data than to protect it the way you need to protect it. Companies must make the decision of “How much business value do we get out of storing, say, our customers’ addresses, and does it exceed the cost of protecting that data?” Often the answer is no.

On the surface, calling IP addresses PII is ridiculous. IP addresses are found on every packet anyone sends on the Internet; if IP addresses count as a personal identification, then logging basically anything about Internet traffic makes the logs PII. It takes a label currently applied only to a small amount of high-value data and applies it to something that everyone everywhere logs; it seems absurd. But as I think about it more, I’ve come to realize that Scharr has a point.

The EU is much more aggressive about privacy law than the United States. The United States Constitution guarantees privacy from the government through the Fourth and Fifth amendments; this sharply limits what the government can collect on you and what it can do with the data it does collect. However, there is no Constitutionally or legislatively defined general right to privacy — anyone can collect whatever data they want, so long as they’re not a branch of government. This is usually an adequate protection against government abuse, but it does mean the private sector can accumulate a frightening amount of data about you, and that could be prone to abuse as well. EU nations, on the other hand, often have a general right to privacy and various data collection expected in the United States is often illegal; in addition, where the data can be stored, sharing it with any third party without express user consent is almost always illegal.

If IP addresses are PII, what really happens? It requires changing a lot of current practices, but this is not the same as breaking scenarios. Remember, the privacy issue isn’t with transmitting or using IPs — it’s with storing them or sharing them with a third party.

• Currently search engines like Google use your IP to identify where you are geographically, so as to establish search profiles for regions and target ads. They store the first 24 bits of your IP (dropping the last octet) as a proxy for location. They would need to switch to storing a different proxy for location (e.g. latitude and longitude), though they could still base this proxy on your IP.
• Pay-per-click ad networks would still function. When they’re clicked, the ad network records the click (so as to be able to bill the advertiser), then issues a 301 redirect to the advertiser, who also records the click (to know it happened and the ad was effective.) These records would need to leave out IP, or be protected as PII. Lacking IP, however, would make detecting and preventing click fraud (spoofed clicks, or many clicks from the same person) much more difficult. Currently a skilled fraudster can evade IP-based click-fraud prevention, but losing even that would make click fraud easy. Also, without IP addresses, the ad networks would have a hard time proving to advertisers that clicks were real if an advertiser chose to sue them. Large ad networks would probably have to just eat the cost of protecting their logs as PII.
• Contrary to RSnake’s comment, I do not think this would affect embedded content. Embedded content comes in two forms — content linked to on a page, which your browser loads (objects), and content retrieved by the server and displayed on the page (mashups.)
  • In the object case (e.g. viewing a YouTube video on someone’s web page), the web site owner is not leaking your IP to the third party — you are. The web site is not sending your IP to YouTube at all; your web browser is sending it in response to a link tag in the page.
  • In the mashup case (e.g. web pages that get data from an API, like Facebook pages, pages embedding Google Maps, etc.), the web site owner is also not leaking your IP to the third party. You access the site, and then the site accesses the third party not as you, but as itself. The site leaks its own IP, not the customer’s. No PII is released.
• Sites that do user tracking (via logins simply recognizing users between sessions) would be unaffected; they use cookies, not IP, to track users. Most ad networks work this way, too.
• The biggest change, though, is to simple website logs. Currently every time you access any web page, it makes a note in a log of your IP and which site you accessed, which is used for statistical analysis, forensics, etc. Even this blog is doing it; with most web providers you can’t even turn this logging off if you want to. Sites will either have to stop doing this or take substantial steps to protect the logs (or else be subject to significant statutory liability if they don’t.) Not keeping logs is, from a security perspective, very dangerous — if something happens, you have no idea what happened and thus may not be able to fix it.

However, despite all that cost and difficulty, when you think about it… IP addresses really are personally identifying. If you have an always-on broadband ‘net connection, your IP address changes very rarely (maybe only once in several months), so all your web traffic everywhere, complete with your search queries, emails, etc., can be tied together with that number. Your ISP can connect that number to your name, address, etc. If you’re at a corporation, the IP is tied to a corporate gateway or proxy… which has logs tying each communication (based on date and time) to your desktop’s IP, which once again likely uniquely identifies you (unless you always compute from a shared machine.)

IP is a unique identifier for confirming identity, but not so much for initially finding it. In other words, if someone attacks my website, and I have only their IP address, it may not do me much good in finding out who they are unless I can get someone with subpoena powers to get it from the ISP. However, if I suspect a specific person of something, I can probably find out their IP and check it against my attacker’s IP, thus confirming their identity. Likewise, if I am an ad network or search engine with a lot of IP data, I don’t know who you are based on your IP, but the commonality in IPs between all the data I have may enable me to figure it out based on data aggregation.

I think this is a case where something is considered ridiculous merely because it changes things. Yes, a lot of business models and current practices would have to change if IP-as-PII became the default assumption. Yes, it would make some security people’s jobs harder, and cause web providers to incur a lot of costs. But does that mean it’s wrong? Perhaps what it means is that current businesses & web sites under-value their users’ privacy, and are freeloading while providing inadequate protections. It’s a different world if we have to discard IPs or protect them as PII, but I’m not convinced it’s a worse one.

However, this system does rather more tracking than your average grocery store “membership card.” When you join, it installs a local proxy on your system and reroutes all your web traffic through it, including SSL sessions on port 443 (yes, it actually mounts local man-in-the-middle attacks on your online banking.) It then monitors this traffic, and based on some algorithm that has not been disclosed, sends some of it to comScore. Sears’s privacy policy promises not to share your data with anyone, and so does comScore’s, but it’s pretty hard to figure out what that means in this case. After all, comScore’s policy also promises not to collect any information that’s personally identifiable, but your My SHC Community data is tied to a personal ID at Sears, so in this case they’re clearly collecting personally identifiable information. Also, I think most people would consider copies of my online transactions in SSL sessions to be “personally identifiable;” while we can’t be sure comScore gets all of these (since the algorithm by which some traffic is rerouted is unknown), we do know the software is capable of sending them to comScore so we just have to take their word for it. Also, CA’s research did show an SSL transaction being rerouted, credit card numbers and all.

Bruce Schneier points out that if an average piece of spyware did this, it would be considered criminal. However, not only is Sears a large corporation and thus able to get away with this sort of thing (remember the Sony Rootkit debacle?), it also did have a pretty clear privacy statement that the user agrees to before installing it, so it may be on good legal ground. However, even if it’s legal, it’s a terrible idea for all involved.

First of all, the app is silent — once it’s been installed, it gives no indication it is monitoring your traffic, and no clear way to remove it. Second, the fact that the app comes from Sears, providing their privacy policy, but the data goes to comScore, while both parties claim the data is not shared with “any other party,” makes the privacy policies border on nonsensical. If it takes a lawyer to figure out what exactly your click-through license agreement means, it’s pretty disingenuous to claim that end users have been properly informed and have voluntarily waived their privacy rights. And third, comScore & Sears are collecting data (such as your credit card numbers and favorite non-commercial websites) that they don’t even want along with the information that they’re trying to collect. This puts on them a legal burden to protect and secure huge volumes of information that provides them no benefit.

When you have private data that you have a moral, legal, or regulatory responsibility to protect, the first thing to consider, before looking at security measures, is whether you need the data at all. It’s a lot easier to delete it and stop collecting it than it is to put in encryption systems, network access controls, auditing and logging systems, etc. A lot of companies collect reams of useless private data simply because “they’ve always done it that way,” and thus have to spend money protecting things of no value to them. This is probably the logic behind Sears’s data collection here — “we might as well have everything, it could be useful someday” without thinking about the cost that having that data imposes on the enterprise. You can’t have a catastrophic data breach if you don’t have the data.

This is also another symptom of a larger problem — people are increasingly unable to control the code running on their own computers. The separation of code and data is becoming increasingly porous with the web’s “active content,” and DRM software exists to keep the user from controlling their own system’s activity. Microsoft’s Vista User Account Control and Integrity Levels systems try to mitigate this, but it’s really not enough.

The problem is that they rely on the user to determine what code is allowed to run, but the user is unable to verify what that code will do until he runs it. It’s impossible for the computer to tell the user what it will do, as native code is unverifiable. With some technologies, such as Microsoft .NET code, it is possible for the system to tell the user what the code will do, but people writing malicious or underhanded apps like this Sears spyware and the Sony rootkit will not use these technologies, sticking to the unverifiable native code. It is my hope that virtualization will offer a way out of this in the long term — a way for each application to have its own enforceable security boundary. However, to avoid these same problems from occurring, application developers will have to give up functionality — that is, certain types of inter-application interaction will have to be categorically prohibited, which will sometimes inconvenience the user.

I think we’re more likely to see these solutions come from the open-source world than the commercial operating system world (i.e. Microsoft and Apple.) The commercial OS world is very concerned about a.) ease of use for the user, and b.) backwards compatibility for applications, as these things sell software. The open-source world is less concerned with these things, which inhibits their adoption in the marketplace but also results in software that is often much more under the user’s control than commercial software is. The real trick will not be developing these security technologies (not that that will be easy); it will be adapting them so that they can be used every day by non-technical users.

This is not an uncommon perception.  Many people who think nothing of handing over their credit card or writing a check when at a store or restaurant hesitate to use the same card online, regardless of communication protections (e.g. SSL/HTTPS), third-party assurances like the preposterously-named HackerSafe, or the size and stability of the vendor.  After all, it’s the Internet, there are bad people out there.

However, the perception just isn’t true.  There are two ways in which the Internet particularly helps thieves, though:

1. Once they’ve stolen an identity or credit card number, thieves often use the card online, as they don’t have to present themselves (and thus show up to witnesses and potentially security cameras) to use the card.  This is actually probably what the credit card company in my experience meant — not that the transactions are more dangerous, but that fraudsters often use stolen cards online.
2. Hackers stealing credit card information online often steal entire databases.  They don’t steal your credit card while you’re buying something online — they break into the online store and steal everybody’s card.

However, they could just as easily have broken into the servers of a brick-and-mortar store — it’s not the fact that you used the card online that makes it possible for them to steal it, it would have been just as at risk handing it to a cashier.

In many ways, it’s a lot more risky to make non-cash payments in person!  When you hand your credit card to a waiter or clerk or cashier, they could easily copy the number, expiration date, and CCv2 code (the three-digit code on the back than an online site often won’t even get.)  With a debit card, they have the opportunity to watch PINs being typed.  Whereas in an online store, only relatively few, well-paid professionals will have access to your data (system administrators, etc.), every $7 per hour sales clerk can see a hundred card numbers per day, and probably has significantly more financial motivation to steal them (although in my experience, the fact that someone doesn’t need money won’t stop them from stealing it if they’re the type to steal — just look at Michael Milken, who defrauded people out of hundreds of millions of dollars at the same time he was making hundreds of millions legitimately.)

Some people — usually those of us who remember the days before debit cards — eschew all these fancy online and electronic forms of payment and instead stick to good old fashioned checks.  After all, no one can possibly steal those!  They’re paper, and have your signature on them.  This is the ultimate in perception differing from reality — it’s hard to imagine a less secure way to make a payment than a paper check.

First of all, there’s the ease of committing fraud with checks.  A thief with a stolen check (or deposit slip) has all they need to take money from your account — the routing number and account number (found at the bottom of the check in MICR letters.)  Note that the thief doesn’t need any kind of ID… or a PIN… or a physical card… or a CCv2 code… or even to know your name.  No, the numbers will do.  What can they do with a stolen check?  There are three basic things:

• Order up a whole book of checks with your information and account numbers on them.  No ID is required to order checkbooks online.  They can then spend these checks anywhere, and the bank will process them — you probably won’t find out until your account is empty and you start getting NSF notices.
• Remove the amount and recipient from the check and write it out to themselves instead.  This is a bigger problem for institutional checks, which are often printed on a laser printer.  It’s really easy to remove laser-printed text from an offset-printed check — just lay some Scotch tape over the laser text, rub it hard with your fingernail, and peel the text off.  Then you can print out a new amount and recipient with your own laser printer, and it looks just like the real thing.  Chemical agents (“check washing”) can do this with ball-point pen ink, too, though it’s not so easy.
• Issue a demand draft (“paperless check.”)  This is what happens when you pay by phone with your checking account number, or use an automated bill pay service, or send money via PayPal.  Using your routing number and account number, money is simply removed from your account and put into someone else’s.  No authorization or authentication is used, your name is not even required.  Yes, really.  Anyone can do this from any account to any other account.  For a while, you used to be able to do this from a web site.

Second, there’s the difficulty in getting your money back or even stopping the fraud!  With a credit card (and to a lesser extent, a debit card), it’s pretty simple — you call the bank, say you did not authorize a charge, and the credit card company removes the charge.  It is then up to them to prove you did make the charge, such as by getting a signed receipt from the merchant and matching your signature.  So long as you report the fraud within 30 days, you are not liable — the worst the card company can do to you is to cancel your card (but you still don’t have to pay for the charge you didn’t make.)  In theory, you’re liable for up to $50, but almost no card issuers really charge this since it’s terrible customer service (“Sorry you were stolen from!  Give us $50!”)

With checks, the money is already gone.  If you report a check as fraudulent, there is no federal law saying the bank is liable — it’s up to the bank’s own policies and in some cases a hodgepodge of state laws whether they have to help you at all.  The bank may get back to you in 60 to 90 days (during which you don’t have the money, even if it was the entire contents of your checking account.)  You have to report the fraud on a paper letter, with a notarized signature, usually by certified mail.  What’s more, you have to prove that the checks were not authorized — the burden of proof is on you, not the bank or merchant — and you have to do it to each party from which you’re trying to reclaim money.  If a thief wrote bad checks in 20 different jurisdictions, you may be dealing with this for years.

Worse yet, you can’t stop the fraud from taking place.  The thief can keep writing checks on your account even after you’ve started reporting them as fraud, and even after you’ve closed the account.  Every time the thief writes a bad check on a closed account (the classic practice known as “paperhanging”, a favorite of Frank Abagnale during his criminal youth), your bank will reopen the account and send you an NSF notice.  You have to dispute all of these, too.  And finally, your account (and possibly your name) will go into ChexSystems (the equivalent of the credit bureaus used to check people’s checking account history) as fraudulent, which will make it difficult or impossible to get new checking accounts for many years.  On the bright side, it will make it harder for the thief to open accounts in your name, but that’s little consolation since he can keep using the closed one he already has.

From a security perspective, checking accounts are horrid.  They come from a day when authentication and authorization were unheard-of, and security came mainly from the idea that no one would figure out how to subvert the system.

What can you do to protect yourself?

• Don’t use checks.  If any method of payment is offered aside from checks, use that.
• Don’t use demand drafts, either — they’re checks.  Don’t pay by phone using a checking account number — use a credit/debit card.
• If you must write paper checks, use them only to pay bills, dealing with relatively trusted merchants.  It doesn’t make you totally safe, of course, but it helps some.  Use gel ink to write checks (it’s harder to wash), or a dot-matrix printer to print them (the impact-printed ink is nigh-impossible to remove.)  According to Abagnale’s The Art of the Steal, this makes check-washing nearly impossible (though ordering up new checks in your name still works.)  Incidentally, The Art of the Steal is a fantastic (and very short) book, and I highly recommend it to anyone interested in security — it gives a great view into the security mindset, looking at all parts of a system and seeing how it can be subverted.
• Don’t store any more money in your checking account than you have to.  You’ll still have to fight every fraudulent transaction to stop the bank trying to collect it from you, but at least you’ll still have your money while you’re doing it.

The sooner we move on from this antiquated and unsafe payment system, the better.

To people not very familiar with cryptography, it may seem odd that random number generators are very significant.  However, all modern key-based cryptography is based on having a source of entropy (true randomness) — somewhere it can get a key that is unlikely to be guessed or otherwise determined.  When we talk about “40-bit” or “128-bit” encryption, we’re really talking about the key length, which provides an upper bound on available entropy.  Ideally, cryptography would be based on true random numbers, for which every bit of number is a bit of entropy.  However, true random numbers have to be generated physically — we have devices that do it based on radioactive decay, but you can also get it by asking a human to move a mouse around or bang on a keyboard, as PGP does when generating keys.  Thus, for most applications, we settle for pseudo-random number generators — programs which generate a stream of numbers that are unrelated to each other, have a uniform distribution, and are for most purposes entirely random.

However, a psuedo-random number generator usually needs a seed — a starting point for the generator.  If you use the same seed, you’ll get the same stream of “random” numbers.  Thus, the seeds chosen are usually very large numbers.  Cryptographic pseudo-random number generators are considerably more processor-intensive than the regular “random” number generators used in non-security applications, as they’re usually based on multiple iterations of a hashing algorithm.

What happens if your pseudo-random number generator isn’t very good?  Well, in the early 2000s, an online casino in the Caribbean (I wish I could remember the name of it to provide a link to the news coverage) lost several million dollars.  Apparently, a player realized that to shuffle the decks of cards, they used a standard, non-cryptographic random number generator — the sort of thing that’s built into Windows and Linux and such.  A shuffled deck of cards is very random — there are 8×10⁶⁷ ways to shuffle a deck, which is about 225 bits of entropy. However, the random number generator used only a 32-bit seed!  There are only 4×10⁹ 32-bit numbers.  This is still a lot, but with modern computer aids, it’s a manageable number.  So what did this player do?  He had his computer generate shuffled decks for each of the four billion 32-bit seeds.  He then wrote a program that let him enter specific cards that were drawn (e.g. “fourth card was a queen of spades, fifth card was a 9 of diamonds…”) based on the draws he could see (such as his own cards in poker, or the up cards in blackjack) and it would pare down the four billion decks to the ones that could have potentially produced those draws.

It turns out that when you know that almost all decks are invalid (not able to be generated by the random number generator in use), there aren’t many decks that can produce a given set of cards.  Thus, within 3-5 known cards, his program would spit out the entire deck, and that player could now predict the future.  He would know exactly what cards would be coming out, and what ones already had.  Thus, poker and blackjack were trivial, and he won a ton of money.

Many things in cryptography operate similarly.  If you can predict the random numbers being used, you drastically simplify cracking the code.  It is generally still not what a layman would call simple — but it brings a message from “even the National Security Agency with its thousand acres of supercomputers couldn’t crack it in our lifetime” to “it’s still out of reach for you and I, but, well, the NSA could probably crack it in a day or two.”  Well-funded, skilled adversaries can use any small defect in a cryptosystem that lowers entropy to shorten the time to break codes.

And that’s why the NSA would be interested in putting a back-door in a pseudo-random number generator.  Did they actually do this?  In my opinion, the evidence Schneier presents is pretty convincing, and while Schneier is today best known as a popularizer of security rather than a technical expert, one would do well to remember that he also wrote Applied Cryptography, a very technical book that sits on the bookshelf of basically every security developer, including mine.  The NIST publication presents four random number generators, based on different algorithms, and then recommends the use of one, Dual_EC_DRBG, that is about 1,000 times slower than the other three.  Unlike the others (Hash_DRBG, HMAC_DRBG, and CTR_DRBG), however, with this particular algorithm it would be possible to craft a set of input constants that are defective in a specific way — such that someone armed with a corresponding set of constants could predict the output of the generator.

Now, we don’t have proof that the NSA actually did this.  It’s possible that the input constants in the NIST publication are truly random, chosen arbitrarily, and the NSA does not have a matching key that will break the generator.  But the NSA is pretty smart, and almost certainly knew about the flaw in the algorithm — in general, people in the cryptographic industry assume that the NSA is a few years ahead of them and just hasn’t said so.  The old adage about not attributing to malice what simple incompetence will explain usually applies to government pretty well, but not to the NSA.

Really, this is a rather ingenious way to backdoor a crypto algorithm.  The normal method — just make a cryptosystem with a mathematical flaw or known backdoor key — has a serious issue: if you can figure out the mathematical flaw, so can someone else.  The NSA wants to be able to listen to our phone calls — it doesn’t also want every other country to be able to do so.  To backdoor a cryptosystem requires making it so you can read messages without also weakening it for everyone else.  This method does exactly that — without the specific numbers that match the provided input constants, the system isn’t flawed at all.  The NSA has the key (if, indeed, they do), and no one else does.  Putting it in the random number generator rather than the cryptosystem itself is a good way to draw attention away from it, too.

And if the NSA didn’t choose the constants to have a backdoor, why recommend an elliptic-curve based generator that’s three orders of magnitude slower than several other generators, all believed to be just as secure, that are based on much more easily understood mathematics like hashing?  It just doesn’t seem to make much sense.

Currently, the Pirate Bay is probably the world’s most popular BitTorrent tracker for downloading pirated media, receiving 1.5 million unique visitors a day.  With a quick trip to the Pirate Bay, you can quickly acquire any piece of music, any episode of any recent television show (usually within a couple hours of its first airing), any movie (generally while it’s still in theaters), etc.  Membership is required to enforce ratios (i.e. ensure you upload as well as download), but is free and open to all.  However, they’re unsatisfied with the BitTorrent protocol for a variety of reasons — chiefly the legal risk that their “customers” take.  Downloading from the Pirate Bay via BitTorrent runs two risks — first, that a copyright holder will grab your IP address and send a cease-and-desist order to your ISP, or worse, a subpoena which under the DMCA in the United States could carry a fine of tens of thousands of dollars, and second, that your ISP itself will cancel your subscription for using too much upstream bandwidth.  Comcast, in particular, is notorious for doing this without being willing to admit how much “too much” is, even as they cut you off for using it.

BitTorrent is an ingenious protocol.  The idea is to prevent massive load on single servers for downloading popular files by ensuring that everyone who downloads the file also shares it with others, even as the download occurs.  You don’t need the entire file to start sharing it — you register with a BitTorrent “tracker” like (The Pirate Bay) as working on a file, and all the other peers who either have or want that file are notified of your existence.  Peers then communicate with each other, swapping whatever parts of the file they have for the parts they don’t.  Thus, everyone’s upload bandwidth is being used at the same time as the download, unlike some previous P2P protocols.  This is used for many legal purposes — for one, Blizzard’s World of Warcraft uses it to update the game, to get around the obvious difficulty of having about 4 million of its 6 million subscribers all trying to download a 450-meg content update on the same day.  Thanks to BitTorrent, these updates go smoothly every time.

The problem, however, comes when the files being shared are illegal.  In the United States, uploading copyrighted media can result in rather substantial fines and statutory damages, and the RIAA and MPAA are actively suing people by the thousand to get them charged.  People want to download copyrighted media, so sites like the Pirate Bay exist.  But RIAA and MPAA agents can connect to these trackers, too — they’re open to all — and the tracker shares everyone’s IP address with them.  Since with BitTorrent, downloading and uploading go hand in hand, there’s no way to download copyrighted material without not only breaking the law but also advertising your IP to anyone who wants it.  There are blacklists of known RIAA/MPAA peers that will protect a pirate from the most ham-fisted detection, but it would be trivial for the copyright holders to evade this sort of blocking.  The Pirate Bay itself is largely immune to prosecution — they are located in Sweden, where copyright law subjects them to at worst a $300 fine every time they’re arrested (which has happened more than once.)  For the most part, legal threats just amuse them.  However, they’re concerned about their downloaders — as without people sharing files, they cannot exist.

In addition to the legal issues, there is the issue with ISPs.  “Unlimited” low-cost home broadband survives because people generally use only the tiniest fraction of their upstream bandwidth.  Comcast allocates me, and everyone else in my area, 384 kbit/sec.  If I used this bandwith to full utilization for an entire month, I’d have uploaded 118 gigabytes.  This is actually quite a lot — by way of comparison, playing World of Warcraft 24/7 for an entire month would use only 1.2 megabytes, or 1% as much.  This is fine by Comcast, because most of their users are only surfing the web, using only a few hundred kilobytes per month.  If everyone used their entire allotment of 118 gigabytes, Comcast would have to raise rates tremendously — from the current $50 or so per month to probably 5 times as much (or more.)  Compare business Internet rates (which assume you are hosting servers, and thus upload a lot) with residential ones (which assume you almost always download and upload very little) to see the difference. Instead, the many light users subsidize the few heavy users.  BitTorrent, in which everyone helps take load off servers by uploading everything they download, often many times over, threatens this model — if everyone uploads, Internet rates will have to go way up.

Thus, ISPs often try to stop BitTorrent and other peer-to-peer systems.  They use copyright as an excuse, but really, they don’t care about copyright — they care about cost.  Your downloading costs very little.  Your uploading to other customers on the same ISP costs very little.  Your uploading to the Internet costs them quite a lot by comparison.  The most primitive way they’ve tried this is simple port-blocking — they ban connections to the port TCP/6119 (BitTorrent’s default) on all their customers PCs.  This doesn’t work very well — for one, it’s obvious (BitTorrent simply fails to function), and for another, BitTorrent doesn’t need to use any port in particular.  Due to the tracker, other peers can find you no matter what port you choose, so simply changing the default in your BitTorrent client gets around this.  Slightly less primitive is “traffic shaping” — the ISP slows traffic to the default port, or it inspects all traffic for BitTorrent headers and slows any packets showing them.  (The latter approach is much more expensive for the ISP, since it requires a deep inspection firewall on all traffic.)  Once again, changing port is easy.  In addition, some BitTorrent clients have added a header encryption feature to evade traffic shaping — this limits which peers are usable (specifically, to only other peers that support the header encryption), but evades the traffic shaping.  Comcast has recently been using the Sandvine intelligent traffic management system, which has caused some controversy since it actually impersonates the user and sends forged traffic on their behalf, in a further attempt to limit BitTorrent and other P2P traffic.

The above problems are inherent to BitTorrent, and at first, they seem inherent to all peer-to-peer systems.  However, the buccaneers of the Pirate Bay have come up with a rather ambitious plan to improve on BitTorrent, developing their own protocol to better suit their needs.  They’re still working on the specification (there’s a wiki up for suggestions), but I find it interesting the security and privacy issues they need to overcome.  At first glance, it seems the problems they must solve are the following:

• How can people upload pirated files without their IP addresses being detected by groups like the MPAA and RIAA?
• How can people hide the use of a file-sharing application so their ISP does not detect it and cut them off?

But that’s actually rather short-sighted, and the suggestions on the wiki seem to indicate that they’ve realized that, too.  Creating a new peer-to-peer protocol to replace BitTorrent for pirates requires not looking at the current attacks, but rather at the threats themselves.  The problem they really want to solve is simply to defend against these two threats:

• Legal prosecution for uploading pirated files
• ISP retribution for uploading large amounts of data

This is rather different!  What they want to avoid is not detection per se, but rather the current consequences of that detection.  In addition, they seek to address several technical/functional shortcomings of the BitTorrent protocol while they’re at it (such as that the tracker software does not scale to their traffic volume, and that upload bandwidth use in BitTorrent is suboptimal — many peers are not uploading anything.)

Right now, ISPs face no legal liability for transferring all this pirated media, since they are only content-indifferent carriers.  Thus, a system that allowed users to also be content-indifferent carriers (i.e. sharing data they did not choose to download as well as the files they acquire on purpose) might provide some legal protection.  The problem is that right now, users are from a legal standpoint sharing media they have, not simply transmitting media.  Thus, a system of “reflector nodes”, where the aforementioned suboptimal bandwidth use instead has the empty bandwidth filled by data relayed from other peers might work.  The ideal from an anonymity perspective would be onion routing, as performed by the TOR Project.  Unfortunately, this causes a serious growth in bandwidth requirements for all peers — basically defeating the purpose of BitTorrent.  Some balance must be found between true anonymity, as can be provided by a high-latency encrypted mix network with traffic-analysis resistance like TOR, and simple obfuscation, or even juggling around what is transmitted to be able to stick to the letter of the law while violating its spirit.  No one would believe that pirates don’t mean to transmit pirated software, the mix network just makes it look that way, but it doesn’t matter if anyone believes it so long as they can’t prove it beyond a reasonable doubt in a court of law.

Avoiding ISP retribution is a bit harder.  You can encrypt and use random ports, thus making detection impossible.  However, this causes a problem — if everyone does this, and everyone uses P2P, then everyone’s Internet rates go up!  This is hardly the desired outcome.  An ISP administrator has contributed some novel suggestions regarding changing the protocol to help ISPs save costs.  If the peer-to-peer system would deliberately prioritize other peers on the same ISP (ideally using WHOIS/ARIN data, though even simple CIDR subnets would help) for uploads, it could drastically reduce the ISP’s costs.  Napster provides a good example — during their heyday, when Napster pirated transfers were killing college networks, they worked with universities to institute just this type of solution.  The Napster client would look for other users at the same university to share with, only going to the Internet when this failed.  This type of solution — not fighting the method by which ISPs hurt P2P but rather fighting its motivation — is bound to work better.  It’s a good example of thinking about the threat, not about the particular vulnerability.  In addition, it’s probably the only way to fight things like Sandvine (which, due to the way it works, can’t be stopped by a BitTorrent client unless it went to full encryption with all the negative effects that has — lightweight ways to evade Sandvine require patching the TCP/IP stack and altering RFC-mandated behavior, which is doable by people willing to hack their OS but not something you can just bundle into your P2P software.)

Another issue that the Pirate Bay has is with fake files.  Sometimes, a user (either an RIAA/MPAA shill or just someone who likes being obnoxious) will upload a file of the approximate right size with a filename matching something new and popular (like a just-released movie or album) that contains no or bad data.  With nothing but the filename to go on, users download the fakes, causing the seed count to go up and making the fake appear even more “realistic” on the tracker — and hundreds of gigabytes of bandwidth are wasted.  Currently, the only thing to be done about this is to look at the uploader and ensure he is someone trusted, but identity is impossible to verify.  Some sort of digital signature/PKI system would be very helpful here.

Overall, it will be very interesting to see what they come up with.  Like all open-source projects, it may or may not actually get off the ground, and pirates are of course not well-known for their altruistic contributions.  However, it’s not likely the BitTorrent creators (who don’t get any money from pirates) will work on these problems, so it falls to people like the Pirate Bay to try.  Even if you don’t want pirated media, the resultant system could be useful for a host of purposes — the same technologies being used for fighting piracy and cutting ISP bills in the United States are used for hunting down dissidents and limiting free access to information in totalitarian nations.  In addition, a sufficiently large peering system with deep storage and forced reflectors (i.e. people sharing data they did not specifically choose to download or share) could result in a sort of distributed information well in which any human knowledge could be stored for easy access and rendered almost indestructible.  Criminals have been putting legitimate technologies to underhanded uses for centuries — an illegitimate technology can be put to beneficial uses as well.

Unfortunately, there is a slight problem with either solution: it’s pretty much impossible.

First, a bit about how ad networks work.  Whenever your browser loads a page with a banner or text ad on it, the page contains a link to the ad network’s web server telling it to load the ad.  As it does with any site, your browser first checks to see if it has a cookie recorded for that site.  If it’s the first time you’ve ever visited that ad network, then it does not; if you have visited before, then there is a unique ID number for you in the cookie.  The browser then sends a request to the ad network, along with a cookie (if any) and a referrer header (saying what page the ad was loaded from.)

The ad network site then looks up the ID in the cookie.  This ID is linked with a list of all the referrer headers it’s ever received from you — this is the “tracking” component.  It adds the new referrer header to the list, and then uses the list to try to puzzle out what sort of things you like and pick the ad it thinks you’re most likely to click on.  It then returns that ad.  If no cookie was received from you, it also creates an ID for you and sends that so as to set the cookie for next time.

That’s pretty much all it does.  There are variants, which also use script to inspect the pages you linked from and use that to make better predictions of what you want to see adds for, but the overall effect is the same.  The ad network doesn’t know who you are, or any demographic info about you — all it knows is that some person with a random ID has visited a specific list of sites.  In addition, there’s a simple way to dump all that tracking information — tell your browser to delete all the cookies (or just the ones for ad networks.)  Whenever you do this, the ad networks will all think you’re a “new” person and provide you with a new ID number.

So, how do we stop the ad tracking (should you even really want to)?  I can see a few possibilities, but all have some significant difficulties associated with them:

1.) Set a cookie that essentially sets your ID as “don’t track me, use random ads instead.”  Whenever you visit an ad network, this “do-not-track” ID is sent, and the ad network sends you back a random ad without bothering to record your referrer.  Issues: due to the same-site rule, this cookie must be set by each ad network itself.  So there’s no common registry — you have to opt out with each ad network, and then trust each ad network to continue to obey the opt-out.

2.) Install an app or modify the browser to dump cookies.  Works great; no more tracking.  Issues: also breaks half of the Web.  If you allow even per-session cookies, some limited tracking is possible, and if you don’t allow session cookies, you break pretty much all of the Web.

3.) Have your ISP scan all your web traffic, find cookies that are going to ad networks, and strip only those.  This makes the web work normally while killing ad networks.  Issues: requires all the ISPs offering this sort of technology to keep track of every ad network in the world so they know which cookies to block.  What about single-site ad networks? (e.g. the New York Times tracking which articles on their site you read and targeting ads based on those.)  There are probably tens of thousands of them.

Also, the above three examples are only pointing out issues when ad networks are not malicious – that is, they want to allow you to opt out if you so desire.  If they are hostile, then they can work around any of the above options.  They can simply disregard the do-not-track cookies and set a different ID, or track you via codes embedded in image tags.  The latter method is inferior, since it does not persist across sessions (it forgets who you are whenever you close your browser) without the cooperation of the actual sites the ads are on, but it does still allow some tracking capability.  Affiliate networks are constantly advertising and improving their “cookieless traffic” capabilities.

Of course, if the government cares to get involved, it can simply mandate that all ad networks offer an opt-out, and pursue legal action against any who don’t, or who evade their own opt-out systems.  However, what it can’t do is offer a centralized list like the Do Not Call Registry.  After all, the ad networks do not know who you are – they only know you are some random ID number who has visited various sites in the past.  Thus, they have no way to check against a list and see if you’re on it.  And since cookies can only be sent to the site they came from, the government site can’t set some kind of master “do-not-track” cookie — your browser would refuse to send the cookie to any ad networks!

However, before instituting a system like this at all, we should perhaps consider the unintended consequences.  The reason that ad networks institute tracking is that targeted ads are more valuable to advertisers than random ones.  A car company would rather show ads to car buffs than to people who don’t drive, and it will pay more for ads it knows are going to interested parties.  Thus, if ad networks cannot target ads with tracking, they will have to charge less for ads.  This means that sites will get paid less per ad for placing ad network links on their sites.  Therefore, eliminating ad network tracking means sites will have to carry more ads.  Is “more ads” really what we want here?  Are we willing to accept more ads to ditch the tracking?  How big a privacy threat is this, anyway?  There are people I don’t want to track my web surfing, certainly, but DoubleClick and Aquantive are not the people I’m thinking of here.  Perhaps what we need is not a way to opt out of ad tracking, but more limits on who can get that data?  Were ad tracking data illegal to resell and not admissible in court, would we care about it at all?  I’m not sure that I would.

Of course, much of this is moot if instead of opting out of the tracking systems, you just “opt out” of the ad networks altogether, either with a plugin like AdBlock (which advertisers hate) or a custom hosts file.  It doesn’t get 100% of the networks, of course, but it sure gets a lot of them.