Despite many representations of a nefarious underground operating out-of-sight, Data Doctor's Ken Colburn explains what the deep web really is.
Q: What exactly is the “deep web” and how do you get to it?
Despite many representations of a nefarious underground operating out-of-sight, the deep web is mostly benign private databases and web resources not meant to be accessed by the general public.
The “surface web” is essentially what can be indexed by search engines such as Google or Bing, while the “deep web” consists of items that can’t be accessed using a search engine through a standard web browser.
Protected internet databases like those for banks, anything past a login screen like your private files stored in the cloud, and data stored by private companies aren’t indexed by search engines.
Websites can also specifically tell the search engines that they don’t want to be indexed, making them relatively “invisible” to the average user.
Most estimates put the deep web in the 90 percent range of the entire internet, because so much of what is stored online is protected information that requires some form of authentication or knowledge of a hidden web address.
The dark web
There is a very small percentage of the deep web where secret, and sometimes nefarious, activity is taking place, often referred to as the “dark web” or the “darknet.”
The tools used to access the dark web focus on anonymity by incorporating encryption and specialized privacy browsers like Tor. Also known as “The Onion Router,” Tor uses a large network of relays to bounce internet traffic through; it’s much like the layers of an onion, to make it much more difficult for anyone conducting any type of surveillance to see who is doing what.
The core technology used in Tor was actually developed by the U.S. Naval Research Lab in the mid ’90s for use by the intelligence community for protecting online communications. To this day, Tor and other similar tools are used by governments, activists and whistleblowers to communicate anonymously.
The Tor Project states: “Tor users include ‘normal people’ who wish to keep their internet activities private from websites and advertisers, people concerned about cyber-spying, users who are evading censorship such as activists, journalists and military professionals.”
Using Tor alone doesn’t mean you’re completely anonymous, and for most users, the trade off in slow performance isn’t worth the increase in privacy for daily surfing.
Tor’s dark side
Tor and other similar tools are also being used for illicit activities such as buying and selling drugs, stolen credit card numbers, stolen IDs, money laundering and more via black markets only accessible on the dark web.
One of the most famous dark web marketplaces was called Silk Road, which was shut down by the FBI in 2013. The site’s founder, Ross Ulbricht, was sentenced to life in prison without possibility of parole.
Despite law enforcement attempts to control illegal underground marketplaces, when one is shut down, two more seem to pop up.
To be fair, not all dark web resources operate in an illegal manner, and much of the activity taking place is vital to the efforts of law enforcement and the intelligence community. As with any technology, it’s impossible to control its uses for only legal purposes; so, as always, it’s the good with the bad.