World Video | Defence | Foreign Affairs | Natural Events | Trade | NZ in World News | NZ National News Video | NZ Regional News | Search


Widespread Hijacking of Search Traffic in the US

August 4th, 2011

Widespread Hijacking of Search Traffic in the United States

Technical Analysis

By ICSI researchers Christian Kreibich, Nicholas Weaver and Vern Paxson, with Peter Eckersley.

Earlier this year, two research papers reported the observation of strange phenomena in the Domain Name System (DNS) at several US ISPs. On these ISPs' networks, some or all traffic to major search engines, including Bing, Yahoo! and (sometimes) Google, is being directed to mysterious third party proxies.

A report in New Scientist today documents that the traffic is being rerouted through a company called Paxfire. This blog post, coauthored with one of the teams that discovered the phenomenon, will explain the situation in more detail.

Who is rerouting this search traffic?

The published research papers did not identify the controller of the proxy servers that were receiving the traffic, but parallel investigations by the ICSI Networking Group and EFF have since revealed a company called Paxfire as the main actor behind this interception. Paxfire's privacy policy says that it may retain copies of users' "queries", a vague term that could be construed to mean either the domain names that they look up or the searches they conduct, or both. The redirections mostly occur transparently to the user and few if any of the affected ISP customers are likely to have ever heard of Paxfire, let alone consented to this collection of their communications with search engines.

The proxies in question are operated either directly by Paxfire, or by the ISPs using web proxies provided by Paxfire. Major users of the Paxfire system include Cavalier, Cogent, Frontier, Fuse, DirecPC, RCN, and Wide Open West. Charter also used Paxfire in the past, but appears to have discontinued this practice.

Why do they do this?

In short, the purpose appears to be monetization of users' searches. ICSI Networking's investigation has revealed that Paxfire's HTTP proxies selectively siphon search requests out of the proxied traffic flows and redirect them through one or more affiliate marketing programs, presumably resulting in commission payments to Paxfire and the ISPs involved. The affiliate programs involved include Commission Junction, the Google Affiliate Network, LinkShare, and When looking up brand names such as "apple", "dell", "groupon", and "wsj", the affiliate programs direct the queries to the corresponding brands' websites or to search assistance pages instead of providing the intended search engine results page.

What can I do about it?

If you want to know if the network you're currently on is subject to this type of traffic redirection, you can run a Netalyzr test. And the best protection against the privacy and security risks created by this type of hijacking is to visit sites using HTTPS rather than HTTP, which can easily be achieved using EFF's HTTPS Everywhere Firefox extension.

More technical details below...

A detailed explanation

For most users of the World Wide Web, visiting a website equals clicking on a link to the site or entering the site's name into their browser, and receiving the corresponding page from the site. Users generally assume that the site's name is identical to the site itself, and essentially trust the site's authenticity if it looks as usual and the browser does not pop up phishing warnings or other signs of trouble. Paxfire's misdirection of search traffic undermines this trust.

The ICSI Networking group develops and operates the ICSI Netalyzr, a tool that tests the characteristics of users' Internet connections. Netalyzr's measurements show that approximately a dozen US Internet Service Providers (ISPs), including DirecPC, Frontier, Hughes, and Wide Open West, deliberately and with no visible indication route thousands of users' entire web search traffic via Paxfire's web proxies.

To explain these redirections further, we first need to delve into the workings of the Internet a bit. Since the Internet does not route traffic to names but to network addresses, contacting a website involves translating the site's name (say "") to the IP address (say of a computer that runs Google's web server. It is to this address that the browser actually sends its request. The Domain Name System (DNS) is in charge of facilitating this mapping of names to addresses. It is the Internet's equivalent of telephone books.

Usually, ISPs provide DNS servers (directory assistance, essentially) for their users. When a user's computer asks to map a name to an IP address, the user's system contacts the ISP's DNS server, which looks up the correct IP address for the name and returns it to the user. As currently implemented, this process does not provide any guaranteed correctness. In essence, users must trust their ISP's DNS servers to correctly return IP addresses that indeed belong to the site the user intends to visit. In some instances, however, this trust may not be warranted.

For a while now, a number of ISPs have worked in cooperation with Paxfire and similar businesses like Barefruit and Golog to profit from mistakes that users make when typing names into their browsers. Paxfire provides a product for ISPs that rewrites DNS errors (effectively conveying "the name you asked for doesn't exist") to responses sending users to search pages that host advertisements, for which Paxfire then shares the corresponding ad-related revenue with the ISPs. This practice has already been controversial.

Rerouting of requests to and responses from search engines

Paxfire's product also includes an optional, unadvertised, and more alarming feature that drastically expands Paxfire's window into users' traffic. Instead of activating only upon error, this product redirects the customers' entire web search traffic destined for Yahoo!, Bing, and sometimes Google, to a small number of separate web traffic proxies.

These proxies collect the users' web searches and the corresponding search results, mostly forwarding them to and from the intended search engines. This allows Paxfire and/or the ISPs to directly monitor all searches made by the ISPs' customers and build up corresponding profiles, a process on which Paxfire holds a patent. It also puts Paxfire in a position to modify the underlying traffic if it decides to.

Under specific conditions, the Paxfire proxies do not merely relay traffic to and from the search engines. When the user initiates searches for specific keywords from the browser's URL bar or search bar, the proxy no longer relays the query to the intended search engine, but instead redirects the browser's request through affiliate networks, as the equivalent of a click on advertisements. Using the names of popular websites, we have so far identified 170 brand-related keywords that trigger redirections via affiliate programs and result either on the brands' sites or on search assistance pages unrelated to the intended search engine results page.

The subset of customers affected varies from temporally localized deployments to apparently entire customer bases. The DNS-based redirection operates in a surgical fashion, affecting only search engines but not other services such as Google Maps or Yahoo! Mail, and remains completely invisible to the user. The treatment of Google queries varies. Charter and Cogent appear to redirect only Bing and Yahoo, while DirecPC, Frontier and Wide Open West also used to redirect Google to Paxfire proxies located within their own networks. Google has recently put significant pressure (see the answer to the question) on the ISPs to get them to stop redirecting Google searches. As of August 2011, all major ISPs involved have stopped proxying Google, but they still proxy Yahoo and Bing.


© Scoop Media

World Headlines


Tale Of Two Pandemics: Follow The Science And Do Not Forget One At The Cost Of The Other

Covid-19 has posed innumerable health, economic, and social challenges for all, including people living with HIV. It has exposed the fragility of health systems around the globe and has diverted political attention and funding from other infectious diseases like TB and HIV... More>>

UN: Rights Chief Calls For Prompt Release Of Protestors Held In Cuba
The UN High Commissioner for Human Rights on Friday called for the prompt release of protestors and journalists detained during anti-government demonstrations in Cuba, some of whom are being held incommunicado... More>>

Scarce Goods: Isolating Daraa Al-Balad Threatens 40,000 With Starvation

The siege imposed by the Syrian government forces on Daraa al-Balad since June 24 would lead to serious humanitarian repercussions if it continues, Euro-Mediterranean Human Rights Monitor said in a statement on July 15, calling for lifting the siege urgently and allowing the entry of basic humanitarian supplies... More>>

Focus On: UN SDGs

Shaping The Future Of Food Systems: Thousands Commit To Dialogues Amidst The COVID-19 Pandemic

More than 130 governments are making food systems a top priority amid the pandemic and committing to an unprecedented programme of Dialogues in the run up to the UN Food Systems Summit in September... More>>

UN: Play:Fair For People And Planet – A Major United Nations Music Activation
organized by the UN SDG Action Campaign in partnership with Music Innovation Hub, Keychange, the city of Milan, the Milan Triennale, and partners from the SDG Music Network, will be held at an unexpected location in the center of Milan, Italy, taking into account safety measures with a limited on-site audience consisting of activists and fans... More>>

UN: Next 18 Months Seen As Pivotal In Global Efforts To Achieve Key Goals

Next 18 months seen as pivotal in global efforts to reverse punishing pandemic impacts and boost actions to achieve key goals - Even as pandemic erases decades of gains in development, response efforts show signs of renewed global commitment to accelerate SDG progress... More>>