Tag Archives: search

Blocked from Google search: agree our terms or else go away

Late last week I encountered behaviour from Google that I had not seen before. It was related to my habit of not signing into Google automatically; I only sign in when I want to use Google+ (I know; but I have quite a few followers there and use it from time to time). Nor do I always use Google for search; I have Bing set as default, but Google is better for some kinds of searches – such as the kinds of searches admins and developers make when trying to fix a problem – so I use whichever one I think will get me the best results.

I therefore hit Google, only to find that I could not proceed. The only thing on the page was a notice stating that I could only continue using Google services if I “review key points” of Google’s Privacy Policy.

google-privacy

The word “review” turns out to be misleading. The next banner you see asks you not only to review but also to click “I agree” to a range of statements including delivering “ads based on your interests”.

google-privacy2

If you click “Other options” you find that there are none, other than a list of minor tweaks you can make to your settings on a per-browser basis.

In my experience these do not work well anyway. I have opted out of every interest-based ad that I can, but I still see many ads that are obviously “interest based”, such as ads that mysteriously match recent searches on ecommerce sites.

Once you have reviewed these options, you have to go back and click “I agree”, or give up using Google search.

Most web sites in the EU now have at least some form of cookie consent banner, but in my experience it is rare that a site blocks you completely. Some simply state that by continuing you implicitly agree to their terms. Some let you dismiss the banner with an x, leaving ambiguity about whether or not you agree. Google has gone for the nuclear option: unless you specifically agree, no search for you. I found the same banner both on Google.co.uk and Google.com.

My immediate question was in what circumstances Google chooses to block search (and other services) until you agree its policies. I asked Google, but have yet to receive a reply; if and when I do, I will update this post.

It seems that some others also noticed this change of behaviour. Privacy advocate Aral Balkan tweeted about it.

image

Google is doing the right thing here, if it is not willing to let you use search or YouTube, for example, unless you agree its policies. No doubt it is trying to stay the right side of the law, especially in the EU.

At the same time, I do for some reason find this disturbing. Agreements like this are one-sided; there is no use in my trying to get Google to change some clause or other because I do not agree. Further, the extent to which I have choice in the matter is limited. Perhaps I can do my job without Google search, but without *any* Google services? What if I want to report on a conference where the sessions are on YouTube? Google has not created this content, but does deliver it. Is it reasonable for me to tell people, sorry, I cannot report on that, because it is on Google and I do not agree with its privacy policy, that feeds me pestilential interest-based ads and records my data in ways I cannot control?

You have choice but not that much choice; and the same applies to Facebook, where you may information that requires log-in and cannot easily be obtained elsewhere. I am sure that to most people putting something on Facebook is exactly equivalent to putting it on the Internet, but it is private property and there is a distinction.

Reflecting on Google’s power: a case for regulation?

Via Martin Belam’s blog I came across this account of how the well-known flower vendor Interflora has, it is claimed, been penalised by Google for violation of its webmaster guidelines on paid links:

Searching for the terms [Flowers], [florist], [flower delivery], [flowers online] and hundreds of other related search terms yielded the interflora.co.uk domain in first place – until yesterday afternoon.  Now the website does not even appear for its own brand name.

Possibly by no coincidence, an official Google post reminds us of the rules:

We do take this issue very seriously, so we recommend you avoid selling (and buying) links that pass PageRank in order to prevent loss of trust, lower PageRank in the Google Toolbar, lower rankings, or in an extreme case, removal from Google’s search results.

I find this troubling. Here are a few statements (some may be contentious) that taken together will, I hope, express why.

1. Google has a market-dominating position in search, certainly in the UK. With good reason, users wishing to visit Interflora’s site are more likely to type “interflora” into a search engine, probably Google, then to type the URL directly. The combined address bar and search box in most browsers encourages this. Many users probably do not appreciate the difference. Of course they might also type “order flowers” into the box, delegating to Google the responsibility for finding suitable sites.

2. In consequence of 1, Google has direct and immediate power over the amount of business that will be achieved by a company trading online. In some cases that might be make-or-break, in some cases not, but it is a significant influence.

3. A further consequence is that Google’s search and ranking algorithms form an incentive to businesses to do all they can to climb higher in the search ranking. Since this appears to be influenced by incoming links (though probably less so than it once was) Google’s algorithms attempt to judge which incoming links are meaningful and which are not. Paid links fall into into the latter category, hence the guidelines which prohibit them.

4. Despite (3) above, the internet is infested with paid links and link exchanges. Even running a small site like mine, I get thousands of paid link and link exchange requests every year. The implication is that Google is not all that good at ignoring and/or penalising them, otherwise the activity would cease.

5. Worth noting: web site owners are free to accept paid links and vendors are free to buy them. They are not doing wrong. The only disincentives are first, whether you want to fill your site with worthless links, and second, whether you will be penalised by Google for doing so.

6. Google’s process for determining whether or not a particular web destination is down-ranked is not transparent. This is for good reasons, insofar as a transparent process would arguably be easier to game. On the other hand, this also means that a business which is penalised has no recourse other than to plead with Google, unless it felt inclined to experiment with legal action (prohibitively expensive and uncertain for most).

7. In fact there is another option, which is to advertise with Google, a form of paid link which the search giant is happy to accept. It seems to me obvious that this form of advertising is designed to look similar to unpaid search results, despite some small effort to distinguish them with small print and a light background colour change:

image

It is not clear to me that this intermingling of paid and organic results is in the user’s best interests.

8. It is also obvious that advertising in this form is more important in cases where a business is absent from organic search results. It follows that Google has a direct incentive to penalise businesses by downranking them, since it has the potential to bring more advertising business. Please do not misunderstand: I am not accusing Google of doing this and have no reason to believe that it does.

9. Users of Google will be grateful that it attempts to improve the value of its search results by reducing the influence of meaningless incoming links. On the other hand, I find it difficult to understand why a user who typed “interflora” into Google would not want to see the official site at the top of the list, since it is a legitimate business and not in any sense malicious. Of course they do in fact see this, judging from my own experiment minutes ago, but it is an advertisement and not an organic link. The top organic link is not Interflora’s own site.

10. Pause for thought: what would be the effect on Google’s business if it put ads below organic search results rather than above?

11. The only rationale for (9) above is that Google considers it worth inconveniencing its users (presuming you do not accept that it simply wants to sell more ads) for the sake of the higher objective of penalising sites which, in its view, breach its guidelines.

12. We all have a choice whether to use Google or not; but this choice is not one that fixes the problem. The problem, rather, is the choice which our customers or potential customers make, over which we have no control.

13. It is a company’s duty to maximize returns to its shareholders. Making a profit is not wrong, and Google is entitled to design its search algorithms and web site as it wishes. None of the above is intended to imply that Google is doing wrong.

14. Despite (13) above, the combination of this concentration of power in a single business entity, the lack of transparency in its procedures, and the difficulty smaller businesses (in other words, almost everyone else) have in fixing issues, is something I find troubling.

15. It is also worth noting that the power of a dominant search engine goes beyond SEO (Search Engine Optimization). There is a long-standing debate over how easy it should be to find sites which offer illegal music downloads, for example. Another recent case I encountered showed how Google can make it hard to find a business in the real as well as the online world. I also note the influence of search engines on education, as the first destination of students and pupils looking for answers, and on human knowledge in general.

These issues are both complex and important. Should Google be regulated? Should all search engines be regulated? I do not know the answer, but believe that the question merits wider discussion. In this instance, it is not obvious to me that the free unregulated market will achieve the best outcome.

Extraordinary: the FTC says it is OK for Google to bias search results in its own favour

The most remarkable statement in the report from the US Federal Trade Commission’s investigation of Google is this one:

The FTC concluded that the introduction of Universal Search, as well as additional changes made to Google’s search algorithms – even those that may have had the effect of harming individual competitors – could be plausibly justified as innovations that improved Google’s product and the experience of its users. It therefore has chosen to close the investigation

In other words, the FTC did not find that there was no bias in Google’s search results. It found that bias is OK if it “improves Google’s product and the experience of its users”, a phrase which is something I would expect to hear from a company’s own public relations team, not from a government report.

It is an extraordinary conclusion and runs counter to normal expectations of what a government body investigating anticompetitive business practices would be likely to support. It does make me wonder if the FTC appreciates the power of Google over which web sites are visited; given the use of the search engine by people such as students and journalists the company has remarkable potential influence over a wide range of human knowledge, as well as the power to make or break a company for which the web is critical either for direct sales or for marketing.

I also wonder what precedent it sets. In other words, can any company justify activities that harm competitors unfairly by claiming that they “improve the experience” of customers?

Update: It looks like the EU may take a stronger line, according to this article in the Guardian. From which I cannot resist posting a screenshot.

image

All about Search in Windows 8: a feature every user has to understand

Windows 8 is a frustrating experience until you work out how Search works. Once it is discovered though, it is an elegant and powerful feature.

A confusing aspect of Windows 8 apps (on the tablet side) is that features such as menus and toolbars are hidden by default. There can be menu bars at top and bottom of an app, but you have to display them either by right-click or by swiping in from top or bottom. There is a philosophy behind this. Microsoft has called it the “immersive user interface”, one which puts content first and hides anything distracting.

Even this will not generally show Search though. Rather, Search is in the Charms bar, which you show by swiping from the right or pressing Windows key and C. There are also better shortcuts for Search, which I will come to in a moment.

When you click or tap Search in Windows 8, you are really in a kind of search centre. Take a look at the following screen, where I have displayed Search and typed “Keyboard”:

image

Note that the active search is restricted to Apps which have the word keyboard in their name. However, if you look at the right column, you can also see numbers: 25 against Settings, and 911 against Files.

This means Windows has found 25 settings for the keyboard, and 911 files. If you click or tap the different context, you see the new results.

image

Even this is not everything. The column below shows apps which have a search feature, such as Wikipedia, Store, Maps and Bing. If I tap or click Wikipedia, for example, I see encyclopaedia entries for keyboard.

image

Bing of course would give me an Internet search. Even Tunein Radio gives me results, and I can hear a broadcast about How does a QWERTY Keyboard work:

image

What this means is that the Search feature is a fast and efficient means of navigating the Windows user interface, finding documents and files, and discovering information from a variety of sources.

To get the most our of search, learn the following shortcuts:

  • Windows key and Q: Search apps
  • Windows key and W: Search settings
  • Windows key and F: Search documents and files

These shortcuts save you some clicks. If you use the mouse or Windows key and C to show charms, you have to then click or tap Search, and then choose the context you want. The shortcuts on the other hand get you a cursor ready to type your search. Search is incremental, so often just a few letters will do.

There is one subtlety. If you are  in the Start screen or on the Desktop and press Windows key and Q, you will search for an app by default. However, if you are in a Windows 8 tablet app, the shortcut will search within the current app by default. This is an inconsistency, and annoying if it comes up “This app can’t be searched”, but you can understand why it is designed that way. Otherwise, we would have four shortcuts to learn.

Google: a search engine, or affiliate site?

According to my current web stats, 95.6% of those using a search engine to find a post did so using Google. That represents market dominance, and power to make or break a business which depends on web traffic.

Google’s search engine is the best in my experience, but I am increasingly concerned about the quality of the results, which are noticeably worse today than they were in the early days.

Ideally (from the user’s perspective) its search results should be objective as far as possible; for example, it should not favour sites which spend more money advertising with Google, nor should it favour Google’s own web properties above rivals.

I noticed an article in the Guardian stating that this is not the case:

A Google search for credit cards returns with an advert at the top of the screen, far bigger than the rest and bigger than any other website link. Adverts of this size and prominence will attract a high click-through rate. This will prevent searchers going via other affiliate sites or applying directly for a credit card.

I tried it. Here is what I got:

image

The most prominent results is the one with the images, admittedly marked “sponsored” but in a grey, small font that you could easily miss. This is actually an ad for Google’s own affiliate site for credit cards, just click Apply:

image

I do not get the same issue with Bing, although I do think the designation of which results are ads is too small:

image

Still, at Bing has not awarded itself a large ad with images that links to its own affiliate scheme.

Of course I can choose not to use Google. Unfortunately though, businesses cannot choose what search engine their customers or potential customers use to find their sites.

I am one of those who believes regulation should be as light as possible, but considering the power Google currently exerts and the lack of fairness in examples like this, it seems to be that some kind of regulation is needed.

Disclosure: this site uses Adsense, a web advertising scheme operated by Google

The power of Google: how the Panda update hit Experts Exchange

Searching Google recently it struck me that I rarely see results from Experts Exchange. I used to see a lot of these, because I typically search on things like error messages or programming issues for which the site is a useful source.

The site is controversial, because it (kind-of) charges for access to its knowledgebase but does not pay its experts. I posted about this back in 2009. That said, the quality of its advice is often good, and most answers are available without payment if you scroll far enough down the page. You can also get free access as an expert if you answer a few queries successfully.

Experts Exchange has to some extent been replaced by the StackOverflow group of websites, which are nicer to use and free, but I have found that the chances of getting your obscure query answered can be higher on Experts Exchange, particularly for admin rather than programming queries (of course for admin I am comparing with ServerFault).

Still, I wanted to test my perception that I no longer see Experts Exchange results in Google. I had a look at the Alexa stats for the site.

image

Wow! That vertical line is around April 2011, which is when Google rolled out its "High Quality Sites Algorithm". The site still ranks in the top 3000 in the world according to Alexa – 2787 at the time of writing – but according to the chart it lost around 50% of its visitors then, and has since declined further.

As noted above, the site is controversial, but I personally never minded seeing Experts Exchange results in my searches since the advice there is often good.

The bit that disturbs me though is simply the power Google has over what we read on the Internet. I appreciate the reasons, but it is not healthy for one corporation to have this level of influence, especially bearing in mind the black box nature of its workings.

Finding apps in Windows 8

I’ve been spending some time in Windows 8, complete with the Metro-style Start menu. The new Start menu is not great in a virtual machine without touch. You have to navigate with the horizontal scroll bar at the bottom of the screen, which I find somewhat jerky. DirectX is not particularly fast on the VM, which I view using Remote Desktop. Another puzzle is that many apps are not represented by tiles on the big menu. How do you find them?

The answer is that the tiles are not exactly equivalent to the items on a Windows 7 Start menu. They are more equivalent to the items that you have pinned to the taskbar in Windows 7; and in fact, if you pin an app in Windows 8, it appears as a tile. To get the full list of Start menu items you are meant to start typing.

image image

When you press a key in the Metro Start menu, the tiles disappear, and you get a list of all the apps that match what you have typed. The match seems to be based on words with initial letters that match your string. So if you type CL you match Disk Cleanup, but typing PAD does not find WordPad.

If you press Windows key – Q it brings up the Search Apps screen with an empty search box, and lists all your apps in a columnar A-Z view.

image

Now, the thing that gives me hope here is that the search is very fast, much faster than I am used to with search in the Windows 7 start menu. It could almost convert me.

Search is baked into Metro, and searching apps is only one of the options. You can also search settings and files (though files seems to mean documents), and any applications that implement the search contract. For example, from the same search pane, you can search the custom BUILD app for c#:

image

One other thing: “All Apps” is not really all apps, of course; just the ones that would have been in the Windows 7 Start menu. If you want, say, Dxdiag, you will have to press Windows Key – R and type it in there. It would be nice if typing DX would find it, but it is not that smart unfortunately (update – but see comments below).

Google favours big brands over diversity

Google has made a change to its search algorithm that means most of the results shown for a search may now come from a single domain. Previously, it would only show a couple of results from one domain, on the assumption that users would prefer to select from a diversity of results.

The example chosen by searchengineland is a good one. Search for Apple iPod and you get a page that is mostly links to Apple’s site.

image

If you search for the single word ipod you get more diversity – odd, since only Apple manufacture the ipod so you could argue that the searches are the same. Some people use ipod as a generic name for MP3 player, but that doesn’t seem to be reflected; all the results still relate to Apple’s device.

image

Personally I’d rather see diversity. I don’t see the need for this change, since the site summary with deep links works well when a particular domain closely matches the search term. You can see an example of this in the top result for the ipod search above. Note that it even has a link for “More results from apple.com”. What is the value of suppressing the results from other domains?

The overall impact is that big brands benefit, while smaller businesses and new entrants to markets suffer. It also makes independent comment that bit harder to find.

While to most of us changes like these are only of passing interest, to some they make the difference between a flourishing business and a dead one. Google has too much power.

Incidentally, I generally find Google significantly better than Bing, now its major competitor. However in this case Bing impresses, with categories such as reviews, prices, accessories, manuals and so on; and in the case of the Apple ipod search, a better balance between the official site and independents.

Oracle still foisting Google Toolbar on Java users

Oracle may be suing Google over its use of Java in Android; but the company is still happy to take the search giant’s cash in exchange for foisting the Google Toolbar on users who carelessly click Next when updating their Java installation on Windows. If they do, the Toolbar is installed by default.

image

This is poor practice for several reasons. It is annoying and disrespectful to the user, particularly when the same dialog has been passed many times before, bad for performance, bad for security.

Sun at least had the excuse that it needed whatever income it could get.

I know certain other companies do this as well with their free runtimes – Adobe is one – and I like it just as little. However, as far as I can recall Adobe only adds foistware on a new install, not with semi-automatic updates.

Google flexes page rank muscles, hits Daily Express?

It’s been reported that the Daily Express newspaper is selling paid links, in other words links that look to Google’s web crawler like links from independent editorial, but in fact are paid for by advertisers.

The consequence of doing this, according to Google:

However, some SEOs and webmasters engage in the practice of buying and selling links that pass PageRank, disregarding the quality of the links, the sources, and the long-term impact it will have on their sites. Buying or selling links that pass PageRank is in violation of Google’s webmaster guidelines and can negatively impact a site’s ranking in search results.

I then saw a report saying that Google has indeed penalised the Daily Express. I checked the page rank of the Daily Express home page here and found that it is indeed lower than could be expected.

image

While it is difficult to say what the page rank should be, Alexa shows the Express site as among the top 15,000 worldwide and in the top 1000 in the UK, with over 4,000 incoming links.

Although the Daily Express is not to my taste, I have misgivings about this process. A significant proportion of web traffic comes via Google, and lower search rankings have a direct effect on traffic and therefore business. Exactly how Google determines search rankings is a commercial secret, even though the gist of how it works is well known. There is not much you can do if mistakes are made, other than to complain to Google and hope someone pays attention to you.

There is nothing illegal about selling paid links, and the article highlighted in the Express is marked as “Sponsored”. While it is right for Google not to count these links as genuine recommendations, I am less sure about whether there should be additional punishment for running them. It puts too much power to make and break other companies into the hands of Google.

Of course we don’t know if the low ranking is a result of the paid links or not; that is speculation. Nor do we know how closely the published page rank corresponds which how Google actually determines the order of search results. Looking at Alexa’s report, there’s no conclusive evidence of declining traffic, though Alexa’s figures are based on a relatively small sample.

Still, I get a ton of these paid link requests and this kind of story makes me glad that I always turn them down.