Michael Aaronson MD
Break the Monopoly: Offer Google Alternative Yacy
to Search For Content on Your webpage or Microblog.
Penguin boys and Linux fans: the Aaronson Brothers.
Lamenting the FTC decision to allow Google to Monopolize Search, I reflected on ways we patriots can overcome the Big Brothers. My "must not miss" commentary on the Google-alternative, open source, search engine Yacy came to mind. Brilliant on many levels, this search engine has not yet reached critical mass and appears to have lost the war against the empire.
Why didn't this web crawling, peer to peer search engine take off?
In my opinion, at this juncture, the focus of competing with Google in hand to hand combat is impossible because they are a monopoly. Perhaps an alternative strategy is in order? Perhaps, until the higher ups realize we need protection, we can innovate such that Yacy can become a viable option in search allowing for the unbiased retrieval of information.
I submit for your consideration a new approach to web search that has the small possibility of bringing Google back down to earth. The secret is to use Yacy on a micro level, as opposed to a macro level, to search individual webpages or microblogs instead of the way many wanted which is to search the entire internet. Given the processing power to achieve search, too big a task or too many people searching at one time can break a search engine. However, if different peers were set up such that fewer people were using less processing power, Yacy might be able to slow Google down until the anti-trust police appreciate that Google is in fact a monopoly.
Let me show you the power of Yacy in this context. Let me show you the things it can do. Let me show you why the product is actually better than the Google product on many levels. By the time you have finished reading the blog you will see why the in-your-face ad approach has limited the ability of the people to fine-tune search by marginalizing key features that used to be present but cannot be present now because there is no more room on the physical page.
Let's start with my proof of concept. Here is the current, WORKING, view of Dr. Michael Aaronson's integrated search portal which allows for a detailed search of my website and microblog (twitter / facebook alternative). Please try it out by going to the main page.
Using Yacy, I have web-crawled, that is to say indexed, both my blog and microblog. I have made the search available to the public for your consideration. Disclosure: Google custom search is still present on many pages, because I'm addicted to Google, but the main index pages proudly display the Web Search powered by Yacy, sponsored by www.michaelaaronsonmd.com.
Let's compare the results of the two options (Yacy versus Google) using the search term "patient satisfaction" to see which experience is better when searching my website. Please note: we are not trying to search the entire web, just Dr. Aaronson's site. I'm interested to see which experience you find more enjoyable.
Notice the screen setup for the two products.
Yacy allows you to choose a Yacy Doodle, or a picture of your choosing to display with the search results.
The picture, when selected, will refer the user to the website of the webmaster's choosing. You are also allowed to provide a "snippet" of information above the search box such as an important message, a humorous saying, or MORRISSEY quote.
I decided to promote my main blog as well as give Linux a plug, showing the Aaronson boys pretending to be penguins.
Google provides you with nothing. You do not have the option to customize the search experience.
Cache for the website, directory tree, outbound and inbound links, and the ability to personally promote or demote particular results: Yacy provides you all of this detail. Webmasters could use Yacy as a way to back up their site. If the main site goes down, Yacy still provides functionality and allows for productivity, providing an exact copy of the live webpage (also defined as a cached page).
Yacy provides the directory structure of the webpage. It also gives you the outbound links from the webpage as well as the inbound links to the webpage. You can easily select the website you want to check out and be transferred there:
Yacy, instead of paying people to review websites to improve results, allows the consumer to recommend (+), delete (-), and/or bookmark (B) a search result. This functionality allows the consumer to use the human factor to promote or demote a search finding.
Again, Google provides you with nothing.
Back in the day, Google provided cached pages. My understanding is that they were inadvertently providing password-protected content, and businesses were getting upset, so the cache was removed for everything. Unfortunately, if your main page goes down, your site is totally down. If you change servers or leave the internet, the information is lost forever. Not so using the Yacy cache. That said, why doesn't Google offer cache to non-password protected websites? I dunno.
Yacy found 43 results.
Google found 37 results.
The tag cloud:
Yacy shows on the right side of the figure a list of key terms that form the shape of a cloud. The bigger-sized words suggest the term is used more than the smaller-sized words. If you select one of these terms you have the ability to further narrow your search.
You can click on more than one term here. As you select options in the tag cloud, your search is further narrowed. This feature is really amazing!
Google does not provide you with this functionality. The right hand side of the screen is filled with ads. Go figure!
On the right hand side of the Yacy screen you will see different "Navigators" which allow you to further narrow your search to the level of the individual website.
That said, when setting up the program to work with my webpage, I asked the search engine to have a "crawl depth" of 3-4 links. This just means that I have spidered, or crawled, 3-4 links associated with the outgoing links I provide on my website. I chose this setting to demonstrate the domain navigator concept.
You will notice that there are 43 found pages for "patient satisfaction" from www.michaelaaronsonmd.com, 5 from youtube, 1 from twitter, and 1 from Google plus.
If you select the box next to the website you want, the search narrows further. This approach lets you view results from trusted sites and avoid those websites you do not have interest in.
Google does not provide this navigating functionality.
Yacy by default does not show ads. There are ways to make this happen if you wish it, however.
Google does display ads shown by the black arrows. Notice the ads are placed above and to the side of the search results.
With the latest FTC ruling, Google can now favor their webpages below the ads in the "organic" part of the results portion (which is basically now another ad for them), which may push meaningful results from other sources to the 2nd, 3rd, or 4th page.
Research suggests that instead of going to page 2, most people will just search for different terms in an attempt to find what they are looking for (unpublished data evaluated by me.)
With respect to ads, Google has you covered. You see ads on all the pages, both top and side, regardless of whether you re-submit a search query or go on to page 2.
Sidebar: thanks to the FTC, Google can now promote itself and its other services on the entire front page. If you are a content provider and want people to see you, you can forget about obtaining high results based on content and creativity in the "organic" search because Google knows better and owns the entire experience.
If you are an advertiser, in order to exist and for your ad to be displayed, you will have to pay Google for the pleasure and privilege of product placement. Let's develop this concept:
Let's say that I provide the same "advertisement / tweet" on Google plus and Twitter.
Let's say that it costs less to advertise on Twitter.
Let's say Google doesn't allow for competition because the Tweet on Twitter is now on page 2 because of the shenanigans while the Tweet on Google plus is preferentially placed in the organic search (main section) by design on page 1.
Let's say that in order to be seen, the advertiser must pay Google the pound of flesh (defined as something that is one's legal right but is an unreasonable demand -- Shakespeare, The Merchant of Venice, Act IV, scene 1, 1596).
Even though Twitter is offering less for the same tweet, they lose money not because they are charging less but because nobody sees the tweet and therefore can't click on something they do not see. Twitter can't compete. Search is forcing people toward the "Google plus" product which hurts competition.
That said, if you define the consumer as the advertiser, that person pays more for the same thing in order to be seen and survive. In my opinion, this is no different practice than Microsoft "forcing" Office on Windows users like they did back in the day. For those that do not remember, this was viewed as monopolistic behavior. And that's why I think the Justice Department or the European Union needs to step in and stop the monopoly.
With respect to autocomplete and "did you mean" functionality, it turns out both products offer this feature.
Searching takes processing power. So does crawling a particular site. Yacy has the power to be uber-powerful given its peer to peer design. However, if nobody offers up the processors, there is no power. That said, from personal experience running a simple, country nephrology webpage, I found it important to have most of my website crawled before opening up my search to the public. Too many people trying to do too many processes at one time slows the system down too much.
Conclusion: when comparing a search result of the entire web, Google at this juncture will beat Yacy on the basis of computing power and time to obtain results. This is on a macro level. That said, when you compare functionality and quality, Google's innovations do not seem to be in the search arena as I have just demonstrated. They seem to be offering us less real search results and more monopolistic maneuvers and getting away with it! It turns out that Yacy is extremely powerful, and if enough peers joined the movement, there would be enough processing power to beat Google. That has not yet occurred and likely never will as the world runs away from Windows 8 in droves favoring tablet and smartphone based devices, as you know.
On a micro level, in this context meaning placing a search engine that provides results for a particular website, the argument is the same. The results Google now displays are biased based on the person searching, objectivity is lost, the ads are distracting, the brainwashing affects my ability to find a truly meaningful result, my bank account shrinks, and thanks to the misinterpretations of the FTC mistakenly opining that consumers are not harmed) Google now has carte blanche to favor their products in organic search, increase advertising costs which hurts consumers, put competitors out of business by placing them on the second page -- you know, the one that nobody goes too -- and in sum, operate their monopoly untethered using an inferior product.
It is possible, although unlikely, to break the monopoly by offering Yacy to users to search for content on the individual webpages. Mom and Pop operations such as myself do have the capacity to do this to a certain extent. However, to really break the monopoly, in addition to Yacy, I think we need help from people with power. I am still hoping the justice department will step in, but for now it seems that the European Union is our only hope.
About the Author: Dr. Aaronson blogs about relevant topics to keep you and your family happy and healthy using the wisdom of MORRISSEY.