It was June 6, 2002. My desk phone (remember those?) rings and someone named Ray Sidney from Google wants to talk with me about OptiLink. Seems it might violate their ToS.
This was only 15 days after OptiLink was announced, so I’m calling that proof that someone at Google was reading Michael Campbell’s newsletter! 🙂
I invited Ray to just download the software and try it, but he had visions of Trojan horses invading his computer and galoping throughout the Googleplex [wish I’d thought of that!] so he declined.
All I needed to do was shoot a 3 minute screen recording to show him how OptiLink worked, but did I mention this was 2002? That was hard work back then! And a 10Mb mov file doesn’t seem like much now, but you notice it on a 250Mb hard drive (yes, that’s an M, not a G!).
I finally got the new-user video recorded and sent him a link on November 11th.
Ray:
OK, I watched the movie.
I’m not quite following your argument. It seems as though your app. violates our ToS and our robots.txt file by fetching as many backlinks for a page as the end user sets it to.
Leslie:
OptiLink is not a robot and does no ‘spidering’ of google, so i do not see how, or why, it would make use of a robots.txt file. OptiLink makes no more use of google than does any other browser. I noticed, upon looking at the robots.txt file, that /search is blocked, which is a bit misleading, since that is precisely what all browsers use, OptiLink included.
Presumably, the /search entry is intended to block meta-search crawlers – a very different sort of animal than OptiLink.
At the time, Google had a search API. Why? Who knows. It was expressly not for “commercial” use. Research? I guess once a grad student always a grad student. Anyway, that might help understand this exchange since I had suggested that any issue they had with tools like OptiLink could be solved by marketing and regulating the API.
Ray:
We produce all our pages as HTML for human eyeballs to view them, and our advertisers pay us money expecting exactly that.
Leslie:
Yes, as I suggested in our initial phone contact, it is always about the money.
But. I’m wondering if you’ve noticed that a “link:” query never displays ads?
Furthermore, if that really is the argument, then the google API is the answer.
It was a long message so I summed up:
Let’s review.
1. The ToS says “automated” – OptiLink ain’t.
2. There are no ads in the “link:” query results and in any case the OptiLink operator can in no event escape having a browser window open. Sure, he doesn’t have to look at it, but he can close his eyes even with IE, putting OptiLink on the same footing with any other browser.
There’s some more really good stuff in that email, but you’re likely itching for the conclusion, right? Here it is – sortof. On November 19th, Ray Sidney wrote:
Heh. A pretty well thought-out response, I must admit. You do indeed make some good points.
I see what you mean– you’re only fetching a single page of [backlinks] results from Google for each action of the user, and you make this page completely available and viewable. I hadn’t quite grasped this. You’re right; this seems fine. I’m not going to give you a hard time.
Hurray, we win, right? No so fast. Here’s my reply to him the following day (Nov 20th):
Glad to hear that i will be escaping ‘hard time’, but it wasn’t homestly me that i was concerned about.
Over the last several months, no less than three web sites have been dropped from the google index with optilink association being the primary, if not exclusive, justification communicated to these site owners. Each of the sites was subsequently restored upon pleading with various folks at google (Matt was the point of contact in at least one case). [emphasis added]
If this ToS issue is truly resolved, then is it safe to say that people like John Heard, Terry Plank
and Robin Nobles can restore there links to optilink? Is it safe for these folks, and others, to restore their public endorsements without threat of being dropped (again)?My goal here is clearity. I want to make sure that this issue is resolved so that I can be clear with my customers and affiliates.
You see, what Ray did not know, is that my affiliates were being banned! Not all of them, but it did ultimately put one of them out of business. The “last straw” was when John Heard’s main domain, that he only ever used for email, was dialed to PR0. Strange don’t you think that this happened just days after I posted his personal endorsement to the top of my sales letter?
So I suggested that Ray talk to Matt and oh how the world changed by November 25th.
Hi, Leslie.
Upon further reflection, I’m afraid I’m going to have to do a bit of a turn-around on this.
Google produces search results of all sorts intending them to be viewed by human eyes. Any kind of programmatic/automated querying on Google consumes our server resources, and is unwelcome and not recommended.
Although OptiLink makes available for viewing the results pages of backlinks that Google produces, that’s clearly not the actual focus of what it does. We view this program as something that violates our Terms of Service, and we might pursue a variety of remedies against anyone associated with it, including (but not limited to):
– banning the use of Google services;
– permanently removing domains from our index;
– legal action.No need to write back if it’s to ask how you could engineer your software to be something we don’t object to.
Regards,
Ray
The grand arrogance of this message is amazing enough, but more astonishing than that is their complete naivety regarding even the most basic tenants of the law regulating commerce.
That’s the subject of the next episode.
Speak Your Mind