I didn’t need Google’s approval of OptiLink. They never were able to detect it, or even materially impact my sales, but banning my affiliates? That’s not okay.

During this same time, Bob Massa, the owner of Search King, was suing Google over (essentially) doing “manual overrides” to PageRank, despite it being widely described and promoted as an “algorithm” not subject to human whim. In speaking with Bob, I do think he was arguing the wrong issues, but I also think that he got the attention of some “adult supervision” at Google.

That’s not even a metaphor! I had just entered my 46th year of life and my 16th year as a full time business owner. That made me about twice as old as Google’s founders and about 10 times as experienced in business. Bob had even greater creds than mine.

Calling Google “young and reckless” was in no way an exageration.

They were so naive and unsupervised as to actually admit to actions that were illegal acts. Here’s an excerpt from a real email from Matt Cutts to one of my affiliates:

I can confirm that ***.com and ***.com were removed from our index pending a more full review. Among other things, someone reported your endorsement of a program (Optilink) that clearly violates our Terms of Service; please see http://www.google.com/terms_of_service.html. The use of any programs to access Google without permission is unwelcome.

Oh wait, here’s one more:

Your quote on that page verifies that you’ve violated our Terms of Service (link) by sending programmatic queries to Google. … If you are willing not to use programs that violate our terms of service, we’d be happy to re-instate your domain.

Huh? Which domains? Every one that I own? And how are my domains connected to my use of OptiLink? I don’t use OptiLink from my domain, I use it from my house. The Google ToS governs the use of Google as a searcher. My domains are another matter entirely.

If you want to do the legal deep-dive on this, start with “tying arrangement” and “abuse of market power”. You might just find FTC’s case against Microsoft presented as an example, a lawsuit that even a couple of young computer science students must have heard of.

So should I call the FTC and get a case started? Maybe, but let’s try the court of public opinion first. That turned out to be really effective and cost nothing.

This part of the story starts November 25, 2002. By now, I had already gotten the “green light” from Ray Sidney, followed by his abrupt about face after I suggested he talk to Matt.

Back in the day, there was this poster in WebMasterWorld named “Google Guy”. Thanks to Jim Boykin, the forum trace from late 2002 is still preserved for everyone to enjoy. I’ll hit the highlights, but here’s the link to the whole conversation: https://www.webmasterworld.com/forum3/7164.htm

Of course, I can’t say that it was Matt Cutts that black-balled OptiLink. I also can not say that Matt Cutts was Google Guy. Nor do I have first hand knowledge that Matt Cutts admitted to doing the bannings of my affiliates. You’ll have to gauge the truth of those rumors and third-party allegations for yourself as you read what follows.

In response to a pre-sales question on WMW, a piece of my response included:

Google is currently “conflicted” concerning OptiLink, having taken a stand, retracted, and gone back again, but this is only an issue for people _promoting_ the product and will soon be resolved one way or another anyway.

I guess that was too much for GG because he posted this reply just 2 hours later:

Someone said that “Google is currently “conflicted” concerning OptiLink…”

I think I can solve that conflict. Our Terms of Service do not permit programmatic queries without permission. Someone who uses OptiLink may have their IP addresses banned or their domains removed from our index. It’s that simple.

This stance should be pretty clear from our webmaster guidelines, but I’ll check with folks to make sure that the next time we update our webmaster guidelines, we make it more clear.

Added: just to clarify on one point, Windrose states that “using OptiLink is completely
invisible” to search engines. As always, you may want to take such claims from the creator of a program with a grain of salt. Other people have claimed their programs were completely invisible before, and such claims have also been proven false before.

In all seriousness: please do not use programs to send any automatic queries (of any kind) to Google. Such programs take server resources away that should be used to serve queries to actual searchers.

Of course, that’s just the same B.S. argument I’d already rebutted with Ray Sidney only days earlier, but it bore repeating in public where everyone could see the emperor’s invisible clothing for themselves, so…

Dearest GoogleGuy,

Google’s Terms of Service are at best ambigous. ALL queries are “automated” to the extent that programs are the only means to make queries. IE is an “automated” query. If anyone doubts this, they should try doing one by hand. Or if you can’t figure out how to get your hand stuck into an RJ11 jack, then use telnet. Mechanically that’s less painful, of course telnet is a program (read: automated), but in trying to query Google with nothing but telnet you will soon find out just how much today’s browsers “automate”.

What does Google imagine separates a standard browser, which presumably you would not classify as automated, from programs that you presume the ToS bans? Programs that run unattended or run from server farms might arguably qualify, but what about programs that will not operate w/o the end user pressing the start button from the comfort of his own desk chair? OptiLink does nothing more nor less than fetch the same result page IE does when doing a link query and provides a very similar graphic to the user. This is in some way more automated than IE? A strange notion indeed.

But no more strange than Google’s notion of what constitutes appropriate countermeasures, which in most legal contemplation would be instantly deemed both arbitrary and vindictive. Blocking an IP due to server load, a variation of denial of service, is reasonable and justifyable. But banning an unrelated domain? Or what about banning a site for linking to a banned site? Or banning a domain unrelated to the “offensive software” just because the whois data shows that it is owned by someone having made a public endorsement for said product? What about doing this three times in one month? I suspect that you know first hand that these are more than just theoretical possibilities. Who knows, we might find out through discovery. [emphasis added]

 

We go back and forth a couple times, with GG repeating himself and me being even more pedantic, but this tidbit from page 2 of the thread is worth repeating:

And more than that, the types and extent of counter-measures that can legally be used by a company in response to a violation of a ToS, whether well-formed or not, has also been repeatedly addressed. [emphasis added]

someone suggested here that a violation of the ToS would likely (and by implication correctly) result in banning the ip or complaining to the isp. But wouldn’t it be better to just look up the person in the whois database and ban one or more of their websites? if you said yes, there might actually be a job for you in the search engine industry.

That bit at the end was me having some fun exercising my sarcasm gene, but the bolded stuff is the meat of the issue.

But did any of this really matter?

Hard to tell. John’s site was restored to PR5 a few days before these forum posts and the other banned sites were still clawing their way back into results after taking down their public endorsements of OptiLink.

Bob Massa’s suit went nowhere and I never made an FTC case of this (because I was technically not even damaged!), but there was never a banning of any of my affiliates ever again, for any of my products, so that’s a win, plus John’s endorsement remained on my site for years to come.

As my brand grew, I had close to 100 affiliates and not one of them saw any retribution from Google – even in those cases when the domain name itself had “optilink” in the name!

All I wanted was to be left alone, and they did that. I didn’t expect them to mail an apology. It was enough for me that they stopped being petty and vindictive and instead got back to work improving what was then and still is the best search engine available.

Epilog

Sure, Google was young and stupid back then, but so was I once, so I can’t really begrudge them that.

Ray Sidney is long since gone, and Matt Cutts is “functionally gone”, but is Google really “older and wiser”? Hard to say. They did learn one thing: transparency can trip you up.  So maybe they’re still just as wild and crazy behind their near-absolute opacity.

OptiLink lived on to a ripe old age in software years and died a nearly anonymous death, starved by lack of linking-data to process. While it remained a game-changer for some years, most of that story is boring and routine. It was really the first 6 months of its life that I’ve chronicled in these posts that were exciting and entertaining. I hope you enjoyed the traipse down memory lane as much as I did recounting it.

Have an Optimal Day

It was June 6, 2002. My desk phone (remember those?) rings and someone named Ray Sidney from Google wants to talk with me about OptiLink. Seems it might violate their ToS.

This was only 15 days after OptiLink was announced, so I’m calling that proof that someone at Google was reading Michael Campbell’s newsletter! 🙂

I invited Ray to just download the software and try it, but he had visions of Trojan horses invading his computer and galoping throughout the Googleplex [wish I’d thought of that!] so he declined.

All I needed to do was shoot a 3 minute screen recording to show him how OptiLink worked, but did I mention this was 2002? That was hard work back then! And a 10Mb mov file doesn’t seem like much now, but you notice it on a 250Mb hard drive (yes, that’s an M, not a G!).

I finally got the new-user video recorded and sent him a link on November 11th.

Ray:
OK, I watched the movie.
I’m not quite following your argument.  It seems as though your app. violates our ToS and our robots.txt file by fetching as many backlinks for a page as the end user sets it to.

Leslie:
OptiLink is not a robot and does no ‘spidering’ of google, so i do not see how, or why, it would make use of a robots.txt file.  OptiLink makes no more use of google than does any other browser.  I noticed, upon looking at the robots.txt file, that /search is blocked, which is a bit misleading, since  that is precisely what all browsers use, OptiLink included.
Presumably, the /search entry is intended to block meta-search crawlers – a very different sort of animal than OptiLink.

At the time, Google had a search API. Why? Who knows. It was expressly not for “commercial” use. Research? I guess once a grad student always a grad student. Anyway, that might help understand this exchange since I had suggested that any issue they had with tools like OptiLink could be solved by marketing and regulating the API.

Ray:
We produce all our pages as HTML for human eyeballs to view them, and our advertisers pay us money expecting exactly that.

Leslie:
Yes, as I suggested in our initial phone contact, it is always about the money.
But. I’m wondering if you’ve noticed that a “link:” query never displays ads?

Furthermore, if that really is the argument, then the google API is the answer.

It was a long message so I summed up:

Let’s review.
1. The ToS says “automated” – OptiLink ain’t.
2. There are no ads in the “link:” query results and in any case the OptiLink operator can in no event escape having a browser window open.  Sure, he doesn’t have to look at it, but he can close his eyes even with IE, putting OptiLink on the same footing with any other browser.

There’s some more really good stuff in that email, but you’re likely itching for the conclusion, right? Here it is – sortof. On November 19th, Ray Sidney wrote:

Heh.  A pretty well thought-out response, I must admit.  You do indeed make some good points.
I see what you mean– you’re only fetching a single page of [backlinks] results from Google for each action of the user, and you make this page completely available and viewable.  I hadn’t quite grasped this.  You’re right; this seems fine.  I’m not going to give you a hard time.

Hurray, we win, right? No so fast. Here’s my reply to him the following day (Nov 20th):

Glad to hear that i will be escaping ‘hard time’, but it wasn’t homestly me that i was concerned about.

Over the last several months, no less than three web sites have been dropped from the google index with optilink association being the primary, if not exclusive, justification communicated to these site owners.  Each of the sites was subsequently restored upon pleading with various folks at google (Matt was the point of contact in at least one case). [emphasis added]

If this ToS issue is truly resolved, then is it safe to say that people like John Heard, Terry Plank
and Robin Nobles can restore there links to optilink?  Is it safe for these folks, and others, to restore their public endorsements without threat of being dropped (again)?

My goal here is clearity.  I want to make sure that this issue is resolved so that I can be clear with my customers and affiliates.

You see, what Ray did not know, is that my affiliates were being banned! Not all of them, but it did ultimately put one of them out of business. The “last straw” was when John Heard’s main domain, that he only ever used for email, was dialed to PR0. Strange don’t you think that this happened just days after I posted his personal endorsement to the top of my sales letter?

So I suggested that Ray talk to Matt and oh how the world changed by November 25th.

Hi, Leslie.

Upon further reflection, I’m afraid I’m going to have to do a bit of a turn-around on this.

Google produces search results of all sorts intending them to be viewed by human eyes. Any kind of programmatic/automated querying on Google consumes our server resources, and is unwelcome and not recommended.

Although OptiLink makes available for viewing the results pages of backlinks that Google produces, that’s clearly not the actual focus of what it does. We view this program as something that violates our Terms of Service, and we might pursue a variety of remedies against anyone associated with it, including (but not limited to):

– banning the use of Google services;
– permanently removing domains from our index;
– legal action.

No need to write back if it’s to ask how you could engineer your software to be something we don’t object to.

Regards,
Ray

The grand arrogance of this message is amazing enough, but more astonishing than that is their complete naivety regarding even the most basic tenants of the law regulating commerce.

That’s the subject of the next episode.

It’s hard to explain the stress of that time, but as I write this account, I can feel it all over again. I knew that what I had found would revolutionize SEO, but to profit from it, I had to be first!

But I had no idea how much time I had. Was someone a week away from releasing the same tool? Was someone about finished with a training course or ebook exposing the secret? There was no way to know and caution would only make the matter worse.

I coded at a breakneck pace, but that was only half of it – and the easy half at that! I had to verify that what Google wrote was true and that I could create actionable advice from my software. Plus, I didn’t have a website, a cart, an affiliate system, or even a merchant account. Truly, I did not sleep well for almost 6 months.

And remember, this was “the old days.” I coded webpages in the Linux text editor. I built my (primitive) shopping cart in Perl. I built an entire affiliate tracking and reporting system from scratch, also in Perl, in about 48 hours because that was faster than testing the commercial packages. Oh, and gosh, maybe I should have a sales letter?

As I got the code to work and proved what it could do, Michael and I got more excited – and I got even more scared: what if I was even just one day late? Michael was mailing his newsletter every other Thursday, so we decided to announce OptiLink on May 22, 2002.

That was an agressive schedule. I did not even get my merchant account approved until the morning of May 21st! It took me a couple hours to setup Authorize.net and then Michael ran a test order – which failed!

Yes, I was naive. I had all the recommended fraud settings in place, which blocked ordering even from Canada. Unbelievable. In fact, in the first 48 hours after launch, I removed every single anti-fraud setting, one-by-one, as customers from around the world tried to purchase.

By early Thursday morning, well, actually I guess you’d call 2 am part of Wednesday night, affiliate tracking through Michael’s link was tested, the download page working, and the OptiLink installer (one more thing!) was finally working as well.

2 PM Pacific. Michael asked “are you ready?” I lied… I said yes! 🙂 He hit send. My world changed.

As I said, this was the old days. I’d didn’t know Jeff Walker and PLF back then, so there was no “pre-launch sequence”. Michael simply wrote an article along with his blessing and a link. That was it. No warm up, no videos, no sequence even. But it worked.

Now, we did not “melt the server”, that silly expression was not invented yet, but early sales proved beyond any doubt that we really had something. I got drunk and bedded down for some much needed sleep while orders poured in overnight.

I learned a lot from this. Here are two tidbits you might need someday as well.

Turns out, my fears were unfounded. Not only was I first to market, I was the only one that even noticed the secret of OptiLink hidden in plain view. I did not really need to race against the clock. The only competition for OptiLink would occur more than 2 years later from a customer that simply copied both the tool and the sales message.

Lesson: Innovations happen when a unique combination of person, place, and time come together. In 2002, I was a rarity: a programmer, turned marketer, trying to do SEO. That’s no longer rare, but in your niche, there is likely – right now – some undeveloped innovation that just needs the same fresh viewpoint that I broght to SEO in 2002. You could be that person.

I was moving so fast, that I didn’t even try to “prioritize” or “plan” – I just worked. It all had to get done so it almost didn’t matter what I worked on or in what order. Yes, it was chaos but I build a million dollar business in six months from scratch.

Lesson: Fast really is the new big. Just create value. Organizing your work is not value. Only work can create value. Sure, if you’re building a rocket, you need some process, but almost everyone I know is too concerned with planning and process and not fast enough at just getting shit done. I did not plan OptiLink, I just did it.

And chaos will continue. The launch of OptiLink was not the end of that. Success breeds a kind of chaos of it’s own and that continued all summer long, starting with…

a phone call from Google.

I’ll tell you about that tomorrow.

In our last episode, John Heard told me to go read the original Google paper. I did. Over and over. And “the secret” (to ranking, not the movie!) was totally plain and obvious, well, at least to me.

These guys are programmers.

So am I. We literally speak in code, and what they said in their technical description of Google told me precisely what to build! [Thanks guys!]

In fact, OptiLink is not all I discovered that day, but we’ll get to that later. For now, let me show you by example the thought process that launched my career in SEO and has sustained me ever since.

Programming is a practice of linguistic precision. Words must mean very specific things to us because that is what they mean to the very fast, but equally stupid, machines that we code.

In almost all cases, people view us as “pedantic” – I mean what really is the difference if you say domain, URL, link, anchor, link text, back-link, whatever … humans will know what you mean from context, right?

Not so the machine.

So now let’s read that paper those grad students wrote with a programmer’s precision.

The phrase “anchor text” is used 13 times and “link text” mentioned 5. Here are some of the best bits from my original frantic markup:

• “This idea of propagating anchor text to the page it refers to…”
• “…anchor text can help provide better quality results.”
• “Aside from PageRank and the use of anchor text, Google has several other features.” [other means minor IMHO]
• “There are two types of hits … Fancy hits include … anchor text … Plain hits include everything else”

I could go on, but this snippet is the real crusher:

Google considers each hit to be one of several different types (title, anchor, URL, plain text large font, plain text small font, …), each of which has its own type-weight. The type-weights make up a vector indexed by type. Google counts the number of hits of each type in the hit list. Then every count is converted into a count-weight. Count-weights increase linearly with counts at first but quickly taper off so that more than a certain count will not help. We take the dot product of the vector of count-weights with the vector of type-weights to compute an IR score for the document. Finally, the IR score is combined with PageRank to give a final rank to the document.

This was their entire algorithm. If you don’t understand all of it, that’s okay, but aren’t the key factors absolutely clear? It was to me, and it fueled everything I did in SEO for about 7 years.

More than that, they gave a real example of how to rank with ONLY link text. It’s in section 5:

Notice that there is no title for the first result. This is because it was not crawled. Instead, Google relied on anchor text to determine this was a good answer to the query. Similarly, the fifth result is an email address which, of course, is not crawlable. It is also a result of anchor text.

 OMG! I can rank with anchor text alone!

If was right there – plain as day – but no one was teach this and there were no tools to analyze anchor text.

I couldn’t believe it. What a huge opportunity. But I was scared. What if someone else got to market first? How fast could I get a product to market? And what if I got it all done and it didn’t work the way the guys said it does? It was a risk, but I had to try – fast!

From start to finish it took me a brutal 6 months. Tomorrow I’ll tell you the lessons I learned in the process and what I’d change if I did it again.

A thing I was calling OptiText.  Never heard of it?  Not surprising: I never released it because of this weird thing called “ethics”.

During late 2000 and all of 2001 I was getting serious about growing organic traffic to some sites I was running (that’s another story).  The “gold standard” of SEO analysis of the day was a little tool named GRKda.

It measured “keyword density” which was all the rage back then and “proven” over-and-over to be the best way to rank a page.

Meanwhile, I was regularly talking with Michael Campbell and he suggested that SEO Software was a hot market that I should pursue, so I started building OptiText as the GRKda-killer – since it was really clunky and tedious to use.

About six months later and a bunch of review with Michael, I had a damn fine prototype.  There was just one little problem…

It didn’t work!  Well, I mean, the software did work.  It measured keyword density.  It even did it in real-time as you typed in a WYSIWYG editor (more-or-less … it was 15 years ago!), and even ignored your customizable stop-word list.

But it didn’t “work”.  That is, keyword density in no way whatsoever correlated with ranking.  I compared my results with all the other products available and not one of them gave actionable information.  Shock and horror – they were all lying their asses off.  And they still are!!!  GRKda is – unbelievable – still for sale today.

So I had a real ethics issue! I had a product nearly ready for market.  It did what everyone expected, plus just a bit better and more friendly.  What to do?  Tell the same lies?  Somehow sidestep the question?  Figure out what it was good for other than ranking?  Micheal suggested we send it to a few Internet marketers on his list and have them make suggestions.

Brilliant!  And if not for that, I would never have heard THE ONE SENTENCE that caused me to create OptiLink…

That is the subject of tomorrow’s gripping episode.  🙂