Archives for May 2017

It was June 6, 2002. My desk phone (remember those?) rings and someone named Ray Sidney from Google wants to talk with me about OptiLink. Seems it might violate their ToS.

This was only 15 days after OptiLink was announced, so I’m calling that proof that someone at Google was reading Michael Campbell’s newsletter! 🙂

I invited Ray to just download the software and try it, but he had visions of Trojan horses invading his computer and galoping throughout the Googleplex [wish I’d thought of that!] so he declined.

All I needed to do was shoot a 3 minute screen recording to show him how OptiLink worked, but did I mention this was 2002? That was hard work back then! And a 10Mb mov file doesn’t seem like much now, but you notice it on a 250Mb hard drive (yes, that’s an M, not a G!).

I finally got the new-user video recorded and sent him a link on November 11th.

Ray:
OK, I watched the movie.
I’m not quite following your argument.  It seems as though your app. violates our ToS and our robots.txt file by fetching as many backlinks for a page as the end user sets it to.

Leslie:
OptiLink is not a robot and does no ‘spidering’ of google, so i do not see how, or why, it would make use of a robots.txt file.  OptiLink makes no more use of google than does any other browser.  I noticed, upon looking at the robots.txt file, that /search is blocked, which is a bit misleading, since  that is precisely what all browsers use, OptiLink included.
Presumably, the /search entry is intended to block meta-search crawlers – a very different sort of animal than OptiLink.

At the time, Google had a search API. Why? Who knows. It was expressly not for “commercial” use. Research? I guess once a grad student always a grad student. Anyway, that might help understand this exchange since I had suggested that any issue they had with tools like OptiLink could be solved by marketing and regulating the API.

Ray:
We produce all our pages as HTML for human eyeballs to view them, and our advertisers pay us money expecting exactly that.

Leslie:
Yes, as I suggested in our initial phone contact, it is always about the money.
But. I’m wondering if you’ve noticed that a “link:” query never displays ads?

Furthermore, if that really is the argument, then the google API is the answer.

It was a long message so I summed up:

Let’s review.
1. The ToS says “automated” – OptiLink ain’t.
2. There are no ads in the “link:” query results and in any case the OptiLink operator can in no event escape having a browser window open.  Sure, he doesn’t have to look at it, but he can close his eyes even with IE, putting OptiLink on the same footing with any other browser.

There’s some more really good stuff in that email, but you’re likely itching for the conclusion, right? Here it is – sortof. On November 19th, Ray Sidney wrote:

Heh.  A pretty well thought-out response, I must admit.  You do indeed make some good points.
I see what you mean– you’re only fetching a single page of [backlinks] results from Google for each action of the user, and you make this page completely available and viewable.  I hadn’t quite grasped this.  You’re right; this seems fine.  I’m not going to give you a hard time.

Hurray, we win, right? No so fast. Here’s my reply to him the following day (Nov 20th):

Glad to hear that i will be escaping ‘hard time’, but it wasn’t homestly me that i was concerned about.

Over the last several months, no less than three web sites have been dropped from the google index with optilink association being the primary, if not exclusive, justification communicated to these site owners.  Each of the sites was subsequently restored upon pleading with various folks at google (Matt was the point of contact in at least one case). [emphasis added]

If this ToS issue is truly resolved, then is it safe to say that people like John Heard, Terry Plank
and Robin Nobles can restore there links to optilink?  Is it safe for these folks, and others, to restore their public endorsements without threat of being dropped (again)?

My goal here is clearity.  I want to make sure that this issue is resolved so that I can be clear with my customers and affiliates.

You see, what Ray did not know, is that my affiliates were being banned! Not all of them, but it did ultimately put one of them out of business. The “last straw” was when John Heard’s main domain, that he only ever used for email, was dialed to PR0. Strange don’t you think that this happened just days after I posted his personal endorsement to the top of my sales letter?

So I suggested that Ray talk to Matt and oh how the world changed by November 25th.

Hi, Leslie.

Upon further reflection, I’m afraid I’m going to have to do a bit of a turn-around on this.

Google produces search results of all sorts intending them to be viewed by human eyes. Any kind of programmatic/automated querying on Google consumes our server resources, and is unwelcome and not recommended.

Although OptiLink makes available for viewing the results pages of backlinks that Google produces, that’s clearly not the actual focus of what it does. We view this program as something that violates our Terms of Service, and we might pursue a variety of remedies against anyone associated with it, including (but not limited to):

– banning the use of Google services;
– permanently removing domains from our index;
– legal action.

No need to write back if it’s to ask how you could engineer your software to be something we don’t object to.

Regards,
Ray

The grand arrogance of this message is amazing enough, but more astonishing than that is their complete naivety regarding even the most basic tenants of the law regulating commerce.

That’s the subject of the next episode.

It’s hard to explain the stress of that time, but as I write this account, I can feel it all over again. I knew that what I had found would revolutionize SEO, but to profit from it, I had to be first!

But I had no idea how much time I had. Was someone a week away from releasing the same tool? Was someone about finished with a training course or ebook exposing the secret? There was no way to know and caution would only make the matter worse.

I coded at a breakneck pace, but that was only half of it – and the easy half at that! I had to verify that what Google wrote was true and that I could create actionable advice from my software. Plus, I didn’t have a website, a cart, an affiliate system, or even a merchant account. Truly, I did not sleep well for almost 6 months.

And remember, this was “the old days.” I coded webpages in the Linux text editor. I built my (primitive) shopping cart in Perl. I built an entire affiliate tracking and reporting system from scratch, also in Perl, in about 48 hours because that was faster than testing the commercial packages. Oh, and gosh, maybe I should have a sales letter?

As I got the code to work and proved what it could do, Michael and I got more excited – and I got even more scared: what if I was even just one day late? Michael was mailing his newsletter every other Thursday, so we decided to announce OptiLink on May 22, 2002.

That was an agressive schedule. I did not even get my merchant account approved until the morning of May 21st! It took me a couple hours to setup Authorize.net and then Michael ran a test order – which failed!

Yes, I was naive. I had all the recommended fraud settings in place, which blocked ordering even from Canada. Unbelievable. In fact, in the first 48 hours after launch, I removed every single anti-fraud setting, one-by-one, as customers from around the world tried to purchase.

By early Thursday morning, well, actually I guess you’d call 2 am part of Wednesday night, affiliate tracking through Michael’s link was tested, the download page working, and the OptiLink installer (one more thing!) was finally working as well.

2 PM Pacific. Michael asked “are you ready?” I lied… I said yes! 🙂 He hit send. My world changed.

As I said, this was the old days. I’d didn’t know Jeff Walker and PLF back then, so there was no “pre-launch sequence”. Michael simply wrote an article along with his blessing and a link. That was it. No warm up, no videos, no sequence even. But it worked.

Now, we did not “melt the server”, that silly expression was not invented yet, but early sales proved beyond any doubt that we really had something. I got drunk and bedded down for some much needed sleep while orders poured in overnight.

I learned a lot from this. Here are two tidbits you might need someday as well.

Turns out, my fears were unfounded. Not only was I first to market, I was the only one that even noticed the secret of OptiLink hidden in plain view. I did not really need to race against the clock. The only competition for OptiLink would occur more than 2 years later from a customer that simply copied both the tool and the sales message.

Lesson: Innovations happen when a unique combination of person, place, and time come together. In 2002, I was a rarity: a programmer, turned marketer, trying to do SEO. That’s no longer rare, but in your niche, there is likely – right now – some undeveloped innovation that just needs the same fresh viewpoint that I broght to SEO in 2002. You could be that person.

I was moving so fast, that I didn’t even try to “prioritize” or “plan” – I just worked. It all had to get done so it almost didn’t matter what I worked on or in what order. Yes, it was chaos but I build a million dollar business in six months from scratch.

Lesson: Fast really is the new big. Just create value. Organizing your work is not value. Only work can create value. Sure, if you’re building a rocket, you need some process, but almost everyone I know is too concerned with planning and process and not fast enough at just getting shit done. I did not plan OptiLink, I just did it.

And chaos will continue. The launch of OptiLink was not the end of that. Success breeds a kind of chaos of it’s own and that continued all summer long, starting with…

a phone call from Google.

I’ll tell you about that tomorrow.

In our last episode, John Heard told me to go read the original Google paper. I did. Over and over. And “the secret” (to ranking, not the movie!) was totally plain and obvious, well, at least to me.

These guys are programmers.

So am I. We literally speak in code, and what they said in their technical description of Google told me precisely what to build! [Thanks guys!]

In fact, OptiLink is not all I discovered that day, but we’ll get to that later. For now, let me show you by example the thought process that launched my career in SEO and has sustained me ever since.

Programming is a practice of linguistic precision. Words must mean very specific things to us because that is what they mean to the very fast, but equally stupid, machines that we code.

In almost all cases, people view us as “pedantic” – I mean what really is the difference if you say domain, URL, link, anchor, link text, back-link, whatever … humans will know what you mean from context, right?

Not so the machine.

So now let’s read that paper those grad students wrote with a programmer’s precision.

The phrase “anchor text” is used 13 times and “link text” mentioned 5. Here are some of the best bits from my original frantic markup:

• “This idea of propagating anchor text to the page it refers to…”
• “…anchor text can help provide better quality results.”
• “Aside from PageRank and the use of anchor text, Google has several other features.” [other means minor IMHO]
• “There are two types of hits … Fancy hits include … anchor text … Plain hits include everything else”

I could go on, but this snippet is the real crusher:

Google considers each hit to be one of several different types (title, anchor, URL, plain text large font, plain text small font, …), each of which has its own type-weight. The type-weights make up a vector indexed by type. Google counts the number of hits of each type in the hit list. Then every count is converted into a count-weight. Count-weights increase linearly with counts at first but quickly taper off so that more than a certain count will not help. We take the dot product of the vector of count-weights with the vector of type-weights to compute an IR score for the document. Finally, the IR score is combined with PageRank to give a final rank to the document.

This was their entire algorithm. If you don’t understand all of it, that’s okay, but aren’t the key factors absolutely clear? It was to me, and it fueled everything I did in SEO for about 7 years.

More than that, they gave a real example of how to rank with ONLY link text. It’s in section 5:

Notice that there is no title for the first result. This is because it was not crawled. Instead, Google relied on anchor text to determine this was a good answer to the query. Similarly, the fifth result is an email address which, of course, is not crawlable. It is also a result of anchor text.

 OMG! I can rank with anchor text alone!

If was right there – plain as day – but no one was teach this and there were no tools to analyze anchor text.

I couldn’t believe it. What a huge opportunity. But I was scared. What if someone else got to market first? How fast could I get a product to market? And what if I got it all done and it didn’t work the way the guys said it does? It was a risk, but I had to try – fast!

From start to finish it took me a brutal 6 months. Tomorrow I’ll tell you the lessons I learned in the process and what I’d change if I did it again.

A thing I was calling OptiText.  Never heard of it?  Not surprising: I never released it because of this weird thing called “ethics”.

During late 2000 and all of 2001 I was getting serious about growing organic traffic to some sites I was running (that’s another story).  The “gold standard” of SEO analysis of the day was a little tool named GRKda.

It measured “keyword density” which was all the rage back then and “proven” over-and-over to be the best way to rank a page.

Meanwhile, I was regularly talking with Michael Campbell and he suggested that SEO Software was a hot market that I should pursue, so I started building OptiText as the GRKda-killer – since it was really clunky and tedious to use.

About six months later and a bunch of review with Michael, I had a damn fine prototype.  There was just one little problem…

It didn’t work!  Well, I mean, the software did work.  It measured keyword density.  It even did it in real-time as you typed in a WYSIWYG editor (more-or-less … it was 15 years ago!), and even ignored your customizable stop-word list.

But it didn’t “work”.  That is, keyword density in no way whatsoever correlated with ranking.  I compared my results with all the other products available and not one of them gave actionable information.  Shock and horror – they were all lying their asses off.  And they still are!!!  GRKda is – unbelievable – still for sale today.

So I had a real ethics issue! I had a product nearly ready for market.  It did what everyone expected, plus just a bit better and more friendly.  What to do?  Tell the same lies?  Somehow sidestep the question?  Figure out what it was good for other than ranking?  Micheal suggested we send it to a few Internet marketers on his list and have them make suggestions.

Brilliant!  And if not for that, I would never have heard THE ONE SENTENCE that caused me to create OptiLink…

That is the subject of tomorrow’s gripping episode.  🙂