Technology Integration

I’m sure you’ve got a picture in your head by now that there’s an annoying manual process here. Well, the technology piece gets worse. Today, most technologies that companies will consider “competitive advantages” are either optimization/prediction algorithms, contextual engines or behavioral datasets. Say we have this company called Google that has a great technology that matches ads to page content. The payout on each click depends greatly on what the ad is for. Pages relevant to autos or medication will net far higher payments than pages about the local pizza joint. So how is the publisher to decide when to show ‘Adsense’ over the $1.00 CPM deal he just signed? There’s absolutely no way for the Publisher to know the effective CPM of the adsense ad before he shows it.

Currently there are two ways that publishers manage this problem. The first is to simply accept an inefficiency in pricing. He may prioritize Google Adsense at $0.95 CPM across all pages because on average that’s what he can expect to receive at the end of the month. The other option would be to setup tens or hundreds of different tags targeted to different types of pages and then setup different prices for each. Again, this would require setting up tens or hundreds of categories in Adsense and then trafficking tens or hundreds of different tags into the Publisher’s adserver. Not really the best solution.

What about a behavioral advertiser that wants to buy New York Times traffic because he thinks he has data on some of the users. Since the behavioral data is stored in his cookie (see my post here about behavioral advertisers) there is no way for the Publisher to know which of his users will be valuable to the advertiser! How is he supposed to price this? This one isn’t so easy. One way is for the advertiser to simply buy a flat rate for all New York Times users and then simply count the users for which he doesn’t have data as a loss. The other would be some sort of rudimentary integration where the advertiser drops a pixel for the Publisher’s cookie domain for his users. Again, not ideal, not simple and INEFFICIENT!

High Latency/Slow Adserving

Each call to a different adserver costs time. The more hops that a user’s browser has to go to to receive the actual ad, the more likely that he is to click-off to another page before he actually sees it. Try it, go to myspace.com and click through on a few links. How often did the actual ad finish loading? The higher the number of systems involved in an ad call, the higher the difference between how many impression the Advertiser and Publisher’s systems count. Last I heard, 5-10% of impressions are lost for each additional adserver that is added to an ad-chain.

The Exchange Model

Fundamentally I believe people don’t quite understand why Ad-Exchanges are key is because they don’t realize that an ad-exchange is simply a glorified adserver. Really… nothing special, just one centralized system instead of three or more. Take a look a the following diagram:

Notice a difference? In this case, there’s only ONE request to ONE system, the ad-exchange. The exchange is the ecosystem through which advertisers, publishers and networks all manage their businesses. Some may integrate their contextual technologies, some may simply use the system to set prices. But it’s ONE ADSERVER! Lets revisit the three main challenges we listed before.

Pricing/Operational Inefficiency — On the Exchange

Now that the advertiser and the publisher are both on the same system the whole world changes. There is no longer a need to transfer “tags” back and forth, as all the data is in the same system. Pricing is also no longer an issue as the advertiser has integrated his technology solution with the exchange and can now bid a different price on each ad impression depending on the page the user is visiting.

Technology Integration

Although there is still a significant amount of work to be done to integrate an Advertiser’s technology solution with the exchange, it’s worth it since it’s work that only needs to be done once. Once done, this technology will work across any and all publishers. Imagine this, the New York Times starts using Googleclick as their adserver, which is, of course, fully integrated with adsense. The NYT no longer has to worry about inefficient pricing for Adsense as the technology will be integrated with exchange. On every ad-call, Adsense can check the page content and place and appropriate bid according to the types of ads that it would place. Genius right?

Now what about the Behavioral network? Of course it will integrate it’s data with the exchange, and again, on every ad call it can enter bids on the users that it has data on and even price differently based on the types of data. Of course it will be slightly more difficult to integrate technology with the exchange adserver as by nature it’s more complex than a basic adserver, but this doesn’t really matter. Since the Advertiser only has to integrate his technology once, with one adserver it’s worth the effort.

High Latency/Slow Adserving

This one should be pretty obvious. Since there is only one request, ads serve faster and fewer impressions are lost.

Final Thoughts

I realize this post is long, but I think it’s important that people realize the true value that Exchanges bring to the market. It’ll be fascinating to see how the market changes as Google and Yahoo each attempt to take control of the billions of dollars of advertising that flow through the internet every day. This new model will truly change the online advertising world for the better, except perhaps for those ad-networks out that there purely benefit from the pricing and operational inefficiencies that exist in todays world.

Stay tuned for more thoughts on the potential acquisition of 24/7 by Microsoft, securing an exchange, ‘broker networks’!

This will make things interesting for my employer Right Media — is this the beginning of the next era of online advertising?

-Mike

You see, there’s no need for more marketplaces, there are plenty of them — Adsense, Ad.com, Tribal Fusion, and any other ‘ad-network’ that you can think of. The problem with each of them is that they are ‘closed’ marketplaces. Not just from a ‘business’ perspective, but from a technology perspective as well. Sure, anybody with credit and a legitimate offer can buy from any ad-network, but lets say you want to buy based on your own data. It’s simply not technically possible to do that today. Well, sorry, it is technically possible, just a royal pain in the a$$ if you do.

Lets say you’re a marketer and you have some information about a set of online visitors that is extremely valueable. E.g., lets say you know for a fact that they want to buy a car because they took part in one of your promotions. Now, first off, lets just assume that every network has behavioral capabilities (which they don’t). If you wanted to buy your users when they hit inventory that’s available on one of the many marketplaces (networks) out there, you would have to pass that data into your users’s cookie for each marketplace’s cookie. You see, when you come to this site, you get a cookie from mikeonads.com. I can write whatever I want into that cookie. I can’t see any of your others cookies nor can anybody see my cookie when you go to other sites. So, how do you get data into each Marketplace’s cookie? Sadly, this is not something you can “push out” to the user, you can only set or change cookie information when a user comes to you. E.g., if after-the-fact you decide you want to work with Marketplace D when you’re already working with A,B & C, then the only way in which you can tell put your data into Marketplace D’s cookie is when the user comes BACK to your site!

So how do you actually set information in another person’s domain if you can only access information in your own cookie space? Rather simple actually, anywhere on your page you include a link to a tiny 1×1 gif image to the 3rd parties server with some information attached to it. For example, you could put:

http://ad.someadserver.com/setbehavior?site=marketer1&behavior=likescars

Someadserver.com would then interpret that as “marketer1″ just told me that this user “likescars”, let me store that in my cookie. Can you imagine putting 20 pixels on your promotion page just to gather data? Well, that sounds like a pain doesn’t it? Well it is — and hence, most people don’t do it. So, you say, I hear there are these new ‘exchanges’ out there (e.g. Right Media has one, DoubleClick is building one, and several others are trying). Well, what really defines an exchange? It’s a platform that lets buyers, sellers and brokers all work together using the same system. And what is the only way that will work in online advertising — a single cookie domain.

If all the mini-marketplaces out there worked on the same platform, then our marketer earlier wouldn’t have much of an issue reaching his audience. He would simply put his data into the exchange cookie and then buy from whichever sellers on the exchange have ads to show to his users. It’s not just behavioral targeting that benefits from an exchange, many other factors as well.

Lets say a second marketer wants to reach as many unique users as possible with his new ad campaign. So, he goes out to all the major ad networks and places a buy with the top 10 networks and portals and informs each of them that they are only to show each user his ads once and only once. Well, ignoring the fact that some people clear cookies and hence they lose the ‘frequency cookie’, how many different ad networks serve you ads on a given day? If you go to one site, they might be working with Ad.com, another with Tribal Fusion, etc. etc.. Well, if this marketer is paying big bucks, each network will choose to show you this ad, but only once of course! The problem is, none of these mini marketplaces knows about the other, so nobody will know if you first saw the ad on the NY-Times from Ad.com when you show up on Yahoo. The end result, you might actually end up seeing the same ad many times! Guess what — a unified cookie solves that problem!

So why the title “Battle over the Cookie”? Well, people are starting to catch on that if they own the cookie, they own the market. Think about it, if one exchange succeeds in capturing a massive percentage of the market, the barriers to entry will be practically insurmountable. What value is your exchange if you can’t provide services like global frequency caps, cross publisher behavioral targeting, etc. etc.

So who’s fighting? Well, my employer (Right Media) for one. Doubleclick is said to be launching a marketplace, AdECN is another. I’d expect others to start soon — as long as someone hasn’t won, there’s still a chance.

Doesn’t it sound scary that an entire industry is so dependent on a little piece of data that many users hate? How many spyware-detection programs flag cookies as ‘desktop software’? Well, in case it hasn’t gotten to you, it’s terrifying and the industry is scared. If Congress were to pass a law that said 3rd party cookies are illegal, what would we do? There are some interesting technology solutions out there that claim they can identify you by your computer ‘fingerprint’ that might be able to help, but in the near-term, the industry would be screwed.

So kids… hate online advertising? Clear your cookies. Like the websites you vist? Keep ‘em =).

1. Branded Display — e.g. Ad.com, Niche-networks
2. Contextual — e.g. Google Adsense, Yahoo, Quigo
3. Behavioral — e.g. Tribal Fusion, Tacoda
4. Performance Display — e.g. Fastclick/Valueclick, CPA Empire

Before I delve into these into more detail, everyone has to understand something: advertising is all about reaching relevant eyeballs. Whatever the method used, the end goal is the same — if I’m selling sexy shoes, I want people to buy sexy shoes. I can go about this a couple ways but at the end of the day, I’m going to compare the amount of money I spent on advertising and the amount of profit I made from the sexy shoes that I sold. The smaller and the easier it is to purchase my good, the more likely I am going to be focused on ROI & performance. If I’m Netflix selling $4.99 subscriptions, it’s easy to track who signed up from where as I will have tens of thousands of people signing up per month. On the other hand, it’s difficult for Ford to track ad campaign performance… it’s hard to prove you bought a new Ford F150 because you saw an ad for it on automotive.com.

Ok… so eyeballs it is… lets look at the four methods above now:

Brand: If your site is good enough to be on the brand wagon you’re in luck. Branded advertisers have high standards on the inventory they will run on. The demand that it is relevant to their campaigns (e.g. Ford will want to run only on Auto sites) and they are willing to pay a premium to get there. Campaigns are generally CPM with little or no concern for performance. The problem here is that unless a site’s users are extremely desireable it’s hard for a site with low volume to get onto a branded site list. Ford wants to sell cars — to do this they want to find car buyers and associate the ‘Ford’ brand with their target audience. They might run campaigns for a new sexy sportscar on gaming sites and ads for more fuel efficient sedans on the New York Times travel section (who knows?).

Contextual: Thanks to Google this has become the big mama of online advertising. Contextual engines scrape your pages content and choose advertisers that match the content. Got a site about shoes, expect to see shoe advertisements. Now, going back to the eyeball idea, if I’m selling shoes, I want people interested in shoes. If you have a site that reviews the latest and sexiest shoes, then I will want my ad on there. On the other hand, if you have a personal blog and are complaining about how the shoe salesman smelled bad at the mall today, I probably don’t want my ad on your page. Contextual rocks on relevant page content and sucks on most Web 2.0 sites. When was the last time you saw a relevant text ad on Myspace.com?

Behavioral: Unlike Brand and Contextual, this is the first type of ad where we don’t care much at all about the page-content. We have an idea of what the users eyeballs are like and want to show him ads that he is going to enjoy the most. If you’re not famiiar with behavioral networks, read my post here. The biggest challenge here is for a site to get INTO the behavioral network. Even though for the purposes of serving one ad they don’t care what your site content is like, they will most surely want to use your site and your users to expand their reach & user information databases. Tribal Fusion, for example, requires 2k uniques per day, “professional site design” and a “highly active user-base”.

Performance Banners: Ok, so I saved my favorite (and perhaps worst for a small site) for last. Performance based advertising means that somebody is tracking clicks and/or conversions (actions) on their campaign and either only paying out on an actual “action” or adjusting bids on different sites according to the performance. Don’t be fooled by networks that pay you on a ‘CPM’, unless the ads they are showing are brand advertisements, somebody somewhere is tracking ROI & performance, and it’s impacting the CPM you’re getting paid. So why doesn’t this work for small sites you may ask? Well, long story short… small sites suck. Ok, it’s a bit of an exaggeration, small sites suck for performance would be a more accurate statement.

Lets go back to eyeballs. Your site has eyeballs, and I’m going to pay you everytime one of your eyeballs buys a pair of sexy shoes. Lets say the shoes cost $30, my production cost is $0.50 (yay for sweat shops!) so I’m willing to pay you $14.50 for every person from your site that buys one of my pairs of sexy shoes. Now there is one other advertiser in the network I’m working with he is selling sports gear (hats and stuff) and is willing to pay $10.00 to the publisher for every purchase.

Now, when a users comes to your site, which ad should the network show? Lets say your site is brand new to the network. Well, this network has no contextual technology to tell me what the site is about and he also doesn’t have a behavioral engine, so what can the network do? The only thing he can, test his campaigns on the site and learn what the performance is like.

Ok, for the next 5000 impressions (1k unique users?) the network randomly chooses between the sexy shoes and the sports paraphernalia. Now what… what do you think the chances are that somebody actually bought a pair of shoes or a hat? On the average site, I would imagine, right around 5-10%. Think about it, if the average CPM on your site is around ~$0.50, and this offer were to be competitive, you shouldn’t be expecting a single conversion until you’ve run at least 30,000 impressions. For better statistical accuracy on the performance of this campaign on one site, think more like 300,000 impressions. Starting to see why small sites suck? The amount of volume necessary to ‘optimize’ the average campaign is simply too much for the measly 5k impressions a day. Imagine if the network had 500 campaigns. How can learning be done on 500 campaigns with just 5000 impressions a day?

Of course there are various techniques to limit the amount of learning that needs to be done, but the fact remains that significant volume must be sent to an performance advertiser before one can get a good idea of the actual performance.

So, what should the small site guy do? Think about your eyeballs & page content. Is your content very relevant to the eyeballs? Then go contextual. Do you have particularly valuable eyeballs? Go for behavioral or brand. None of the above? Try performance based networks, but don’t expect good CPMs until you get high enough volume for some sort of learning to be done.

Next up? Not sure, probably some thoughts on learning techniques, merging behavioral and performance, and other ideas…

A few months ago I was trying to explain to my girlfriend how advertising works. After a couple discussions she said a rather interesting thing — “How come the travel industry seems to be the biggest advertiser out there? All I see are travel ads, no matter where I go!”. Well, turns out my girlfriend is a travel fanatic and it’s behavioral networks that have picked up on the fact and no matter what the content of the site she’s visiting they manage to find a relevant travel related ad to show her. Try it, go to travel.nytimes.com and click through to a couple articles. Then go to a random section of the site, say ‘health’, and chances are you’ll see a travel ad! (see screenshot below) Sounds like voodoo or big brother right? Wrong!

And peeking into my cookies, indeed there is some travel related fun in there (hard to tell what all means, since most cookies are encrypted):

There are many different behavioral players out there in the world each with varied success. Perhaps the most successful behavioral company on the internet is this small company based out of California called Yahoo. There are also a couple ad-networks that dabble in this space, one of the most popular is Tribal Fusion. Each approaches the challenges in a different way, but the core of how it works is the same for all.

So how does it work? It’s actually an amazingly simple concept. Lets say we have four websites:

1. gizmodo.com
2. engadget.com
3. slashdot.org
4. myspace.com

I am a user, and I visit each of these websites. Each of these websites also works with a brand new (not real of course) behavioral network called “BigBrother2.0″. On every page of these sites there is a little thing known as an ad tag that points to BigBrother’s adserver and requests an ad. Just for fun, lets assume that gizmodo also know your age & gender and passes that information to BigBrother. So you visit the first page, and an adrequest goes to BigBrother as follows:

http://ad.bigbrother.com/request?site=gizmodo.com&age=25&gender=m

When your browser requests the URL above, the following sequence of events happens:

1. Browser Requests content from ad.bigbrother.com
2. Server requests cookies (if any)
3. Browser sends over cookies
4. Server does some crunching, picks and ad
5. Server returns new cookie data & url to get get ad
6. Browser shows ad

Now the magic here is in the cookies. Even though you are visiting the site gizmodo.com, the cookie is under the domain ‘ad.bigbrother.com’. In step 5 where the server returns new cookie data, they’ll put in there when you last visited gizmodo.com and how many times you’ve been there today.

Now lets say I spend the morning checking out my tech-news, all the while receiving ads from ad.bigbrother.com. Each page I view gives them a little bit more information about me, and even though I’m on three separate sites, because the cookie spans all three they get a holistic view of what I”m doing.

So now I go to Myspace.com. This is a big site. Normally it’d be difficult to pick a good ad to show you. You’re one of the 18 billion myspace users and all you really do is upload photos and post silly comments about your friends on their blogs. BUT, when Myspace decides to let BigBrother2.0 show an ad something interesting happens. Lets walk through the steps of this new ad call in more detail:

http://ad.bigbrother.com/request?site=myspace.com

1. Browser Requests content from ad.bigbrother.com
2. Server requests cookies (if any)
3. Browser sends over cookies
1. Cookie contains your age & gender (25 & male) (encrypted)
2. Cookie shows you visited gizmodo.com 12 times this morning
3. Cookie shows you visited slashdot.org 3 times this morning
4. Cookie shows you visited engadet.com once this morning
4. Server does some crunching, picks and ad
1. Server plugs user data (25, male, 16 visits to tech sites) into a ‘profile engine’
2. Profile engine spits back ‘categories’, lets say ‘male, technology’
3. Server looks for ad campaigns targeted to ‘male & technology’
4. Server picks highest paying ‘male & technology’ targeted campaign
5. Server returns new cookie data & url to get get ad
6. Browser shows ad

Instead of showing you a random ‘punch-the-monkey’ ad, BigBrother is able to show you a highly relevant and targeted advertisement that you are far more likely to click on. The end result? Better ads for you and more money for the websites that you visit.

You can see from the above that building a behavioral adserver really isn’t the most difficult thing in the world. All you need to do is track history & have some sort of ‘mapping engine’ that says that slashdot.org is a tech site. The real challenge for behavioral companies is the classic chicken & egg problem. Without a rich user-base, the ad-network won’t be able to sell deals. Without deals the network won’t be able to convince publishers to put their tags on pages. This makes it very difficult for new behavioral players to enter the marketplace, although emerging marketplaces such as the RMX will make it easier in the future.

So what’s my opinion on all this? As long as the data that is being stored is encrypted and no Personally Identifiable Information (PII) is stored I’m all for behavioral targeting. As I mentioned before, the more relevant the ad is to my interests, the more pleasant my browsing experience is going to be.

Take this SpywareNuker page:

Some of the Cookie.Advertising.com components are listed below. The list is compiled as a reference. The list might not be complete and it doesn’t represent instructions for manual removal. We DO NOT recommend manual removal. Incorrect removal of certain software might make your computer unstable or even unusable.
Removal of adware component might affect the related ad-supported software.

Sorry to say, but this is total bullshit. Ad.com’s cookie doesn’t infect a thing and can be removed easily and safely by yourself. So what are adserver cookies and what’s stored in them? The most basic information that most adserver will want to know is:

• which ads you’ve seen, and how many times you’ve seen them
• which ads you’ve clicked on
• which ads you’ve converted on

Yes — that’s right, adservers know when you buy things after clicking on an ad — but that’s another post. Now more advanced behavioral companies will also want to track which sites you’ve been to and what behaviors (or segments) you belong to. Allowing adservers to track you using cookies means you won’t repeatedly see the same ad, and you should see more relevant ads to your interests as you continue your path along the internet.

So how do companies store this information? There are two ways — client-side or server-side. Client-side involves storing all the information about you in your cookie whereas server-side the cookie simply stores an ID and all information about you is stored on some database somewhere. Doubleclick as seen in the below screenshot seems to have gone the server-side route and simply stores my “id” which maps to some data they have on me in their databases. The server-side route is great if the advertiser wants to store a lot of data since there are size limiations on cookies and you don’t necessarily want to transfer 10kb back and forth on every ad call. It’s not so great as it requires some serious database infrastructure to handle 10-100k read/writes per second to a single database.

Burst seems to have gone the client-side route and stores all ads I’ve seen from them directly in my cookie files. It’s actually interesting to note that the cookie data from Burst is not-encrypted (big nono!).

Ok, so what does this all mean for the end-user? Well, indeed, there are companies out there that are tracking most of everything that you do online. Ad companies like Tribal Fusion know what you’re interested in and what you like to browse for. Hell, if Google succeeds in pushing Checkout to the world they’ll know everything you’ve bought, how much you paid for it and much more! All in all this sounds scary but it really shouldn’t be. Why?

First off, none of the information _should_ be personally identifiable. Cookies are tied to your computer, not your name. So if you switch machines often, or clear your cookies, all history is immediately erased. Now, some argue that if your browsing history is stored it should be possible to figure out exactly who you are. Lets be honest, why go through all the trouble if there are much easier ways of figuring out what you’re doing online? If you’re worried, go download an anonymous browsing tool (there are plenty), or visit the little known-of ‘opt-out’ pages that almost all online advertising companies have. Some examples: Advertising.com Opt-out, Yieldmanager Opt-out, Doubleclick Opt-out. If you do go the opt-out route, don’t clear your cookies because that’s the only way they’ll know you don’t want them to track you!

So whichever route you take, don’t forget that the websites you visit can provide you with free content because of the money they receive from advertisers. Cookies are one tool that advertisers use to help track revenue and regularly clearing them can cost your favorite sites money. My personal choice? I clear my cookies every month or so but am perfectly happy to let companies track my behavior to show me more relevant ads. Hell, I’d much rather look at an ad for some new tech gadget than ‘punch the monkey’!