Mobtown Labs

Quant v. Audience: The Real Challenge and Opportunity is Using Both

Fri, 03 Feb 2012 11:58:00 -0500

Guest Post by: Matt Patton Esq. Director of Optimization and R&D at DoublePositive

Among the many nuances in the debate about which optimization algorithm represents the best RTB media buying strategy, the most prominent might be the audience retargeting v. audience agnostic approach. Although exclusive to our business, this dichotomy is conceptually no different than two Wall Street analysts debating about whether a pure qualitative approach is better than a numbers-only quantitative method in purchasing securities. My engineering training tells me that the only inferences that should ever be drawn as a basis for any decision should originate with the story told by what was actually measured. But my legal training tells me what I think is the more correct approach, which is, as every law school professor professed so eloquently, “It depends!”

The truth is that measured performance numbers, i.e. impressions, clicks, conversions, and audience data, i.e. categorical audience profiling, both have their own benefits and drawbacks, and will almost always provide you different oscillating returns on your media investment. On the surface, the only real difference between the two is that one is instinctual while the other is empirical. With audience data, we infer that a group of users with certain online buying habits will purchase what our clients are selling based on the categorical distance between things previously purchased and what we are currently selling. With fundamental performance data, we just care about which pieces of inventory have already yielded a statistically significant number of conversions regardless of who actually converted. Thus it is blatantly clear under the surface that both methods offer the same benefit, which is a calculated a priori probability of a conversion from which we can derive an expected value of an impression, or what my mentor at Advertising.com used to call the “Crown Jewel”, of DR advertising.

The obvious drawback of both approaches is and will always be the cost of learning. Because of the random nature of Internet advertising, we need to survey the biddable landscape before we can make any type of calculation, qual or quant. In the quant world, it probably takes more time and thus requires a longer discovery period, while in the audience buying game, those “selective” impressions can get quite expensive when we are still learning which audience represents our sweet spot. Since there is no such thing as a free lunch in this business, both are going to require some upfront cash that will initially skew our calculated CPAs.

Instead of simply accepting this as truth, I want to treat this as an opportunity to thwart the economic black hole of overcoming a campaign’s inertia when starting from rest. I see opportunity to utilize both approaches as a function of time and campaign maturity in order to get campaigns running more quickly and performing better long term. I see complex regression analysis and time series plots that will tell us on a campaign by campaign basis of when and how to use audience targeting and when the stats tell us that the audience is worthless. I see recurring real time cost benefit analysis algorithms telling us that we are paying too much for an audience or that an audience is underpriced. I see this as a first step towards creating a RTB marketplace where being smarter actually translates to being more successful. For the first time, I see a scientific approach to an age-old art and an artistic approach to centuries-old mathematics.

Will this be accomplished and will it even make a difference? I leave you as a lawyer by saying “it depends”.

The Rise of Media Quants (adweek.com)
Hey Quants! Hedge Funds And Banks Will Fight For You In Bidding Wars And The Winner Will Be… (businessinsider.com)
New Survey Reveals Increased Focus on Risk Management in Wake of Financial Crisis (prweb.com)
Peter Muller’s Quants Find a New Home (blogs.wsj.com)

Complex UI - There's a Time and a Place

Thu, 02 Feb 2012 13:38:43 -0500

Linkedin maps data visualization (Photo credit: luc legay)

I am currently working on a advertising and marketing reporting interface that is mostly grid driven and we are adding some data visualization to it because quite frankly the thought is that most users just don’t have that ‘ah-ha’ moment when data is presented to them in a grid. Even if they are somewhat simple stats grids can be daunting to look at. That said, a well paginated grid (or better yet infinitely scrolling grid) can be the most powerful tool.

To decide whether to go more complex on a grid or to pop out data visualization you need to consider time spent on that particular part of the application. You also need to decide whether the visualization of the data is just to show off and be fancy or if it really has a use.

For example when presented with a grid of data from an online advertising campaign, it’s easy to go through and see how much money you spent on a particular campaign for…say 30 days or even 90 days but once you get above about 100 rows you really need a trend chart to see where the spikes and troughs in spend are.

Another good use of a data visualization are nested trees or what the javascript d3 library calls a Dendogram. To zero in on, or drill down into data expandable rows in a grid are useful but if you want to come over many layers of nested data to see what is really driving value, a visualization is probably a better choice. That said, as a rule of thumb, I would always present the grid option so you can see precision in numbers once you have identified your ‘significant’ data points.

Data Visualization Resources (secviz.org)
Visual Data Tools, Inc. Announces the Mac App Store Release of DataGraph, an Innovative and Powerful Graphing Application for Mac OSX (prweb.com)
Data Visualization - Is it Important? (mobtownlabs.com)
Cascades: Rich UI Development for BlackBerry Apps (devblog.blackberry.com)

Data Visualization - Is it Important?

Thu, 26 Jan 2012 14:44:51 -0500

I recently came across this tweet from the founder of Posterous (sorry I’m a tumblr fan).

Cutting features is hard post.ly/4RfNf
— Sachin Agarwal (@a4agarwal) January 6, 2012

We are in the midst of brining one of our key products out of beta to it’s ready-for-prime-time v1.0 and like any technology project we need to do it profitably, we wish we had more engineers, and we wish we had more time to include or perfect every feature we have dreamt of in the planning process. That said we need to make cuts. One recurring debate is whether or not charts and graphs and creative ways to visualize data is a bell/whistle or is it a core component that could propel the product into being a wild success.

Like many other companies that deal with online media we get millions or even hundreds of millions of events and logs to parse through each day and run in massive data warehouses and are rolled up into useable statistics each hour. There are the standard stats that we need to watch on a regular basis as well as report out to our clients but when we dig into a campaign or try to figure out how to increase performance we are always downloading out to excel or writing a Hive/SQL query to get a deeper look or a different dimension of the data and then graphing it for a nice friendly viewable representation.

And it never fails that the data representation that wins the business, gets the next round of funding, or wins over an executive’s good graces to allow a budget/project/initiative to move forward is inevitably a very simple graphical representation of possibly billions of data points.

So if this is so important, how can it be looked over? And a better question is why?

azspot: Steve Jobs (1955-2011)

Wed, 05 Oct 2011 22:05:45 -0400

azspot:

Steve Jobs (1955-2011)

pitchfork: R.I.P.

Wed, 05 Oct 2011 22:05:10 -0400

pitchfork:

R.I.P.

[Tornado] ImportError: No module named ioloop

Tue, 27 Sep 2011 14:47:23 -0400

yes, i did the same thing….duh…

alanhogan:

In case anyone is as dumb as I am:

If you are trying to run a Tornado server and keep getting the error ImportError: No module named ioloop, it means you shouldn’t have named your own file tornado.py. There can only be one tornado, and yours ain’t got ioloop. Rename your file to something like serve.py and get on with it.

Introducing Brom.ly

Mon, 01 Aug 2011 11:36:00 -0400

A good friend of mine and former Ad.com and AOL colleague, Greg Skipper, has been working on a startup. It’s a really cool app that they describe as a personal concierge service. Basically you find yourself in a new city (or even your own neighborhood) and you open the app on your mobile device and it recommends things that are going on around you and as you choose things it learns more of what you like i.e. I don’t care for sports so the more I pick non-sports related events the less it recommends sporting events.

Obviously, the success of this depends on the accuracy of the app learning what you like but so far I’ve been impressed. I also think the business model around this to possibly integrate with the daily deal world or start connecting individuals and start mobile tribes of recommendations for like minded people could be really cool and lucrative for the Brom.ly team

Here are links so you can try out the app:

o http://brom.ly

o http://itunes.apple.com/us/app/brom-ly/id444067080

o https://market.android.com/details?id=ly.brom

Here is some great press about them:

wohamusic: Yeasayer-Madder Red Kristin Bell stars in Yeasayer’s...

Sun, 10 Jul 2011 17:08:03 -0400

wohamusic:

Yeasayer-Madder Red

Kristin Bell stars in Yeasayer’s “Madder Red” alongside a creepy freaky creature. Great song, weird music video. What do you guys think?

-Stephanie

sooo weird….

Local Natives - Airplanes

Sun, 10 Jul 2011 17:02:12 -0400

Local Natives - Airplanes

No Alternative

Sun, 10 Jul 2011 16:28:53 -0400

No Alternative

bijan: Cold War Kids - Hospital Beds via daytrotter sessions.

Sat, 11 Jun 2011 11:46:26 -0400

bijan:

Cold War Kids - Hospital Beds

via daytrotter sessions.

DOM - Living in America Yesssssssssssss…..heard this on...

Fri, 27 May 2011 12:06:21 -0400

DOM - Living in America

Yesssssssssssss…..heard this on the radio driving in to work, didn’t know who it was and finally found it by typing in random lyrics to google.

Thank you, google for search, the internet for inventing itself and making this possible, python for text scan search, whoever invented the mp3 and everyone else who worked to make this moment possible.

bijan: The Strokes - I’ll Try Anything Once love this demo...

Sat, 14 May 2011 10:19:47 -0400

bijan:

The Strokes - I’ll Try Anything Once

love this demo version.

Cover of Archers of Loaf Archers of Loaf - The Lowest Part is...

Sat, 16 Apr 2011 10:23:54 -0400

Cover of Archers of Loaf

Archers of Loaf - The Lowest Part is Free

Can’t wait for their summer tour….

Awesome presentation on Media Planning and Media Buying

Thu, 14 Apr 2011 11:29:18 -0400

Awesome presentation on Media Planning and Media Buying

DoublePositive's New Website With our Display Optimization Product

Thu, 24 Feb 2011 15:55:22 -0500

DoublePositive's New Website With our Display Optimization Product

RingRevenue and DoublePositive Join Forces to Make Your Phone Ring

Tue, 15 Feb 2011 16:32:41 -0500

RingRevenue and DoublePositive Join Forces to Make Your Phone Ring

Getting web.py running on IIS

Thu, 20 Jan 2011 19:16:13 -0500

Image via Wikipedia

web.py is a great super light weight framework if you need to get a fast site up that uses python on the backend to process a lot of data. There are plenty of other options like pylons or django or just using python’s cgi scripts but I wanted something that I can throw up, make a lot of reports, paste in different sql from different ms sql tables and make a handful of reporting templates to throw together a few different configurations of this data and web.py seemed like a good solution because there’s no command line builds of scaffolding ala rails or django and you aren’t tied (or necessarily encouraged) to use native ORM’s if you already have the sql ready to go. It does however have a robust python based native templating engine and language that’s really easy to learn and the url routing is increadibly easy and logical so you can make a few different pages or a few different apps and get all of the view templating of rails or django but not all of the forced structure.

One problem. It doesn’t easily run in a CGI bin or on Windows IIS which my company uses. It’s ready to go on Apache or lighttpd or nginx. I figured out how to do this using a discussion here.

1. configure IIS to run python and .py files

2. install web.py on your server. download here. command line ‘setup.py install’

3. put flup on your server. download here. and don’t install it yet….

4. find fcgi_base.py in that flup package and in it comment out this block of code…like this:

   #sock = socket.fromfd(FCGI_LISTENSOCK_FILENO, socket.AF_INET, 
    #                            socket.SOCK_STREAM) 
      #      try: 
        #        sock.getpeername() 
         #   except socket.error, e: 
           #     if e[0] == errno.ENOTSOCK: 
                    # Not a socket, assume CGI context. 
             #       isFCGI = False 
               # elif e[0] != errno.ENOTCONN: 
                 #   raise

5. then above it you’ll see isFCGI = True. change that to isFCGI = False

6. then go back to the flup directory and run ‘setup.py install’ from the command line.

7. then drop your app in a CGI enabled folder on IIS and you should be good to go.

Only catch is your urls will have a root of http://server/app/app.py/ and in your app for the urls variable it will look like this or a variation of this:

urls = (
  '/app/app.py(.*)', 'index'
)

I’m new at this stuff so if I missed anything, feel free to chime in but this is not documented in webpy.org and it took me about a week to figure out so I figured I’d post it.

Comparing Python frameworks: A simple webapp in Flask, web.py, bottle, juno … (agiliq.com)
Term Extraction | fivefilters.org (fivefilters.org)
Welcome to Web2py Utils’s documentation! - Web2py Utils v0.0.13-alpha documentation (packages.python.org)

Archers of Loaf - Wrong This is one of the best bands of all...

Mon, 17 Jan 2011 22:21:47 -0500

Archers of Loaf - Wrong

This is one of the best bands of all time and they played a show last night in North Carolina for the first time since 1998.

Deserted: Archers of Loaf ‒ Vee Vee (kexp.org)

How Important is Giving Credit

Wed, 22 Dec 2010 22:36:00 -0500

Image via Wikipedia

So I have been hard at work at my new job here at Double Positive and we should be publicly launching a new product in the next month or two. This post is not about taking credit for that product or giving others credit for building it (that said they do deserve it, as I work with some incredible people here and they are building some amazing things). It’s about tracking online advertising and the obsession of getting credit for conversions and giving credit for conversions.

Some quick definitions:

Last Click - The ad that is the last to be clicked on before a user converts within a certain window of time (typically 30 days) is the one who gets credit for driving that conversion.

Last View - The ad that is the last to be viewed before a user converts within a certain window of time (default in most adservers is 14 days but people adjust this one a lot more) is the one who gets credit for driving that conversion.

Last Click Trumps Last View - In this model if there are no clicks prior to a conversion the last ad viewed wins the conversion but if there are any clicks prior to the conversion the views are disregarded and the last click wins the conversion.

These are the most classic conversion models. Last view seemed to be sweeping the marketplace a couple years back but I see a lot of big brands going back to the tried and true last click model.

The truth is that everyone knows that it’s a combination of ads that typically drive a consumer to convert but the problem is most adservers these days can’t handle a multiple ad attribution model. The other thing that gets in the way is that a lot of affiliates and ad networks want to offer a CPA model and make some margin for their hard work (which is totally deserved) and the Advertiser and the Affiliate/Network need to have a rock solid audit-able method of tracking this so as to not over-pay for sales.

Given some of the research we have been doing over the past couple months we have found that there might be an opportunity to have your cake and eat it too. That said with all reward comes risk with it. We’ve been messing around with different methods of statistical significance using Tagman and some home-grown tools and of course a lot of excel sheets to determine if a model can be built that will evolve with the campaign.

This is predicated on a couple things:

1. You can track all conversions that are going through an advertiser’s site. [mandatory]

2. You can track all clicks and natural traffic that are going to the advertiser’s site. [mandatory]

3. You can track all ads purchased to drive users to an advertiser’s site. [optional]

4. You can throttle ads up and down by channel. [optional but possibly mandatory]

Number 1 is mandatory because you need one (and only one) count of all the sales coming in to the site and a user id that you can match upstream to the clicks.

Number 2 is mandatory because you need to know how and when users arrive on your site. The how is not as important as the when. The idea here is if a user sees a display ad and then types in the advertisers url, you can link those three events if they are in a chronological time series and then by the user id from the adserver.

Number 3 is optional because first off, there’s no way Google will let you get a cookie dropped on search ad views and what you are really looking for is statistical significance in a chronological time series of advertising, click/type-in to site, and then conversion so tracking all ad views isn’t critical but tracking as much as you can without breaking the bank in adserving fees definitely helps.

Number 4 is optional/mandatory because in a perfect world all ads are RTB enabled and you can write a computer program to do a time series of cascading ad bursts and then track statistical significance of when a particular ad channel is increased and decreased and watch conversions spike and trough and then you can build your model out as to which ones actually drive sales. The fact is that the world isn’t perfect so some ads need to be bought in bulk (homepages), others need to be reserved, and others need to be bidded up and down and hope that the volume of ad impressions follow. So just like #3, do your best but don’t worry about being too much of a perfectionist.

Now that you have all of that set up, the goal will be to make changes over specific periods of time with each channel of your advertising and identify key pathways and then assign a coefficient of statistical significance to the incremental conversion boost you get (or don’t get) and adjust your budget and price (or bid) you are willing to pay for that channel of ads moving forward.

This should help you develop a model for buying advertising based more on throttling price and budget based on what is driving incrementality rather than obsessing about ‘giving credit’.

Increasing the Performance of your Digital Media Spend with Cross-Channel Conversion Attribution (firstrate.co.nz)
Optimise paths to conversion, not channels (econsultancy.com)
From Clicks To Calls (searchengineland.com)