Web Services

A Modest Proposal on How To Commoditize Away Google’s Advertising Revenues.

New eBay blogger Josh Scott muses about Bill Gates’s comments that Google isn’t really free because they serve up ads. To quote Josh paraphrasing Bill:

[S]earch engines like Google get their revenues from advertising because people use these search engines, but they don’t share these advertising revenues with the end users who help them get the revenue.

Later on, Josh hits on my number one long-time issue with making money from search:

[Google should] be concerned about the lack of network effects, but also by the related fact that the switching costs both as a searcher and as an advertiser are so low.

I use Google now because it’s the best. But I used to use AltaVista. And before AV, I used Lycos. If (when?) someone else comes along, I’m going to switch.

Google has no inherent structural advantage over other companies in regards to search. They are no lock-in costs or network effects. They’re just better at it. But companies such as Microsoft and Yahoo! and eBay can also hire engineers who can write search engines. They can even hire away Google engineers.

If eBay sellers will go through the hassle of selling their items through multiple channels, you better believe advertisers will go where they get the best bang for their buck, too. As long as Google continues to aggregate demand and provide the “best” clicks, they’ll continue to capture massive value.

However, when other companies catch up, either by providing a better service or by paying me to use their search engine, Google will need to cut into their AdSense margins to remain competitive. This hurts Google and all search engines on the supply side.

In related news, Tim Bray talks today about “The Future Search Market” He describes an application that has a Web search window and:

When someone types in “Britney Spears” or “Mayan Eschatology”, I send the query off three different search engines who pay me a small retainer for the privilege of getting them…. You could imagine an alternative setup in which you send the search terms to the engines and all they come back with is their per-click bid price, and then you only send the actual search to the winner.

It’s certainly true that companies such as Google and Yahoo! pay to be the search provider for popular applications, such as Firefox and Safari. They also pay to be the search provider for popular Web applications, such as AOL and the Washington Post. However, this doesn’t need to apply to big companies.

Another way Google currently pays for traffic is through their AdWords program. Tim’s idea applies just as well (if not better) to any page on the Web that runs contextual ads — which are essentially embedded search results intelligently served-up based on a fancy back-end algorithm. This algorithm guesses what a visitor would have typed into the search box (if only one had existed).

Right now, Google (presumably) has a nice margin between what they charge advertisers (via AdSense) and payout (via AdWords). This makes them a tidy profit.

But Yahoo! and Microsoft have similar products. As a content provider, you’d switch from AdWords if you made more money from the Yahoo! Publisher Network. But right now, your choice of contextual advertisements is all or nothing: you’re either with Yahoo! or Google or Microsoft. You have a few choices, but you’re forced to pick one and stick with them until you switch.

This is quite coarse and inefficient. You’re leaving money on the table because it’s not really a question of whether you make more money overall from one company or another. What you should really care about is whether you can make more money for them (and thus from them) for this specific visitor at this specific instant in time.

Tim’s idea becomes:

[A]n alternative setup in which you send the search terms page URL to the engines and all they come back with is their per-click bid price, and then you only send the actual search to serve contextal ads of the winner.

If Microsoft and Yahoo! want to make a big dent in Google’s AdWords business, they should provide a pricing Web service for contextual advertisers. By providing transparency in the market, they’d commoditize away Google’s demand side, too.

If you were a company that made money from multiple channels and you were facing a large scary competitor that made 99% of its money from advertising, wouldn’t you do your best to erode as much profit margin from advertising as you could?

ApacheCon Slides Are Finished

ApacheCon has begun and I am happy to say I have finished my ApacheCon slides. My talk on Consuming Web Services Using PHP 5 isn’t until Wednesday afternoon. Therefore, I technically have a few days left before I need to get up on stage and present, so I’m counting this as a victory for getting my act together in a timely manner.

For this talk, I decided to demonstrate Web services using a number of real Web services, so you can get a flavor for how people are actually implementing Web 2.0. Specifically:

  • del.icio.us (bought by Yahoo! earlier this week)
  • flickr (bought by Yahoo! earlier this year)
  • eBay (almost bought by Yahoo!, almost bought Yahoo!)

And, of course, there’s the obligatory Google Maps reference because, well, it’s a rule or something. I think. I can’t remember.

This was my first presentation done in Keynote. It’s certainly easier to make non-ugly slides in Keynote than in PowerPoint, which almost seems to lead you down the path of ugly slides. You think Microsoft could invest in a few good new templates instead of still using the ones their intern programmers designed in 1992.

If you’re going to be at ApacheCon, let me know. I am arriving Monday afternon and leaving Wednesday night, and staying at the conference hotel. I will have a rental car, so if you know a good place to eat or drink or visit that’s off the beaten path, we can help each other.

eBay Developer Challenge 2006

I’m tracking some nice mentions of our coding contest:

I am hiring: Help drive eBay Web Services

I am hiring a person to figure out how we should be exposing eBay using Web services and then creating the product proposals to implement the strategy. You also get to monitor everything else that’s going on at eBay and make sure it’s API-enabled in an platform savvy way.

If you’re interesting in participating in this thing popularly known as “Web 2.0,” then this is a killer opportunity. We serve up 2 billion (with a b) API calls a month, and we just made all calls free, so we’re talking serious scale and opportunity.

The ideal candidate has some strategy, programming, and product experience in her background. A little financial analysis wouldn’t hurt, either.

Here’s the official job description. That’s Requisition No. 7471BR at ebaycareers.com, in case the link doesn’t work. (Mac and Linux friends, please don’t flame me for the Win/IE bigoted job recruiting web site. I had to boot up my girlfriend’s Windows box to use the site when I applied, too.)

If you’re interested, e-mail me or apply directly and let the recruiter know you found out about the job through my blog. My work e-mail address is the first initial of my first name (a), followed by my entire last name (trachtenberg), at the ebay.com domain.

Opening up eBay one free API call at a time

I am extremely happy that today eBay made all of our API calls completely free of charge. You can get 10,000 API calls a month just by signing up, and when you pass a relatively painless evaluation, we’ll increase that limit to 1.5 million calls a day. Now that’s a call limit you can be proud of.

For a long time, eBay’s offered up our API for both commercial and non-commercial usage, but we’ve always hampered ourselves by charging for access. The fees were low enough for commercial companies to write applications, but in a world where information wants to be free, we’ve been pricing out all the people who want to play with our data to see what interesting things they can build, remix, and give away.

Breaking down those barriers has been one of my primary goals since I joined eBay last summer. We got partly there in June, and we’ve completed the journey today. Now all I need is to do my job and convince you to start writing eBay applications because I can’t use the pricing excuse with my boss anymore.

Fortunately, we are also launching a pretty sweet coding contest — eBay Developer Challenge 2006 — today. First place gets $5,000 plus a free trip to the O’Reilly Emerging Technology Conference. We will pay for plane and hotel, and O’Reilly has kicked in a ticket to the show. Runners up can get money, iPods, xBoxen, trips to eTech, etc.

There’s lots of really interesting applications waiting to be built that hook up to eBay. Since the eBay database is always changing: new items being added, existing items being bid on, old items being sold, the data is quite dynamic.

From my point of view, dynamic data -> dynamic applications -> useful applications -> happy users. And I like happy users.

See also:

Zend Framework, IP, and Big Company Lawyers

IP Concerns are very real; I’ve had a couple of customers bring them up in the past. It boils down to trust; It doesn’t matter if you and a thriving community have written the best code in the world; without someone to vouch that all proper measures were taken, there is an increased risk associated with your project.

This is the second time I’ve heard IP as one of the motivations behind the Zend Framework.. The first time was from David, and now there’s this line from Wez.

I never used to think about clean IP. I knew that some organizations, such as the Apache Foundation, make committers sign contributor license agreements, but as I am not an ASF contributor, it wasn’t something I had to deal with.

That’s changed now that I’m at eBay.

Last June, we announced that we were going to open source our Windows and Java SDKs. These are wrappers around our Web services for developers that make it easier to write code that links up with eBay. There’s nothing fancy about them, so the benefits of open sourcing them far outweigh the risks.

However, all of a sudden I have a whole new set of issues to worry about:

  1. What open source license will I use?
  2. What about eBay IP that might be part of the SDKs?
  3. What if someone contributes something they don’t own the rights to, or is under a non-compatible license, or has a patent on, etc, etc.?

What a mess. It’s not easy open sourcing anything non-trivial when you’re a big company — especially when it’s part of an active project.

I must admit our legal team was surprisingly clueful when it came to open source. Somethings they knew. Others they didn’t, but they found a firm to assist who knows a lot about open source licensing and issues.

I had a long conference call with four lawyers, and at no point was I forced to explain that all open source licenses were not like the GPL. Actually, that’s unfair because these guys not only said smart things, they asked smart questions, and had smart answers to my questions.

The original license I suggested was the good old MIT License. However, that didn’t pass our internal IP test. The MIT License was written in a world where software patents weren’t a major legal issue, and for better or worse we no longer live in that world.

So, to protect ourselves, we moved away from the MIT License. At first, our legal team wanted to write their own custom license, but I pushed back and asked them to explore the Apache 2.0 and CDDL. Apache 2.0 didn’t work for them, but the CDDL did. (Much to their surprise. Big shout out to the Netscape and Sun lawyers for writing something the eBay team was happy with.)

Things are not 100% final, but I hope that we’ll be able to release our code under the CDDL late this year or early next year.

Of course, that had some additional side effects, as we had to scour our existing code to make sure there were no license incompatibilities with the CDDL. Apache Axis uses Apache 2.0, which looks to be okay.

Unfortunately, we had spent some time writing an example that hooks up to a MySQL database using MySQL Connector/J. Frankly, I’m not sure if it was ever legal for us to bundle the MySQL code or even publish the sample code given MySQL’s interpretation of the GPL. Anyway, that code is out. It’s not worth it for me to get a commercial license from MySQL and I can’t tell what is or isn’t allowed.

Actually, if someone from MySQL can update their FLOSS exemption page to include the CDDL, we may be good. Since CDDL is similar to MPL, I hope they don’t have any problems doing this. But my lawyers don’t enjoy me playing with GPLed code, so we may not be good after all. We’ll see.

In the reverse direction, we patched Apache Axis with some improvements. However, I believe we just supply the patches with our SDKs as standalone files, and we’ve never contributed those back to the main trunk. I don’t know why. I don’t know if it was for technical or legal reasons, or just because we were lazy. But I would love to figure out what I can do to help share these changes with the extended Apache Axis community.

It just goes to show how licenses and IP really do matter, and big companies really do take this seriously. This is a good thing, and only goes to strengthen open source licenses. However, I think it’s come as a surprise to some open source developers.

I always see open source developers worry immensely about mashing up GPL code with BSD code, yet, at the same time, they don’t bother to check to see where their commits are coming from and if they’re clean. Therefore, they’re even more surprised when a big company cares.

Again, I think this is another big step in the right direction for open source. We all know free refers to more than just cost. It’s also the freedom to modify and redistribute code. At the same time, the cost of something is more than the purchase price. It’s also the risk associated with it. By reducing risk in all forms, we make the code even more free.

eBay SOAP Update: Syntax Matters

I saw a request for some actual SOAP code, so I will try and oblige. I don’t want to publish the entire code because there’s lots of messy stuff that’s specific to eBay’s SOAP API. I’ll talk about that some other time, but for now, those details just get in the way.

Therefore, I’ll pull out a few lines, combine that with some hand waving, and just hope for the best. Here’s the before:


$wsdl = 'http://developer.ebay.com/webservices/latest/eBaySvc.wsdl';
$client = new SoapClient($wsdl);


$params = array('Query' => 'ipod');
$results = $client->GetSearchResults($params);

foreach ($results->SearchResultItemArray->SearchResultItem as $item) {
  print $item->Item->Title . "\n";
}

Without the classmap option, I create a SoapClient, make my request, and then iterate through the results to print out the titles of the matching items. This isn’t complex; however, the iteration is a little klunky due to limitations of SOAP and the design of eBay’s Web service.

For example, the SearchResultItemArray only contains SearchResultItem, so it’s kludgy to reference SearchResultItemArray->SearchResultItem. Likewise, when I’m just getting a quick dump of Item information, it’s not so nice to specifically access the Title element.

By defining a couple of classes and telling the SoapClient object map them to the return data, I can clean this up:


class eBaySearchResultItemArrayType implements IteratorAggregate {
  public function getIterator( ) {
    return new ArrayObject($this->SearchResultItem);
  }
}

class eBaySearchResultItemType {
  public function __toString() {
    return $this->Item->Title . "\n";
  }
}

$wsdl = 'http://developer.ebay.com/webservices/latest/eBaySvc.wsdl';
$options = array('classmap' => array(
  'SearchResultItemArrayType' => 'eBaySearchResultItemArrayType',
  'SearchResultItemType' => 'eBaySearchResultItemType',
  ),
);

$client = new SoapClient($wsdl, $options);

$params = array('Query' => 'ipod');
$results = $client->GetSearchResults($params);

foreach ($results->SearchResultItemArray as $item) {
  print $item;
}

To solve my first problem, the iteration, I make eBaySearchResultItemArrayType implement the IteratorAggregate interface. When a PHP 5 class implements this interface, PHP will invoke the getIterator() method during a foreach loop.

In this case, I return $this->SearchResultItem, wrapping it inside an ArrayObject to make the array iterable.

For the pretty-printing issue, I define a __toString() method inside of eBaySearchResultItemType. Now, when I print an instance of this class, PHP calls that method instead.

With my classes defined, I use the classmap option to map the PHP classes to the SOAP complexTypes, and pass this mapping along as part of the second parameter to the SoapClient constructor.

Once this is set up, everything else in the request is identical. However, when I print out the results, the syntax is clean:


foreach ($results->SearchResultItemArray as $item) {
  print $item;
}

At one level, this is just syntax and icing. However, I don’t think you should dismiss syntax with a wave of your hand. To quote Sam Ruby on C# and LINQ:

[S]yntax matters. Very much so.

So, I am convinced that this is a good thing to spend time on. There are a couple other tricks I’ve pulled out, such as:


class eBayFeesType implements ArrayAccess {

  public function offsetGet($offset) {
    foreach ($this->Fee as $value) {
      if ($value->Name == $offset) {
        return $value;
      }
    }
  }

  /* and the other interface methods... */
}

class eBayFeeType {
  public function __toString() {
    return (string) $this->Fee->_;
  }
}

This lets me do:


echo "Listing fee is: ", $results->Fees['ListingFee'], "\n";

Which I must say is far nicer than either iterating in place or even calling out to a utility method.

I was interested to discover that I needed to manually cast $this->Fee->_ to a string because I thought it was a string. Actually, it is a float, and PHP won’t autocast floats to strings in this instance.

Personally, I think if PHP should autocast here, but I can see the logic. If you’re promising to return a string, you should actually return one.

At first, I was writing each class by hand. But that got boring and wasn’t scalable. Therefore, my new goal is to automate this process. I am writing a script to read the WSDL file, parse out the complexTypes, and then convert them to PHP classes.

My original script was using the SoapClient::__getTypes() method as a pre-processor, but I needed to parse that output again to get it into a usable format. After a little digging, I discovered that PEAR::SOAP implements its own parsing routines and gives me a far more flexible PHP data structure to manipulate. So, I am going in that direction instead.

Since this is a one-time action, speed isn’t vital, so it’s okay that PEAR::SOAP is written in PHP.

If you’re interested in more details on these new PHP 5 features, they’re documented in the PHP Manual. Alternatively, in a shameless plug, you can check out my book Upgrading to PHP 5.

PHP 5 SOAP Hacking

My new favorite PHP 5 SOAP feature is the classmap option. By default, ext/soap converts complexTypes into stdClass objects. However, that’s only useful for reading the data.

Using the classmap option, you can specify how ext/soap should map complexTypes to PHP classes. This allows me to write methods to implement iterators and pretty-printing of objects.

I hope I can construct a series of model classes (similar to what Services_Ebay), so if you have an User object from populated from the output of, say, GetUser, you’ll be able to call a GetFeedback method on that object, and have it make a GetFeedback API call and fill in the necessary data based on what’s already stored within the object as properties.

Ideally, I’ll be able to do some magic __call() overloading, so I don’t even need to explicitly define these methods, but I can just write one magic base class. I’m not sure if that will work, but it’s my weekend hacking project.

Also, while I’m at it, I’m having problems getting the compression option to work. I get a nasty SoapClient::__doRequest(): SSL: fatal protocol error when I try. It looks as if zlib is enabled, so I don’t know if SSL and Gzip don’t play nicely together, or if my SOAP server is busted, or what. I’ll work on debugging this, but if anyone else has something to share, I’d be happy to hear it.