Practice and Experience Talks

The following practice and experience talks will be given on February 3 – 5, 2015.

• New Directions in Recommender Systems, Jure Leskovec (Stanford)
• Semantic Matching in APP Search, Juchao Zhuo (Tencent)
• Boosting Search with Deep Understanding of Contents and Users, Kaihua Zhu (Baidu)
• Regressing Towards Simpler Prediction Systems, Tushar Chandra (Google)
• Global Optimization for Display Ad, Rong Jin (Alibaba)

New Directions in Recommender Systems

Jure Leskovec (Stanford)

Abstract: Recommender systems are an integral part of how we experience the Web today and they have become so ubiquitous that we do not even notice them anymore. However, today’s recommender systems mostly treat items they recommend as black boxes and primarily focus on extracting correlations and co-counts from user behavior data. In this talk I argue that next generation recommender systems will require deep understanding of items being recommended as well as modeling the relationships between those items. I will present examples how auxiliary data about items (descriptions, reviews, product specifications) can be used to improve recommendations.

Slides are available here. Video is available on YouTube and Youku (password: wsdm2015)

Semantic Matching in APP Search

Juchao Zhuo (Tencent)

Abstract: Past years, with the growth of smartphones and applications, APP market has become an important mobile internet portal. As an important function in application market, APP search gains lots of attentions. However, mismatch between queries and APP is the most critical problem in APP search because of less text within term matching search engine. This talk will describe a semantic matching architecture in APP search，which mining topics and tags in big data. It enriches query and APP representations with topics and tags to achieve semantic matching in search.

Some challenge must be considered:

• How to extract tag-APP relationship from large web text.
• How to use machine learning technologies to process denoising and computing confidence.
• How to hybrid ranking apps retrieved by different matching method.

These will be introduced in some of our related works and as examples to describe how semantic matching is used in Tencent MyApp, an application market which serving hundreds of millions of users.

Slides are available here.

Boosting Search with Deep Understanding of Contents and Users

Kaihua Zhu (Baidu)

Abstract: Recent years have witnessed dramatic changes in how people interact with search engines. Search engines are expected to be more intelligent in understanding users’ intention and fulfilling users’ needs with direct answers rather than raw information. Furthermore, search engines are expected to be equipped with recommendation and dialogue capabilities, making the interaction with users more natural and smoother. In this talk, I will introduce Baidu’s work on how to make some of them come true through the deep understanding of users, queries and web pages, and discuss challenges behind these technologies.

Slides are available here.

Regressing Towards Simpler Prediction Systems

Tushar Chandra (Google)

Abstract: This talk will focus on our experience in managing the complexity of Sibyl, a large scale machine learning system that is widely used within Google. We believe that a large fraction of the challenges faced by Sibyl are inherent to large scale machine learning and that other systems are likely to encounter them as well. Thus, these challenges present interesting opportunities for future research.

The Sibyl system is complex for a number of reasons. We have learnt that a complete end-to-end solution has to have subsystems to address a variety of different needs: data ingestion, data analysis, data verification, experimentation, model analysis, model serving, configuration, data transformations, support for different kinds of loss functions and modeling, machine learning algorithm implementations, etc. Machine learning algorithms themselves constitute a relatively small fraction of the overall system.

Each subsystem consists of a number of distinct components to support the variety of user needs. For example, Sibyl supports more than 5 different model serving systems, each with its own idiosyncrasies and challenges. Alternatively, Sibyl configuration contains more lines of code than the core Sibyl learner itself. Finally existing solutions for some of the challenges don’t feel adequate and we believe these challenges present opportunities for future research.

Though the overall system is complex, our users need to be able to deploy solutions quickly. This is because a machine learning deployment is typically an iterative process of model improvements. At each iteration, our users experiment with new features, find those that improve the model’s prediction capability, and then “launch” a new model with those improved features. A user may go through 10 such launches before they run out of good ideas for further improvements. Not only is speed of iteration crucial to our users, but they are often willing to sacrifice the improved prediction quality of a high quality but cumbersome system for the speed of iteration of a lower quality but nimble system.

In this talk I will give an example of how simplification drives systems design and sometimes the design of novel algorithms.

Slides are available here. Video is available on YouTube and Youku (password: wsdm2015).

Global Optimization for Display Ad

Rong Jin (Alibaba)

Abstract: Online display advertisement has been examined by numerous studies. Most online display ad systems take the greedy approach, namely they display, for each user, the set of ads that match best with the user’s interests. One shortcoming of the greedy approach is that it does not take into account the budget limitation of each advertiser. As a result, we often observed that some ads are popular and match with the interests of millions of users; but due to the budget restriction, these ads can only be presented by a limited times, leading to a suboptimal performance.

To make our point clear, let’s consider a simple case where we only have two advertisers (i.e. A and B), and two users (i.e. a and b). We assume that both advertisers have only a budget of one display. We further assume that user a is interested in both ads even though he is more interested in ad A, while user b is only interested in ad A. Now, if we take the greedy approach, we will always present ad A to user a; as a result, if user a comes before user b, we will have no appropriate ad to be displayed for user b. On the other hand, if we can take into account the budget limitation of both advertisers, a better approach is to present ad B to user a and ad A to user b. This simple example motivates us to develop the global optimization approach for online display advertisement that explicitly take into account the budget limitation of advertisers when deciding the ad presentation for individual users.

The key idea of the proposed approach is to compute a user-ad assignment matrix that maximizes the number of clicks under the constraint of ad budgets from individual advertisers. The main computational challenge is the size of variable to be optimized: since the number of users and advertisements involved in our system are 1 billion and ten thousands, respectively, we need to estimate a matrix of billions times ten thousands. We address this challenge by converting the original optimization problem into its dual problem, in which the number of variables is reduced to only ten thousands. A distributed computing algorithm, based on the Nesterov’s method and map-reduce framework, was developed to efficiently solve the related optimization problem. We have observed that, the proposed algorithm significantly improves the effectiveness of ad presentation compared to the greedy algorithm.