Industry Day Schedule

Abstract: A significant portion of user requests in Alexa is about task completion, in particular, fulfilled by taking actions on entities such as listening to music and ordering food from restaurants. With voice interface, users interact with Alexa via natural languages. In this talk, we show an end-to-end system from understanding user’s utterances to invoking service providers to take actions. This includes: (1) organizing entities and service providers to enable actions (knowledge); (2) generating a semantic representing of user utterances in terms of intents and entity slots (interpretation); (3) resolving intents and entities to canonical values (resolution); (4) selecting and ranking service providers and routing user requests (action). On top of this technical stack, we build self service tools to empower both first-party and third-party developers to ingest data/knowledge and build domain-specific ML models, with interoperability and composability support to enable cross-domain complex tasks via dialogues. At the end, we will also discuss the remaining challenges in our system, such as complex user utterances with multiple intents and nested entity slots, entity category search and recommendations, and spam detection.

Bio: Xiaodong Fan is the director of applied science in Alexa AI, where he leads the team to work on ontology, entity linking and resolution, and catalogue ingestion. Previously, he was the Partner Group Engineering Manager in Bing. He has worked on image and video search, web graph analysis, web index selection and static ranking, spam and junk detection, knowledge graph, entity and local search, and personalized content recommendation. Xiaodong holds a PhD in computer vision from Johns Hopkins University.

Abstract: The annual Internet advertising revenue in China skyrocketed to $42 billion in 2017. Improvement on efficiency of ADs auction system is likely to benefits both advertisers and the platform. Founded in 2000, Baidu is the largest web search engine in China, and one of the biggest internet advertising service provider. Fengchao, Baidu’s online ads distribution system, processes more than 2 billion auction requests each day, generating the majority of the company's revenue. Designing&Implementing new algorithms for a mature and major system like Baidu Fengchao is very challenging. In this talk, I will give an overview of the research opportunities and challenges for the Internet ads auction system, followed by our field experience of how we bring reinforcement learning and multi-unit mechanisms into large scale implementation.

Bio: Ruohan Qian (Jeff) is the head of Baidu's mechanism design team. His is responsible for the design and implementation of Baidu’s search ads auction system. His research areas include mechanism design, recommendation system and reinforcement learning. He earned his master’s degree from University of Science and Technology of China (USTC).

Abstract: Airbnb is a two-sided marketplace that brings together travelers and hosts to build a world where every person can belong anywhere. To achieve that mission, the Data Science Team contributes to all parts of the business and product, from informing hosts to optimally set prices to routing customer issues to the right customer experience agent. Many of these problems are instances of applied research, and data scientists must consider how to balance long term outcomes with two different but related constraints: 1) creating impact in the near term and 2) achieving the same impact in less time and/or with fewer person-hours. In this talk, we share how we deal with these two challenges through the case study of characterizing and classifying our highly diverse inventory. This is a prototypical applied research problem where scope is large and impact can extend to multiple parts of the business but carries substantial risk of “failure” depending on the execution. We will discuss 1) the importance of order of operations -- how our choice of milestones around simple use cases (e.g. hotel vs not a hotel) satisfied business needs and maintained a path for the long term, 2) the importance of foundational investments, such as well designed ontologies and quality standards for labeling data, and how initial underinvestment led to poor results and wasted work, and 3) how off-the-shelf, tree-based models gave us competitive results with minimal effort and provided cover to invest in more sophisticated solutions. Along the way, we hope to share interesting challenges specific to operating a global marketplace of homes including the heterogeneity of user generated data and regional differences in the interpretation of accommodation types.

Bio: Gary Tang is a data scientist at Airbnb and leads a team of machine learning engineers focused on characterizing and classifying listings in the Airbnb marketplace. His past research includes statistical estimation in high dimensions and uncertainty quantification of computational fluid dynamics simulations. He has a PhD from Stanford University.

Abstract: Millions of users browse Walmart.com each day with varying levels of intent. Many of them end up making a purchase in the same session and most, well, do not. Display retargeting channels, with ads over open web and your favourite social media sites, are then used to reach out to the potential customers with relevant content. The ad serving comes at a cost and optimizing these costs becomes especially important given the huge scale. Predicting a user’s purchase (or click) propensity and bidding appropriately is crucial for reaching out to the right user with the right content and at the right time. We discuss how we, at WalmartLabs, build the user propensity prediction models to efficiently bid for ad impressions. We start from ground zero - understanding data nuances and formulating the problem. We delve into the finer aspects of offline data crunching and building models and pipelines on top of petabytes of user data. We further elaborate the critical stage of deploying models into the real world, where the model scores are just not enough! We also discuss the effect of multiple user touchpoints on these models and how ‘robust’ algorithms come to the rescue.

Bio: Priyanka Bhatt is passionate about building intelligent machines, and has worked at WalmartLabs for the past 4.5 years. Experienced in building ML models for Bidding and Recommendations in the computational advertising domain, Priyanka obtained an ME degree from IISc Bangalore with specialization in Game Theory under Prof. Y. Narahari, and has publications in a range of Game Theory and ML conferences, including AAAI'15 and WWW'18.

Abstract: While reinforcement learning (RL) has achieved impressive advances in games and robotics, it has not been widely adopted in recommender systems. Framing recommendation as an RL problem offers new perspectives, but also faces significant challenges in practice. Industrial recommender systems deal with extremely large action spaces – many millions of items to recommend and complex user state spaces -- billions of users, who are unique at any point in time. In this talk, I will discuss our work on scaling up a policy-gradient-based algorithm, i.e. REINFORCE to a production recommender system at Youtube. We proposed algorithms to address data biases when deriving policy updates from logged implicit feedback. I will also discuss some follow up work and outstanding research questions in applying RL, in particular off-policy optimization in recommender systems.

Bio: Minmin Chen is a Research Scientist in Google Brain. Her main research interests are in machine learning, currently focusing on reinforcement learning for recommendation systems and sequence modeling. Before that, she was a research scientist at Criteo Lab, building computational models for online advertising, and Amazon, working on the Amazon Go project. She did her PhD study at Washington University in St. Louis on representation learning and domain adaptation. She published over 20 papers at top conferences in machine learning such as NIPS, ICML, ICLR and AISTATs.

Abstract: The problem of building a recommender system from implicit feedback is well studied and has many proposed solutions, from BPR to VAE-CF. But in real world applications it may face significant constraints: billions of products and hundreds of millions of users, biased data sampling, user cold-start problems, and more. No state-of-the-art model is able to solve all of these, and we needed to implement something reliable and working in relatively short time. In this talk we’ll walk you through our explored options and how we managed to solve in a “good enough” way most of these challenging problems.

Bio: Ivan Lobov is a self-taught researcher and engineer, the author of billion-scale distributed randomized SVD implementation in Spark. In Criteo Ivan works on product similarities and personalized recommendations.