Abstract: Fiverr, the talent cloud solution for businesses of all sizes, utilizes advanced AI techniques to provide value in a myriad of business use-cases, both in their core product and in its periphery. From recommendation engines, via spam and fraud detection, and all the way to marketing enablement, Fiverrs data-science team operates on all fronts to provide intelligent solutions to important business problems.
In order to make sure their investment in AI bears fruit, Fiverrs core data science team has long realized that they cannot stay focused only on research and must adopt a product-oriented approach to their work. They cultivated a culture in which data scientists take full ownership of the models performance, which means not only its performance in the test set, A/B test, or the offline environment but in its behavior and performance when running in the production environment.
A primary tool used by Fiverr for this purpose is Mona, a proactive observability platform, which allows them to collect data regarding model behavior and business outcomes, and to proactively get alerted when their AI systems underperform or misbehave before business KPIs are affected.
In this talk, Gal (Senior Data Scientist, Fiverr) and Itai (CPO, Mona) discuss how Fiverr utilizes advanced tools, both home-grown and bought, to bridge the gap between data science and business, empower data scientists to understand the behavior of their models in production and make sure their AI solutions bring the value theyre expected to deliver.
To do this, Gal and Itai will discuss concepts and principles in product-oriented data science and then ground these concepts with real-world examples from Fiverrs experience.
Some of the topics discussed are:
- The creation of a machine learning platform that abstracts productionization from the data scientists, allowing fast development cycles, as well as a shared format of training, inference, and monitoring pipelines
- Utilizing advanced monitoring and analytics to make sure the models perform as expected in the context of the business function they serve.
- How understanding your model in production helps the data science team do better research
- How taking ownership of the model in production improves the chances of a successful data science project
Bio: With over 10 years of experience (Google, AI-focused startups) with big data and as the CPO and head of customer success at Mona, the leading AI monitoring intelligence company, Itai has a unique view of the AI industry. Working closely with data science and ML teams applying dozens of solutions in over 10 industries, Itai encounters a wide variety of business use-cases, organizational structures and cultures, and technologies used in today’s AI world.