Feb
24

East 2020 Preview: Unit Cost Metrics for Customer Churn

Carl is a speaker for ODSC East this April 13-17! Be sure to check out his talk, “Fighting Customer Churn With Data,” there!

In this post I’m going to highlight one of the key takeaways from the session I’m planning for ODSC East in April: To understand and reduce customer churn (cancellations) you should use a measure of the unit cost that customers pay. If you are trained in data science, you would call this feature engineering because you are designing the input data to optimize your results. I emphasize analytics and feature engineering to help companies reduce churn because churn reducing tactics require detailed customer measurements for targeting, and a predictive model by itself has limited utility. But you can design data features so that they predict churn in a way that enables business people to understand and act to reduce churn.

Understanding Churn with Simple Customer metrics

In the ODSC session, I’m going to take some examples from a case study with a company called Versature: Versature is a provider of integrated cloud communication services. The image below illustrates the metric cohort churn analysis with simple customer metrics. In the session, I will tell you more about how to understand features relationship to churn with metric cohort analysis (for now you can read about it in this post.) These are the main points:

  • Local calls per month – This has a typical relationship to churn in the metric cohort plot: The more calls, the less churn.
  • Monthly Recurring Revenue paid by the customer per month – This one is probably not expected: The more customers pay, the less they churn. How does that make sense? If you haven’t done a lot of churn studies this may surprise you. Read on to find out why!

Note: the metric cohort figures show the cohort average metrics as a score (normalized.) Also, the churn rates are shown on a relative scale (with the bottom of the figure fixed at zero churn.)

The ODSC session also contains examples of customer behavior correlation analysis, described in this post. The scatter plot (above) shows that paying more is correlated with making more calls.  And customers who make a lot of calls churn a lot less than customers that don’t. So that explains why it looks like customers that pay more churn less – they also make more calls. That may be true but that relationship is not useful for understanding customers’ price sensitivity. Something is missing from this picture…

Customer churn and the unit cost metric

Advanced customer metrics for churn are combinations of simple customer metrics that help you understand the interaction between two behaviors. The best way of combining two metrics is by making a ratio of one metric to another. The example in the last section is a common scenario where you want to use a metric made from a ratio of two other metrics: Something which that ought to cause customers to churn (paying a lot) is correlated with something that is engaging and makes customers stay (making a lot of calls.)

If you take the ratio of the monthly cost to the monthly calls the resulting metric is the cost per call. The relationship of the cost per call metric to customer churn is shown in the picture below: The more the customer pays (per call) the more they churn. This relationship is very strong! A unit cost metric is an excellent way to segment your customers according to the value they receive.

Code for the ratiometric

Below is the SQL that I use to calculate the ratio. Literally I calculate the ratio of two other metrics. The only fancy part is the case statement to check for zeros in the denominator. In the session, I will teach you more about calculating metrics with SQL, but for now, check out my post on Churn Feature Engineering which goes over the basics of calculating Metrics with SQL.

I think that’s all I can fit in a post! To learn more details about the subject, you have to wait for the release of chapter 7 in the e-book of Fighting Churn with Data. (At the time of this writing that chapter is scheduled to be released in e-book form in February 2020…)

SQL to calculating a metric as a ratio of two other metrics:

with num_metric as (

select account_id, metric_time, metric_value as num_value

from metric m inner join metric_name n on n.metric_name_id=m.metric_name_id

and n.metric_name = 'MRR'

and metric_time between '2020-01-01' and '2020-01-31'

), den_metric as (

select account_id, metric_time, metric_value as den_value

from metric m inner join metric_name n on n.metric_name_id=m.metric_name_id

and n.metric_name = 'Local_Calls'

and metric_time between '2020-01-01' and '2020-01-31'

)

insert into metric (account_id,metric_time,metric_name_id,metric_value)

select d.account_id, d.metric_time, %new_metric_id,

case when den_value > 0

    then coalesce(num_value,0.0)/den_value

    else 0

    end as metric_value

from den_metric d  left outer join num_metric n

on n.account_id=d.account_id

and n.metric_time=d.metric_time

More on the speaker/author: Carl Gold, PhD

Currently the Chief Data Scientist at Zuora (www.zuora.com), Carl has a PhD from the California Institute of Technology and has first author publications in leading Machine Learning and Neuroscience journals. Before coming to Zuora, he spent most of his post-academic career as a quantitative analyst on Wall Street. Now a data scientist, Carl is currently writing a book about using insights from data to reduce customer churn, to be released in 2020 entitled “Fighting Churn With Data.” You can find more information at www.fight-churn-with-data.com.

Feb
24

Data Science in Manufacturing: An Overview

In the last couple of years, data science has seen an immense influx in various industrial applications across the board. Today, we can see data science applied in health care, customer service, governments, cybersecurity, mechanical, aerospace, and other industrial applications. Among these, manufacturing has gained more prominence to achieve a simple goal of Just-in-Time (JIT). In the last 100 years, manufacturing has gone through four major industrial revolutions. Currently, we are going through the fourth Industrial Revolution, where data from machines, environment, and products are being harvested to get closer to that simple goal of Just-in-Time; “Making the right products in right quantities at the right time.” One might ask why JIT is so important in manufacturing? The simple answer is to reduce the manufacturing cost and make products more affordable for everyone.

In this article, I will try to answer some of the most frequently asked questions on data science in manufacturing.

How is manufacturing using data science and its impact?

The applications of data science in manufacturing are several. To name a few: predictive maintenance, predictive quality, safety analytics, warranty analytics, plant facilities monitoring, computer vision, sales forecasting, KPI forecasting, and many more [1] as shown in Figure 1 [2].

Figure 1: Data science opportunities in manufacturing [2]

Predictive Maintenance: Machine breakdown in manufacturing is very expensive. Unplanned downtime is the single largest contributor to manufacturing overhead costs. Unplanned downtime costs businesses an average of $2 million over the last three years. In 2014 the average downtime cost per hour was $164,000. By 2016, that statistic had exploded by 59% to $260,000 per hour [3]. This has led to embracing technologies like condition-based monitoring and predictive maintenance. Sensor data from machines are monitored continuously to detect anomalies (using models such as PCA-T2, one-class SVM, autoencoders, and logistic regression), diagnose failure modes (using classification models such as SVM, random forest, decision trees, and neural networks), predict the time to failure (TTF) (using combination of techniques such as survival analysis, lagging, curve fitting and regression models) and optimal maintenance time prediction (using operations research techniques) [4] [5].

Computer Vision: Traditional computer vision systems measure the parts for tolerance to determine if the parts are acceptable or not. Detecting the quality of the parts for defects such as scuff marks, scratches, and dents are equally important. Traditionally humans were used for inspecting for such defects. Today, AI technologies such as CNN, RCNN, and Fast RCNN’s have proven to be more accurate than their human counterparts and take much less time in inspecting. Hence, significantly reducing the cost of the products [6].

Sales forecasting: Predicting future trends has always helped in optimizing the resources for profitability. This has been true in various industries, such as manufacturing, airlines, and tourism. In manufacturing, knowing the manufacturing volumes ahead of time helps in optimizing the resources such as supply chain, machine-product balancing, and workforce. Techniques ranging from linear regression models, ARIMA, lagging to more complicated models such as LSTM are being used today to optimize the resources.

Predicting quality: The quality of the products coming out of the machines are predictable. Statistical process control techniques are the most common tools that we find on the manufacturing floor that tell us if the process is in control or out of control as shown in Figure 2. Using statistical techniques such as linear regression on time and product quality would yield us a reasonable trend line. This line is then extrapolated to answer questions such as “How long do we have before we start to make bad parts?”

The above are just some of the most common and popular applications. There are still various applications that are hidden and yet to be discovered.

Figure 2: An example of X-bar chart

How big is data science in manufacturing? 

According to one estimate for the US, “The Big Data Analytics in Manufacturing Industry Market was valued at USD 904.65 million in 2019 and is expected to reach USD 4.55 billion by 2025, at a CAGR of 30.9% over the forecast period 2020 – 2025. [7]” In another estimation, “TrendForce forecasts that the size of the global market for smart manufacturing solutions will surpass US$320 billion by 2020. [8]” In another report it was stated that “The global smart manufacturing market size is estimated to reach USD 395.24 billion by 2025, registering a CAGR of 10.7% according to a new study by Grand View Research, Inc. [9]”

What are the challenges of data science in manufacturing?

There are various challenges for applying data science in manufacturing. Some of the most common ones that I have come across are as follows

Lack of subject matter expertise: Data science is a very new field. Every application in data science requires its own core set of skills. Likewise, in manufacturing, knowing the manufacturing and process terminologies, rules and regulations, business understanding, components of supply chain and industrial engineering is very vital. Lack of SME would lead to tackling the wrong set of problems, eventually leading to failed projects and, more importantly, losing trust. When someone asks me what is a manufacturing data scientist?, I show them this nice image in Figure 3.

Figure 3: Who is a manufacturing data scientist?

Reinventing the wheel: Every problem in a manufacturing environment is new, and the stakeholders are different. Deploying a standard solution is risky and, more importantly, at some point its bound to fail. Every new problem has a part of the solution that is readily available, and the remaining has to be engineered. Engineering involves developing new ML model workflows and/ writing new ML packages for the simplest case and developing a new sensor or hardware in the most complex ones. In my experience for the last couple of years, I have been on both extreme ends, and I have enjoyed it.

What tools do data scientists who work in manufacturing use?

A data scientist in manufacturing uses a combination of tools at every stage of the project lifecycle. For example:

  1. Feasibility study: Notebooks (R markdown & Jupyter), GIT and PowerPoint

“Yes! You read it right. PowerPoint is still very much necessary in any organization. BI tools are trying hard to take them over. In my experience with half a dozen BI tools, PowerPoint still stands in first place in terms of storytelling.”

  1. Proof of concept: R, Python, SQL, PostgreSQL, MinIO, and GIT
  2. Scale-up: Kubernetes, Docker, and GIT pipelines

Conclusion:

Currently, applying data science in manufacturing is very new. New applications are being discovered every day, and various solutions are invented constantly. In many manufacturing projects (capital investments), ROI is realized over the years (5 – 7 years). Most successfully deployed data science projects have their ROI in less than a year. This makes them very appreciable. Data science is just one of many tools that manufacturing industries are currently using to achieve their JIT goal. As a manufacturing data scientist, some of my recommendations are to spend enough time to understand the problem statement, a target for the low hanging fruit, get those early wins, and build trust in the organization.

I will be at ODSC East 2020, presenting “Predictive Maintenance: Zero to Deployment in Manufacturing.” Do stop by to learn more about our journey in deploying predictive maintenance in the production environment. 

References

[1] ActiveWizards, “Top 8 Data Science Use Cases in Manufacturing,” [Online]. Available: https://activewizards.com/blog/top-8-data-science-use-cases-in-manufacturing/.
[2] IIoT World, “iiot-world.com,” [Online]. Available: https://iiot-world.com/connected-industry/what-data-science-actually-means-to-manufacturing/. [Accessed 02 10 2020].
[3] Swift Systems, “Swift Systems,” [Online]. Available: https://swiftsystems.com/guides-tips/calculate-true-cost-downtime/.
[4] N. a. T. G. Amruthnath, “Fault class prediction in unsupervised learning using model-based clustering approach.,” in In 2018 International Conference on Information and Computer Technologies (ICICT), Chicago, 2018.
[5] N. a. T. G. Amruthnath, “A research study on unsupervised machine learning algorithms for early fault detection in predictive maintenance.,” in In 2018 5th International Conference on Industrial Engineering and Applications (ICIEA), 2018.
[6] T. Y. C. M. Q. a. H. S. Wang, “A fast and robust convolutional neural network-based defect detection model in product quality control.,” The International Journal of Advanced Manufacturing Technology, vol. 94, no. 9-12, pp. 3465-3471, 2018.
[7] “Big Data Analytics in Manufacturing Industry Market – Growth, Trends, and Forecast (2020 – 2025),” Mordor Intelligence, 2020.
[8] Trendforce, “TrendForce Forecasts Size of Global Market for Smart Manufacturing Solutions to Top US$320 Billion by 2020; Product Development Favors Integrated Solutions,” 2017.
[9] Grand View Research. Inc, “Smart Manufacturing Market Size Worth $395.24 Billion By 2025,” 2019.

About the AuthorSpeaker:

Dr. Nagdev Amruthnath is a Data Scientist III at DENSO and has experience working in manufacturing and full-stack data science deployment experience. He specializes in solving manufacturing problems related operations, quality and supply chain using ML and DL. He has published various articles in international journals and conferences along with various R packages on GitHub. Nagdev graduated with a Ph.D. in Industrial Engineering from Western Michigan University.

LinkedIn: linkedin.com/in/nagdevamruthnath/

GitHub: github.com/nagdevAmruthnath

Website: iamnagdev.wordpress.com

Sep
25

Data Valuation – What is Your Data Worth and How do You Value it?

Editor’s Note: Come to  ODSC Europe 2019 for the talk “Data Valuation — Valuing the World’s Greatest Asset.“

Some people talk about data as the new oil, but this is too simplistic. Oil is a commodity–to be bought and sold. Data is an asset, an asset that grows in value through use. A single person’s data is not very valuable. Combining the data generated by thousands of people is a completely different story. Coupling that with data generated in different situations, combining datasets, creates new insights and value for different actors and stakeholders.

[Related Article: Avoiding Dangerous Judgment Errors in Using Data to Make Important Leadership Decisions]

If data is so valuable, then why do so few firms value it? Some seek to work out a price for their data. They try to understand what the market will pay for it. But the value in data does not always lie in its sale. Take Amazon and Alibaba, for example. Both firms are seeking to optimize a marketplace; to connect customers with a demand, or to organizations that can supply. Individual consumers provide data on what they want and need. Amazon and Alibaba use this to match the consumers to providers with the right products and services. They also aggregate the data to provide insights into market trends and shopping patterns. They don’t sell data, at least not as their primary service, but they do use it extensively to optimize their processes.

This is not about selling data–it is realizing that data is the lifeblood of an organization.

The value of data to Amazon and Alibaba is not captured in a pricing approach. Yes, their data may be valuable to third parties, but it is more valuable to the firms themselves as they seek to optimize their operation. Indeed, without data, they could not continue to operate.

So, we can’t think of data value as simply the price others are willing to pay. We have to think more widely and in doing so we have to create methodologies for data valuation. This distinction, from data value to data valuation, is critically important. Data value is a property. Your data has a certain value and you need to understand what this is in order to make appropriate investment decisions to support your data. To understand the value of your data you need a methodology for data valuation. You need a way of working out what the actual value of your data is.

One way to think about this is to ask the question, why would I want to put a value on my data? Think of data as an asset; organizations deploy assets to create value for different stakeholders. They also invest in assets to make them fit for purpose and, at any point in time, they have to consider which assets are worth investing in. You can think of this as the data value/data valuation cycle. You have to assess and understand what data you have (data assessment). You have to put a value on this data (data valuation) so your people recognize the value of data, treat it with respect inside your organization and work out how to make it more valuable. From this, you then have to invest (data investment) to make sure your data is fit for purpose. You have to ensure you have good governance in place, an appropriate data strategy, standards, systems and procedures to ensure you achieve good data quality.

Once you have good data you can start to use it (data utilization). This is centered around identifying how you can use data to create value for you and your stakeholders. This may be through better operations. It may be through more efficient delivery of products and services. It may be by using the data to generate new and meaningful insights that are, in themselves, valuable. Then you can create data value–by acting upon these insights. Finally, you have to review what you have learnt (data reflection), asking yourself, what have we learned from applying our data? How could we do this better in the future? Are there new and different datasets we need to access?

Data valuation

This cycle is endless–you oscillate between the data valuation and data value phases. But the start of the cycle is data valuation, something that has been spoken about for many years, but no one has been able to properly implement and that is partly why so many data initiatives fail. This is until now. Anmut have completed the development of a rigorous methodology for valuing data and worked with a large public sector organization to implement it.

[Related Article: The Mechanics and Business Value of Speech Technologies]

Come to  ODSC Europe on the 21st November to see how Anmut did it at the talk “Data Valuation — Valuing the World’s Greatest Asset.” You will learn how you can transform the way data is seen through your organization, from a systemic disadvantage into a competitive advantage! Find out more at www.anmut.co.uk.

Originally posted on OpenDataScience.com

Sep
25

Transaction Data Enrichment, an Opportunity for Financial Wellness

Editor’s Note: See Pramod’s talk “Transaction Data Enrichment and Alternative Data: An Opportunity for Business Growth and Risk Mitigation” at ODSC West 2019. 

In a recent financial wellness survey of American adults, 58% of respondents said they did not have the financial freedom to enjoy life. 48% said they were living paycheck to paycheck. Less than half said they were confident in their ability to absorb an unforeseen financial emergency.

[Related Article: How AI will Disrupt the Financial Sector]

Needless to say, Americans have room to grow when it comes to financial wellness. And banks can help.

By enriching a consumer’s transaction data, banks can demystify data, making it easier for consumers to understand their own spending and saving patterns. But transaction information is often highly abbreviated, with ambiguous conventions that make it difficult for consumers to recognize their own transactions. It also lacks categorization, which challenges financial institutions and banks to organize, analyze, and effectively use data.

Transaction Data Enrichment (TDE) can empower consumers to receive personalized financial advice, from their financial institution. Envestnet|Yodlee’s TDE machine learning engine provides insights based on its findings from millions of transactions. It allows for the delivery of vital contextual information to customers. TDE’s accuracy and depth leads to an evolved user experience and the ability to make better financial recommendations for financial wellness.

TDE powered by artificial intelligence can scan the consumers’ past history and account balances to glean new information. Artificial intelligence/machine learning enables speedy and accurate analysis based on predicted transactional categories. Its self-learning capabilities allow constant analysis of billions of bank transactions. The use of structural modeling techniques also gives the required flexibility. Deep learning models and automated processes also learn while they perform. It would otherwise require exhaustive effort to go through and interpret the transactions manually.

New age financial wellness tools supported by AI-based enrichment mark a change in the way financial advice is delivered to consumers. Unlike earlier personal financial management tools, new generation tools give consumers actionable guidance needed to improve their financial health. For example, consumers may find it difficult to track transactions based on merchant identities based on how their names are displayed on their online banking platform. A simple example is Domino’s Pizza which could show up in transactions as either “pizzadom” or ‘domino.” TDE can help organize these merchant variations into a single “Domino’s Pizza” expense category. Such insights will help consumers understand if one must cut down their monthly expense on eating out. App-based systems will also reduce customer support calls to the bank.

[Related Article: Going to the Bank: Using Deep Learning For Banking and the Financial Industry]

The financial industry is witnessing a huge upswing in artificial intelligence technology to support the financial wellness of consumers. AI-based tools can help consumers deal with their day-to-day financial concerns, long-term budgeting or savings, and provide the tools necessary for a better financial future.

Editor’s Note: See Pramod’s talk “Transaction Data Enrichment and Alternative Data: An Opportunity for Business Growth and Risk Mitigation” at ODSC West 2019. 

Originally posted on OpenDataScience.com

Sep
25

Trust, Control, and Personalization Through Human-Centric AI

Editor’s Note: Vincent will be speaking at ODSC Europe 2019, attend to see his talk, “Ethical AI: A Practical Guideline for Data Scientists.“

Our virtual lives lie in the hands of algorithms that govern what we see and don’t see, how we perceive the world and which life choices we make. Artificial intelligence decides which movies are of interest to you, how your social media feeds should look like, and which advertisements have the highest likelihood of convincing you. These algorithms are either controlled by corporations or by governments, each of which tends to have goals that differ from the individual’s objectives.

In this article, we dive into the world of human-centric AI, leading to a new era where the individual not only controls the data, but also steers the algorithms to ensure fairness, privacy, and trust. Breaking free from filter bubbles and detrimental echo chambers that skew the individual’s worldview allows the user to truly benefit from today’s AI revolution.

While the devil is in the implementation and many open questions still remain, the main purpose of this think piece is to spark a discussion and lay out a vision of how AI can be employed in a human-centric way.

HOW MACHINES LEARN: THE VALUE OF MORALITY AND ETHICS

Training an AI model requires two components: training data and a reward or cost function. During the training phase, a model learns how to obtain the maximum reward, or minimum cost, by acting on the data it is fed.

Training cycle of a machine learning model
Figure 1: Training cycle of a machine learning model

For example, in the field of face recognition, the reward can be defined to be proportional to the number of faces correctly classified by the model. After several training iterations, the optimizer then fine-tunes the internals of the AI algorithm to maximize its accuracy.

So how does this compare to the human learning process? As opposed to algorithms, humans are not completely rational. We tend to learn, and teach, in a much more complicated environment, where our reward function is influenced heavily by laws, ethics, morality, and societal norms.

The fact that these principles are unknown to an AI algorithm, can cause it to completely disregard important factors such as ‘fairness’ during training, as long as its reward is maximized. In the field of reinforcement learning and game-theory, for example, it has been shown that AI models often learn to cheat or cut corners, in order to reach the highest reward as fast as possible.

A rational agent should choose the action that maximizes the agent’s expected utility

—RUSSELL AND NORVIG
ARTIFICIAL INTELLIGENCE: A MODERN APPROACH

Extrapolating this to real-world applications of AI poses some interesting questions. What happens when an AI algorithm is taught to show content to its users if we simply use ‘click-through rate’ as a reward? Will the algorithm show content that benefits the user in the long run, or will it simply learn to select content that elicits the most reactions, optimizes for likes, and provokes emotional responses?

Indeed, this is exactly the danger of modern recommendation engines used in social media platforms, search engines, and entertainment frameworks. Deep neural networks implicitly learn to model underlying cognitive factors that cause users to engage with content. Driven by its ill-defined rewards function, the algorithm focuses on short-term gain without regard to the user’s long-term goals or its implications on the user’s worldview.

If based on cognitive profiling and psychometrics, this is often referred to as persuasion profiling, where the main goal is to trigger the user to behave in a specific way, thereby placing the needs of the strategist above the needs of the user.

TOWARDS TRUSTWORTHY ARTIFICIAL INTELLIGENCE

The premise of human-centric AI is a strong conviction that artificial intelligence should serve humanity and the common good while enforcing fundamental rights such as privacy, equality, fairness and democracy.

However, people can only reap the full benefits of AI if they can confidently trust that the algorithm was taught to serve their interests as opposed to those of a third-party institution or corporation. For example, instead of optimizing a recommendation engine to maximize the number of impressions, one might choose to maximize the quality of impressions.

The reward to be optimized should be aligned with the user’s goals. For example, a user might be highly likely to click on a message that promotes fast food, when presented. Yet, a human-centric AI engine should take into account the user’s goals related to weight-loss or health before recommending that message to the user merely for the sake of increasing the click-through rate.

Thus, in an ethics-by-design framework, it is the algorithm creator’s responsibility to design the AI to only engage if an overlap can be found between the individual’s personal goals, the company goals, and potential third-party (e.g. brands, sponsors or clients) goals.

Trustworthy AI is human-centric, thereby putting the user's interest first, while searching for overlap with company goals.
Figure 2: Trustworthy AI is human-centric, thereby putting the user’s interest first, while searching for overlap with company goals.

By doing so, the AI acts as a ranking and prioritization filter, not only providing and recommending the next best thing for a user, but also filtering out anything that will be counterproductive in achieving the user’s goals, e.g. unwanted noise or spam.

Making sure that the user’s goals are understood and taken into account correctly requires some fundamental changes in how data is treated and how algorithms operate. The ability to give users complete control over both data and logic depends heavily on the provisioning of technical tools related to explainability of decisions, traceability of data flows, and curation of data and digital identity on the one hand, and an algorithmic focus on intent modeling, personalization and contextualization on the other hand.

A USER-CENTRIC DATA PARADIGM

Most AI applications today result in a so-called inverse privacy situation. Personal information is inversely private if only a third party has access to it while the user does not. Inversely private information ranges from historic user-declared data which the user does not have access to anymore, to inferred and derived data that was generated by algorithms.

While inversely private data is often used by companies to improve the user’s experience, shielding it from the user makes it impossible to correct wrongly derived information, or to delete data points that the user does not deem appropriate to be shared anymore at a certain point in time.

In a user-centric data paradigm, this unjustified inaccessibility is eliminated completely by ensuring the user has access to all information known about him or her, in a convenient and interpretable manner. Moreover, data is considered fluid, such that something shared by a user at some time, can be retracted or changed at any time in the future.

Inversely private data is personal information that is only accessible by a third-party.
Figure 3: Inversely private data is personal information that is only accessible by a third-party.

This user-centric data fundament naturally leads to the concept of zero-party data. While first-party data represents data that is gathered through a company-initiated pull mechanism (‘ask’), zero-party data is personal information that is obtained through a user-initiated push mechanism (‘provide’) accompanied with clear restrictions on what the data can be used for.

An example of first-party data could be demographic information of a user, asked when the user subscribes to a service and used for the benefit of the company. An example of zero-party data, on the other hand, could be the user’s preferences, interests or goals, voluntarily provided by the user, with the explicit agreement that this data will only be used for the benefit of this user.

Zero-party data is personal information provided by the user with the implicit contract that the data will only be used to improve the user's experience.
Figure 4: Zero-party data is personal information provided by the user with the implicit contract that the data will only be used to improve the user’s experience.

The use of zero-party data, combined with the elimination of inversely private data, leads to bi-directional full transparency which is crucial to build a trust-relationship. Moreover, it allows the user to take the initiative to improve the algorithm by solving algorithmic bias caused by data partiality.

However, to move towards a fully trusted relationship, it is not enough to just give the user control over his or her data. It is as important to provide the user with full historic traceability of who accessed the data and for which purpose.

Imagine the case where a user provides a data point to company A, which in turn derives analytics that are used by company B, after which companies C and D uses it to personalize their respective services. In a user-centric data agreement, company A ensures that each of those transactions is logged and visible to the user.

Full traceability of data usage, consumer identification and purpose enforce trust and transparency.
Figure 5: Full traceability of data usage, consumer identification, and purpose enforce trust and transparency.

The user might then change, augment or complete his or her data, or even restrict future access for specific purposes or by specific companies through a data configuration portal from company A. Technologies such as linked hash chains or blockchain-based distributed ledgers can provide immutability and traceability of access logs to prevent fraud while guaranteeing privacy.

The user-centric data paradigm, therefore, transforms the traditional one-direction funnel into a trusted and mutually beneficial partnership between user and brand through transparency, ownership and a change of control.

THE ALGORITHM OF YOU

The user-centric data paradigm enforces trustworthiness but does not guarantee the human-centric application of AI in itself. Human-centric AI is based on the idea that AI should be used to help individuals reach their goals as opposed to using company business metrics as an optimizable reward.

To make this possible, the algorithm needs to be informed about the user’s goals in some way. Although long-term goals and mid-term goals could be provided by the user explicitly as zero-party data, short-term goals, i.e. intent, are often too short-lived.

Thus, a crucial component of human-centric AI, is intent modeling: Using AI to estimate the user’s short-term goals in a personalized and contextualized manner. For example, if a user is currently in a car, then whether his or her short-term intent is ‘commuting to work’ or ‘dropping of the kids at day-care’ can make a huge difference on how a music recommender should operate.

The combination of zero-party data with intent modeling and contextualization leads to a virtual persona reflecting the user’s digital identity. It’s exactly this idea that was coined the algorithm of you by Fatemeh Khatibloo, Principal Analyst at Forrester in 2018: Modeling a personal digital twin within the constrained framework defined by the user-centric data paradigm.

The 'Algorithm of You' refers to a highly personalized and contextualized algorithm that operates within the user-centric data paradigm, and that is under complete control of the user.
Figure 6: The ‘Algorithm of You’ refers to a highly personalized and contextualized algorithm that operates within the user-centric data paradigm, and that is under complete control of the user.

Having a personal digital twin allows the user to enforce curation of identity by configuring how the digital twin behaves when interrogated by companies. In real-life, we rarely hand out the same information to strangers as we do to friends. Similarly, parts of a user’s digital twin should only be exposed to trusted brands or for specific purposes, while others might be exposed to everyone.

Trustworthiness is thus guaranteed by enabling the user to steer the behavior of the algorithms by changing, removing, augmenting or hiding parts of the digital twin’s DNA, thereby effectively providing a curated version of his or her identity to different institutions or for different purposes.

A key consideration that is necessary for the user to be able to configure his or her digital twin, is automated explainability and interpretability of the AI’s decision making. By explaining the user why certain actions are taken or which data points lead to specific recommendations, he or she is able to judge whether or not the data is used in an appropriate manner that contributes to the user’s goals.

YOUR PERSONAL DIGITAL TWIN

The concept of a personal digital twin embodies the idea of capturing the user’s mindset, lifestyle, and intent in a digital manner, such that it can be interrogated by an AI model. Although the user-centric data paradigm enables the user to curate, edit and enrich this digital identity, it would be cumbersome for the user to have to constantly refine this profile manually. A baseline user profile therefore needs to be detected and updated automatically and in real-time.

To obtain his or her digital identity in an automated manner, a user can enable the algorithm of you to observe his or her day-to-day behavior continuously, thereby extracting meaningful patterns, routines, profile and personality insights, and predictions. Automatically detecting low-level user motion (e.g. walked three steps) and turning those activities into high-level contextual insights (e.g. walking the dog while being late for work) requires advanced AI capabilities that act as multi-resolution pattern detectors on the underlying data. But where would a user find data that is fine-grained enough to unlock such capabilities?

We all run around with a smart phone almost 24/7, to the extent that a smart phone could be considered an extension of our body, albeit a physically disconnected one. So how can we leverage the power of smart phone sensors to bridge that physical disconnect and thus track motion, movement and orientation? Our brain uses a process called proprioception to obtain a continuous sense of self-movement and body position based on sensory activations in muscles, tendons and joints which are integrated by the central nervous system.

Can we emulate this process virtually, in order to give our personal digital twin the ability to sense motion, orientation and position? Smart phones are packed with high resolution proprioceptive sensors such as accelerometers, measuring every small vibration, and gyroscopes which track orientation changes. Integrating those sensors, in a process called sensor fusion allows an AI model to estimate and detect user activities.

Proprioceptive sensor fusion allows the AI model to obtain a sense of self-movement and motion of the smart phone which acts as a digital proxy for the human body.
Figure 7: Proprioceptive sensor fusion allows the AI model to obtain a sense of self-movement and motion of the smart phone which acts as a digital proxy for the human body.

Examples of low-level activities, also called events, that could be recognized by the AI model are transport modes when the user is on the move (e.g. walking, biking, driving, train, etc.), venue types when the user is stationary (e.g. home, work, shop, etc.), and fine-grained driving style insights when the user is driving or gait analysis when the user is walking. A temporal sequence of those activities makes up a user’s event timeline:

The ordered sequence of detected events makes up a user's behavioral timeline.

Figure 8: The ordered sequence of detected events makes up a user’s behavioral timeline.

This event timeline explains what the user is doing, but doesn’t reveal intent yet, as needed by the algorithm of you in the human-centric AI framework. However, by detecting patterns and routines in this timeline, a predictive model can start predicting the user’s next actions, which in turn leads to explanation of intent. For example, if the user was observed to be at home, is currently in a car, and is predicted to arrive at work soon, then the intent of this particular car trip is ‘commute to work’.

Intents, such as commute, kids drop-off, sport routine or business trip, can therefore be considered a short-term aggregation of the underlying activity timeline, enriched with detected patterns and anomalies. This is what Sentiance refers to as moments which are designed to explain why the user is doing something instead of only what the user is doing.

Aggregating events (‘what’), moments (‘why’) and predictions (‘when’) over even longer periods of time then allows the AI model to put those events and intents into the perspective of who the user is. What is the user’s cognitive state? What are his or her long-term patterns? Is this user working out today by chance, or is he or she a sportive person in general?

The long-term view on the user’s personality and traits is what we call segment detection. Segments tend to explain who the user is, apart from just what he or she is doing and why.

Aggregating behavioral events over different time-scales allows the algorithm of you to model contextualize and understand the user's actions, intents and goals.
Figure 9: Aggregating behavioral events over different time-scales allows the algorithm of you to model contextualize and understand the user’s actions, intents and goals.

Apart from proprioceptive sensors, the user can request the AI model to ingest zero-party data, or specific data points such as in-app behavior, that can further deepen and strengthen the user’s digital identity. This allows the personal digital twin to bridge the gap between on-line and off-line behavior.

This type of contextualization results in a digital identity that the user can use to enable highly personalized experiences, recommendations and services by configuring algorithms to interact with his or her digital twin. The fact that all of this happens within the user-centric data paradigm is a crucial component of the algorithm of you, as it guarantees correctness of the inferred insights, privacy towards the user who acts as data owner, and trust between all parties involved.

CONCLUSION

Human-centric AI can be achieved by creating a personal digital twin that represents a virtual copy of the individual’s behavior, goals and preferences. Contextualization and intent modeling are crucial to avoid bothering the user too often. At the same time, the individual should be in control of his or her data, and a trust relation should be built by guaranteeing traceability of data access, explainability of the AI’s decisions, and by providing the user with a convenient way to view, edit, control and curate this data.

The algorithm of you builds on the idea that the individual directly controls the inner workings of the algorithms by having the ability to change or enrich his or her data through a data configuration portal. Models should be optimized based on the individual’s goals, while searching for overlap with company or third-party goals.

In this framework, the healthy tension between company goals and user goals is not a zero-sum game. If AI is used in a trustworthy and user-centric manner, the relationship between users and brands completely changes. Suspicion becomes curiosity, persuasion becomes collaboration, and brands and users unite in fandoms centered around trust.

Originally Posted Here

Editor’s Note: Vincent will be speaking at ODSC Europe 2019, attend to see his talk, “Ethical AI: A Practical Guideline for Data Scientists.”

Copyright ODSC 2019. All Rights Reserved