Data Science Archives | Open Data Science Conference

Jun

Overcoming the Cold Start Problem: How to Make New Tasks Tractable

I woke up this morning (somewhat unsurprisingly). After poking my head out of the covers and feeling a cold chill in the air, I made the executive decision not to proceed with this course of action and promptly withdrew back under the covers. Ten minutes later, my alarm decides it is time to snooze no more and kindly invites me to follow suit. Begrudgingly I got up; another winter morning plagued by the Cold Start Problem.

In the same way owners look like their dogs, Machine Learning Scientists think like their AI. At Tractable, our AI for accident and disaster recovery suffers from the Cold Start Problem too, albeit all year round. Imagine you have an AI that is, say, really good at spotting scratches on Toyota Priuses, because for some reason you chanced upon a large, labelled dataset of Toyota Priuses that may or may not be damaged. Once the AI has trained by looking at all these images, it becomes a pro at knowing when a Prius is damaged and when it’s not.

Now, imagine you want to scale your AI to detect scratches on Lamborghini Aventadors instead. But you don’t have a large, labelled dataset of Lamborghini Aventadors. Your AI can’t really make an accurate judgement, simply because it doesn’t know what a Lamborghini Aventador is meant to look like, scratched or not. The AI, like the researcher who trained it, has trouble getting going when it’s cold, and needs to warm up to function smoothly.

Tractable’s AI is in use for visual car damage appraisal in thirteen countries over three continents. Every time we enter a new country,

the cars look different and
the way those cars are repaired is also different.

When we enter a new country, with new car types and repair standards, we often start cold. To get going, we start with our universal AI models and customize them as we get more and more data. But thanks to (1) and (2), this presents a whole set of challenges! It’s like trying to warm up your room when your furnace is broken!

In Machine Learning, (1) and (2) are types of domain shifts and they make the cold start problem even more challenging:

Input shift: This happens when the inputs —in our case, images— that our AI model sees look different. There’s a wide spectrum covering what we would call a car and they might look very different.
Conditional shift: This happens because the repair decisions, which are conditional on the damage to the car, are different around the world. To put that another way – if you have the exact same accident in Thailand, the UK and Poland, the car repairs will be different in each country. Our AI model can’t make the right decision unless it knows about these specifics.
Output shift: The happens when you use the outputs of one domain for another. For example, in our case, the distribution of our labels—repair or don’t repair—would be very different for different countries. An AI model trained on Japanese data might be inclined to repair everything just because that’s what it has mainly seen from the data, even when used in other countries, where the methodology is very different.

When moving an AI model from one country to another, it needs to adapt to all these three types of shifts. Similar to how a Polish driver can’t drive easily in Japan and needs to adapt to the new environment and rules to drive safely, we need to adapt our AI when moving from one country to another! So the next question is how can we adapt? Can our AI trained for Japanese Adjusters write a repair estimate in Poland?

Tractable is working with the R&D Team at Georgian to expand our ability to handle domain shift. Our AI is data-hungry, and collecting enough of the right quality of data to enter a new country would be very time-consuming and expensive, so it’s critical that we can make the most of our existing dataset of cars from around the world. To address the three types of domain shift we encountered, we used techniques from two different areas of research: Ensemble Learning and Domain Adaptation.

1. Ensemble learning is the AI equivalent of getting a group of experts in a room, and putting their heads together to try and solve a difficult problem.

Many of us make our important decisions in life based on the opinions of others around us simply because often a decision made by a group of individuals results in a better outcome than a decision made by any one of them alone! Similarly, if we combine a diverse set of AI models (often called weak learners) the resulting model is more stable and performs better. This is referred to as Ensemble Learning. Although there are several Ensemble Learning methods, the most common are:

Bagging: In bagging, the weak learners are trained independently from each other in parallel and are then combined by following some kind of deterministic averaging process.
Boosting: in boosting, the weak learners are trained sequentially in a very adaptative way—meaning a weak learner is trained to focus on the misclassified examples of the previous weak learners—and then they are combined using a deterministic strategy.
Stacking: Similar to bagging, the weak learners are trained in parallel and independently from each other, but unlike the previous two methods, the weak learners are combined by training a meta-model on top of them instead of using a deterministic strategy.

2. Domain Adaptation refers to a whole host of techniques designed to understand the similarities and differences between domains, so that we can control how much our AI can generalize or specialize.

As described previously, a domain shift is a change in the data distribution between an algorithm’s training data, and the data it encounters when deployed. Machine learning algorithms often adapt poorly to domain shifts. In Domain Adaptation, two common approaches are Instance-based and parameter-based methods.

a. Instance-based methods address domain shifts by weighting the training samples.
b. Parameter-based methods address domain shifts by adjusting the parameters of the models. Parameter-based methods are the most popular methods for deep learning models.

Through a combination of ensemble learning and domain adaptation techniques, Tractable can leverage its proprietary datasets to the fullest and expand its AI models to new markets quickly and efficiently. Our global AI can achieve a 20%+ performance boost over non-adapted models. We’ve been able to achieve expert-level AI, every time, using 10x less data than training from scratch. Perhaps I’ll set my thermostat to switch on the heating before I wake up tomorrow. Starting warm seems to work well for AI, so maybe it’ll work well for me too.

Editor’s note: Azin and Franziska are speakers for ODSC Europe 2021. Check out their talk, “Overcoming the Cold Start Problem: How to Make New Tasks Tractable,” there!

About the Authors/ODSC Europe 2021 Speakers on the Cold Start Problem:

Azin Asgarian is currently an applied research scientist on Georgian’s R&D team where she works with Georgian’s portfolio companies to help adopt applied research techniques to overcome business challenges. Azin holds a Master of Science in Computer Science from the University of Toronto and a Bachelor of Computer Science from the University of Tehran. Prior to joining Georgian, Azin was a research assistant at the University of Toronto and University Health Network (UHN) where she was working on the intersection of Machine Learning, Transfer Learning, and Computer Vision.

Franziska Kirschner is the Research and Product Lead of Car Inspection at Tractable. Her team uses machine learning to automate car damage appraisal across a range of applications. Her research interests include domain adaptation, and multitask- and multi-instance learning. In a previous life, she did a PhD in Physics at the University of Oxford. In her spare time, she enjoys cooking and making bad puns.

May

Making Explainability Work in Practice

Complex ‘black box’ models are becoming more and more prevalent in industries involving high-stakes decisions (such as finance, healthcare, insurance). As machine learning algorithms take a prominent role in our daily lives, explaining their decision will only grow in importance via explainability.

By now there is enough written about why we may want to explain the decisions and behavior of machine learning models. Model debugging, bias discovery, and increased social trust and acceptance are some of the important and often cited reasons.

What is machine learning explainability?

Before we go into further detail, let’s define explainability. I need to acknowledge that there is no consensus on what machine learning explainability is but the definition I tend to stick to is that explainability (XAI) are:

“Methods and models that make the behaviour and predictions of a machine learning model understandable to humans.”

Source: https://christophm.github.io/interpretable-ml-book/terminology.html

Types of explainability: What can be explained?

What type of explanations can we provide? We can differentiate between:

Intrinsic and post-hoc methods: Intrinsic methods relate to restricting the complexity of the model and/or features before the model training; Post-hoc methods apply an explainability technique after the model training.
Model-specific and model-agnostic: Model-specific methods focus on explaining the behavior and decision or a single type of algorithm, whereas model agnostic methods work with any type of a model
Local and global: Local methods explain the decision of the model for each instance in the data whereas global methods explain the overall model behavior

Figure 1 provides a succinct taxonomy of the most common model-agnostic methods. There are various algorithms to explain the data and/or the model. Some are visual methods, and we have the abovementioned distinction between global and local methods. Don’t be overwhelmed if some of the methods are new to you. The purpose of this flowchart is not to introduce you to a new XAI method but to help you understand how they all fit together. This taxonomy could further help facilitate conversations between various stakeholders involved in a use case. But sadly, only this chart is not sufficient.

Figure 1. Taxonomy of explainability approaches

Explainability in the ML development cycle

We cannot talk about XAI in practice without discussing where XAI fits in the overall machine learning development cycle. Let’s say that we have a clearly defined business problem (I know that is wishful thinking in many scenarios). Even less believable, the input data is clearly identified and even provided to us.

The first important step is to gather the XAI requirements of all stakeholders — if explainability is relevant for the use case, of course. The set of stakeholders will differ with the use case. For example, let’s assume we are using machine learning to help decide whether we should grant a loan to a client. In this case, relevant stakeholders could be:

the loan officer or relationship manager,
the client themselves,
the technical team building and deploying the model,
the business stakeholders who accept the risk of the model,

Depending on the broader organizational setup, we could also have

model validation team, compliance, privacy office and legal teams, audit
In some countries, there are external regulators who may have their own explainability requirements.

And how can we gather the requirements of all these stakeholders? Often utilized approaches are structured and less structured interviews, surveys, and workshops.

Keep in mind that it is very likely that different stakeholders will have different XAI requirements, and thus we will end up utilizing multiple approaches.

The next step is to train our machine learning algorithm of choice (taking into account the XAI requirements, if relevant).

The third step consists of applying the approach or set of approaches. I cannot stress this enough that various XAI techniques help you answer different questions, and sadly there is not a single one that works across scenarios (despite the popularity and nice properties of Shapley values).

And let’s be optimistic (or unrealistic?) and assume the model will be deployed in production. As the model is deployed and we monitor its predictions and explanations, we should ensure we monitor and track the users’ interaction with the explanations and record any possible feedback.

Many of you may know that model monitoring and maintenance is a whole can of worms on its own. Having XAI added to it does not make it easier. While it is (relatively) easy to compare the model’s prediction versus what really happened, it’s much more difficult to validate the explanations or provide a quantitative measure for their accuracy, validity and fidelity.

Figure 2. XAI in the ML development cycle

Conclusion

XAI is definitely a very exciting and important (and a little bit hyped) field. While researchers are still developing new approaches, applying various XAI methods in practice comes with its own challenges. One thing to keep in mind: The earlier in the project you start thinking about XAI, the easier it will be to incorporate it. However, when it comes to XAI,

There is no one-size-fits-all approach, it’s a process rather than a single product.

If you are interested to know more about explainability, about how to pick an approach that addresses different questions from your stakeholders, then join my talk at ODSC East 2021. I am looking forward to that!

Violeta Misheva, PhD, has been working as a data scientist in the financial industry for the past few years. Before that, she worked in consulting, which marked the start of her data science career. Violeta developed an interest for data and algorithms during her PhD studies, and that passion has only grown over time. She is passionate about responsible machine learning, especially topics such as explainability and fairness.

May

Black Box Optimization Using Latent Action Monte Carlo Tree Search (LaMCTS)

Black box optimization has numerous applications in industries. From a/b testing to experimental designs of new ads or UI, hyper-parameter tuning in the machine learning models, or to find the optimal configuration of a system, black-box optimization tries to optimize your decision solely by exploring the problem configurations. Existing black-box solutions such as Bayesian Optimization and Evolutionary Algorithms have been widely employed to solve many challenging real-world problems. One common issue is that they could suffer the curse of dimensionality for high-dimensional optimization problems.

In this article, we introduce our Latent Action Monte Carlo Tree Search (LA-MCTS). Using LaMCTS, two teams won 3rd and 8th places in the NeurIPS2020 black-box optimization challenge (https://bbochallenge.com/leaderboard), in which over 80 teams from both academia and industry labs participated.

LA-MCTS is a new meta-level algorithm that recursively learns the space partition using MCTS. When paired with Bayesian Optimization, LA-MCTS improves the sample-efficiency by optimizing a region (rather than the entire space), which is a lot easier to optimize. A quick comparison of LA-MCTS to Bayesian Optimization can be found below.

LA-MCTS uses a tree to partition the entire search space. Each node in the tree represents a region (or a subset) of the search space, and a node uses a learned classifier (called “latent action”) to partition the node’s region into a good and bad region, represented by its left and right results. We use the Support Vector Machine (SVM) for such a classifier for partitioning the space. By recursively partitioning the entire search space into disjoint regions, a tree is formed. We apply Monte Carlo Tree Search (MCTS) to trade-off the exploration of new regions and the exploitation of the current best region to avoid trapping into a sub-optimal space partition.

Searching on a region learned from LA-MCTS v.s. the entire search space renders many benefits. First, reducing the search space can alleviate the over-exploring issue in Bayesian Optimization, especially for high dimensional problems. Second, LA-MCTS significantly reduces the complexity of acquisition optimization, as classifiers located on the nodes to the leaf can be used as the constraints to sample the candidate solutions.

LA-MCTS introduces a few hyper-parameters to tune, in addition to those in Bayesian Optimization. One important hyper-parameter is Cp that controls the amount of exploration and exploitation. A reference Cp is around 0.1 * max value of f(x), or a Cp that quickly makes the progress in the first few iterations.

For a complete list of hyper-parameters and their potential impact on the performance, please refer to our NeurIPS’20 paper (https://arxiv.org/abs/2007.00708), “Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search”, for details.

We also prepare a tutorial for you to quickly evaluate LA-MCTS v.s. vanilla Bayesian optimization, and evolutionary algorithms. Please follow the steps below.

– Download the source codes from https://github.com/facebookresearch/LaMCTS.

– Run LA-MCTS by following the codes below:
cd LA-MCTS
python run.py –func ackley –dims 10 –iterations 100

– We use the Bayesian Optimization from scikit-optimize.
pip install scikit-optimize
cd LA-MCTS-baselines/Bayesian-Optimization
python run.py –func ackley –dims 10 –iterations 100

– We also use SoTA evolutionary solver NGOPT from Nevergrad.
pip install nevergrad
cd LA-MCTS-baselines/Nevergrad
python run.py –func ackley –dims 10 –iterations 100

Please make sure you run each solvers for at least 3 times to compare the final results. Here is an example of output from LA-MCTS, which shows the tree details, the current best f(x) and the corresponding x. Please note here we intend to minimize the ackley function.

Our code can be found at https://github.com/facebookresearch/LaMCTS. In addition to the new black box solver, our release also includes the application of LA-MCTS to Neural Architecture Search, which automatically designs neural networks from scratch.

Here is a list of the main content in the current release.

Black-box optimization

Performance with baselines: 1-minute evaluations of LA-MCTS v.s. Bayesian Optimization and Evolutionary Search.
Mujoco Experiments: LA-MCTS on Mujoco environment.

Neural Architecture Search (NAS)

Evaluation on NASBench-101 : Evaluating LaNAS on NASBench-101 on your laptop without training models.
Our Searched Models, LaNet: SoTA results: • 99.03% on CIFAR-10 • 77.7% @ 240MFLOPS on ImageNet.
One/Few-shot LaNAS: Using a supernet to evaluate the model, obtaining results within a few GPU days.
Distributed LaNAS: Distributed framework for LaNAS, usable with hundreds of GPUs.
Training heuristics used: We list all tricks used in ImageNet training to reach SoTA.

We will give a 45min talk on Apr. 1 2:20 pm ET at the Open Data Science Conference (ODSC) East 2021, titled “Learning to Optimize High-Dimensional Optimization Problems.”

About the author/ODSC East 2021 speaker:

Yuandong Tian is a Research Scientist and Manager in Facebook AI Research, working on deep reinforcement learning in games and theoretical analysis of deep models. He led OpenGo, a super-human Go bot from Facebook AI. Prior to that, he was a Software Engineer/Researcher in the Google Self-driving Car team during 2013-2014. He received his PhD in the Robotics Institute, Carnegie Mellon University in 2013, Bachelors and Masters degrees of Computer Science at Shanghai Jiao Tong University. He is the recipient of the 2013 ICCV Marr Prize Honorable Mentions.

Mar

Building a Robust Data Pipeline with the “dAG Stack”: dbt, Airflow, and Great Expectations

Data quality has become a much-discussed topic in the fields of data engineering and data science, and it has become clear that ensuring data quality is absolutely crucial to avoiding a case of “garbage in – garbage out”. Apache Airflow and dbt (data build tool) are some of the most prominent tools in the open-source data engineering ecosystem, and while dbt offers some data testing capabilities, enhancing the pipeline with data validation through the open-source framework Great Expectations can add additional layers of robustness.

This blog post explains why data validation is crucial for data teams, provides a brief introduction to Great Expectations, and outlines a convenient pattern for using these tools together to build a robust data pipeline in what we’ve been calling the “dAG stack.”

Why you should test your data

“Our stakeholders would notice data issues before we did… which really eroded trust in the data and our team.” (A Great Expectations user)

Despite the growing sophistication of tools in the data engineering ecosystem in recent years, managing data quality in data pipelines is still often a struggle for data teams: On the stakeholder side, poor data quality affects the trust stakeholders have in a system, which negatively impacts the ability to make decisions based on it. Or even worse, data quality issues that go unnoticed might lead to incorrect conclusions and wasted time rectifying those problems. On the engineering side, scrambling to fix data quality problems that were noticed by downstream consumers is one of the number one issues that cost teams time and slowly erodes team productivity and morale. It’s clear that ensuring data quality through data validation is absolutely necessary for the success of any data-driven organization – but with almost limitless possibilities of what can be tested and how, what are some guiding principles for building a robust data pipeline?

An open-source data engineering stack for robust data pipelines

While there is a large number of data engineering frameworks have established themselves as leaders in the modern open-source data stack:

dbt (data build tool) is a framework that allows data teams to quickly iterate on building data transformation pipelines using templated SQL.
Apache Airflow is a workflow orchestration tool that enables users to define complex workflows as “DAGs” (directed acyclic graphs) made up of various tasks, as well as schedule and monitor execution.
Great Expectations is a python-based open-source data validation and documentation framework.

What is Great Expectations?

With Great Expectations, data teams can express what they “expect” from their data using simple assertions. Great Expectations provides support for different data backends such as flat file formats, SQL databases, Pandas dataframes and Sparks, comes with built-in notification and data documentation functionality.

For example, in order to express that a column “passenger_count” should have integer values between 1 and 6, one would write:

expect_column_values_to_be_between(
column=”passenger_count”, min_value=1, max_value=6
)

These “Expectations” are typically created in an “interactive” mode in which users connect to an existing data set as a “reference” and receive immediate feedback on whether the statement matches what’s observed in the dataset. The Expectations are then stored and can be loaded every time a new “batch” of data is being validated. In addition to the manual creation of tests, Great Expectations also offers a built-in “scaffolding” mode, which profiles a reference data set and creates Expectations based on what is observed in the data. This allows users to quickly build an entire suite of tests for a dataset without going through an entirely manual process.

Putting it all together

A typical pipeline using this “dAG” stack may look like the above image: implement initial data validation of source data (e.g. a CSV file on a web server, or a table in another database) with a Great Expectations Airflow operator, load the data using Python tasks in the Airflow DAG, validate that the data was loaded correctly with dbt or Great Expectations, then execute transformations with a dbt task, test the correctness of those transformations with dbt or Great Expectations, notify users and/or halt the pipeline if any data quality issues are detected, then load the data to a production environment with another Python task. Great Expectations also automatically generates data quality reports (“Data Docs”) at each validation run that can be shared with data stakeholders.

The code snippet below shows a simplified Airflow DAG definition with a task for each of the steps mentioned above. It uses both the Great Expectations and dbt Airflow operators, which provide convenient interfaces to the core functionality of both tools. The Great Expectations tasks make use of pre-configured Checkpoints, which bundle a data asset with an Expectation Suite.

import airflow
from airflow_dbt import DbtRunOperator
from airflow.operators.python import PythonOperator
from great_expectations_provider.operators.great_expectations import GreatExpectationsOperator

default_args = {
   "owner": "Airflow",
   "start_date": airflow.utils.dates.days_ago(1)
}

dag = airflow.DAG(
   dag_id='sample_dag',
   default_args=default_args,
   schedule_interval=None,
)

def load_source_data():
   # Implement load to database

def publish_to_prod():
   # Implement load to production database

task_validate_source_data = GreatExpectationsOperator(
   task_id='validate_source_data',
   checkpoint_name='source_data.chk',
   dag=dag
)

task_load_source_data = PythonOperator(
   task_id='load_source_data',
   python_callable=load_source_data,
   dag=dag,
)

task_validate_source_data_load = GreatExpectationsOperator(
   task_id='validate_source_data_load',
   checkpoint_name='source_data_load.chk',
   dag=dag
)

task_run_dbt_dag = DbtRunOperator(
   task_id='run_dbt_dag',
   dag=dag
)

task_validate_analytical_output = GreatExpectationsOperator(
   task_id='validate_analytical_output',
   checkpoint_name='analytical_output.chk',
   dag=dag
)

task_publish = PythonOperator(
   task_id='publish',
   python_callable=publish_to_prod,
   dag=dag
)

task_validate_source_data.set_downstream(task_load_source_data)
task_load_source_data.set_downstream(task_validate_source_data_load)
task_validate_source_data_load.set_downstream(task_run_dbt_dag)
task_run_dbt_dag.set_downstream(task_validate_analytical_output)
task_validate_analytical_output.set_downstream(task_publish)

Wrapping Up on Great Expectations

In this post, I provided a brief overview of components of a go-to stack for building a robust data pipeline, and gave a short introduction to Great Expectations, an open-source data validation framework. I will go into more detail on the types of tests that can be implemented in such a pipeline in my upcoming talk at ODSC East 2021, titled “Building a Robust Data Pipeline with the “dAG stack”: dbt, Airflow, and Great Expectations.”

About the author/ODSC East 2021 speaker:

Sam Bail is a data professional with a passion for turning high-quality data into valuable insights. Sam holds a PhD in Computer Science and has worked for several data-focused startups. In her current role as Engineering Director at Superconductive, she works on “Great Expectations”, an open-source Python library for data validation and documentation.

Mar

Fake News: Where Do We Stand in the Fight Against Disinformation

If you’re familiar with conspiracy theories regarding COVID-19 vaccines, Barack Obama’s birthplace, or Hillary Clinton’s ties to “Pizzagate”, you’ve experienced only a fraction of contemporary disinformation. With the rise of social media and its capacity for viral posts and fake news, disinformation campaigns have become a common strategy for undermining public trust and discrediting trustworthy sources.

In academia and in industry, researchers are developing new methods for finding, classifying, and understanding the nature of disinformation. But what constitutes “disinformation”? And how is it different from terms like “misinformation” and “fake news”?

Disinformation: content that is verifiably incorrect and is intended to mislead or harm

Example: “5G causes coronavirus”

Misinformation: content that is incorrect, but may have been created or shared without knowing any better

Example: Sharing outdated statistics about COVID-19 deaths

Current technical approaches obtain mixed performance results when it comes to identifying disinformation online. Most methods perform well on curated data that has been thoroughly cleaned and well-standardized, but these methods are subject to accuracy problems when deployed in real-world situations and on “messier” data. These technical approaches can typically fit into three broad categories: language-based, machine learning, and network analysis approaches.

Language-Based Approaches

These methods involve examinations of the text itself. They are not typically concerned with understanding the larger context surrounding potential sources of disinformation, but rather, they are interested in underlying structural patterns associated with false information. These techniques work to evaluate the syntax, parts of speech, or language complexity of a text in order to find similarities between types of documents – credible or otherwise.

Machine Learning Approaches

Machine learning approaches are highly popular and diverse, relying on tabular and unstructured data sets to train effective algorithms for classifying texts. These techniques include deep learning models, decision trees, and clustering methods, and they require data that is tailored to particular types of texts and content to adequately train the models.

Network Analysis Approaches

Network-based analyses are concerned with identifying who is producing and circulating certain narratives. This provides a way to examine how specific groups or digital communities are responding to the dissemination of disinformation narratives. Because this approach requires a lot of contextual information about the network of interest, it can become difficult to track new sources of questionable texts. It does, however, allow for a better understanding of connections between entities on social media and the ability to track or traceback information to its source.

Combining Approaches: How can we apply these ideas?

When used independently, language-based, machine learning, and network analysis methods have seen some success. However, disinformation is complex and constantly evolving, requiring practitioners and researchers to develop new methods that can be just as adaptable. To do this, many new approaches are focused on combining existing techniques to form composite models and analysis pipelines, giving us the right tools to tackle highly niche problems.

In our upcoming talk at ODSC East (April 2021), “Narrative Extraction for Disinformation Detection,” we will outline one of many emerging ways to do this by sharing our strategy of combining deep learning classifiers with topic modeling to identify disinformation in articles and fake news. We plan to walk through our methodology and explore a use case where we examine articles about the 2020 U.S. Presidential Election.

While no single approach will perfectly identify disinformation, continuing to develop new ways of thinking about and detecting it can only be helpful for maintaining media platforms that are dedicated to providing reliable information.

About the authors/ODSC East 2021 speakers:

Amber Chin is a Machine Learning Engineer at Novetta and a student at the University of Texas at San Antonio. Her primary interests are in natural language processing and applying deep learning to analyze digital communication. Prior to starting at Novetta, Amber worked in UTSA’s Digital Politics Studio examining the effects of political incivility on social media platforms and developing methods for detecting uncivil language online. Amber will graduate from UTSA this spring with Bachelor’s degrees in Psychology and English.

Carlos Martinez is a Machine Learning Engineer at Novetta and a graduate student in Computational Linguistics at Montclair State University. His primary interests are in natural language processing, linguistics, and voice application development. Prior to working at Novetta, Carlos worked as a voice app developer for Voicefirst Tech and as a NLP research assistant for the Montclair State University research lab detecting censored language on Chinese social media.

12 3 4 Next ›Last »