Pollution, bad working conditions, and animal welfare are topics that are often pushed away by profit. This certainly holds for the modern ‘fast-fashion’ industry which produces most of our clothes. Fortunately, some brands and initiatives are making an effort, but how do you know as a consumer?

It is difficult to make a well-informed choice when buying sustainable clothes …

Searching for information is very time-consuming. Sustainability information is scattered over several sources. Websites, blogs, and books provide valuable guidelines about clothing brands that try to make a difference. Clothing brands provide more and more information on their website about sustainability. Unfortunately, it takes a lot of manual work to gather and build up this sustainability information, therefore the available information is often outdated.

… but Artificial Intelligence can do the job! 

We believe that the (fashion) industry can move towards sustainability by automating determining sustainable brands. Therefore, we use scraping, artificial intelligence, natural language processing, and explainability, to provide more sustainable clothing information, faster than current approaches.

In data science, you often start with a dataset that needs a lot of cleaning. In this case, we had to build from scratch by first making a database that contained all the clothing brands. Second, for all clothing brands, their homepage was added. From here, the scraping fun could start. However we did not have infinite recourses, so we had to be smart about scraping exactly the right amount of data.

While investigating the data for the first time, after some preprocessing, we got the feeling that the model could give valuable insights. These two word-clouds contain the most frequent words per class and already show a clear difference. Interestingly, we mainly see brand names in the ‘not-sustainable’ word-cloud. Fortunately, the model does not train on the names of the specific brands, which we know through explainability.

Explainability is crucial for people to believe the outcomes. In our case, it also improved the quality of our preprocessing. As a result, we now have several models with over 80 percent accuracy. For the outcomes that are still too uncertain, we make a class ‘unknown’. In this way, people can trust the outcome of the model.

Information at your fingertips

We were quite proud and happy at this point, however, a model itself does not accomplish anything. In our opinion, this model could make a change by providing its output publicly. Therefore we built goodbase.ai. This website provides sustainability information on every clothing brand (we knew). You can add brands yourself that will automatically be assessed.

When you think about the online buying process of clothing, it is still a hassle though to constantly check another website while shopping. You want the information at your fingertips without switching websites. Therefore we built a plugin together with project Cece, that we will launch in the second week of September.

Next steps

Although we are proud of the great things we have accomplished, we do know a lot more impact can be made. Both at the technical side as well as in terms of usage.

This is how we see the future, how do you see it?

• Use goodbase.ai and the plugin and provide us with feedback/add brands etc
• Spread the word
• Help improve the code/build new features: https://gitlab.com/thehup/goodbase.ai/duurzame-benchmark

Our database will provide a piece of the sustainability puzzle. Do you want to know more about how we do this and how you can contribute?

Be part of the change!

If you want to know more about why we built the model: https://amsterdamdatacollective.com/2019/12/04/blog-could-artificial-intelligence-help-you-buy-sustainable-clothing/ or please get in touch.

About the Author/ODSC Europe 2020 speaker: Joanneke Meijer is a Manager at Amsterdam Data Collective and initiator of the sustainable benchmark. She is an experienced data science consultant, focusing on forecasting, pricing, operational research, and text mining. By coaching teams towards delivering actionable insights from data, she continuously manages to create a sustainable impact.

In Natural Language Processing (NLP), language models such as ULMFiT, BERT, and GPT have become the foundation of many solutions for common NLP tasks. The benefit of language models is their ability to be pre-trained with a general understanding of language, such that users can fine-tune models on significantly less data and achieve better performance than when starting from scratch. Prior to language models, NLP models required enough data to simultaneously learn a language and a task, such as classification.

At Novetta, we achieved amazing performance with language models, but it was difficult for developers and new data scientists to train and deploy their own models. To address this, we decided to streamline the implementation of state-of-the-art models for different NLP tasks. We built an open-source framework, AdaptNLP, that lowers the barrier to entry for practitioners to use advanced NLP capabilities. AdaptNLP is built atop two open-source libraries: Transformers (from Hugging Face) and Flair (from Zalando Research). AdaptNLP enables users to fine-tune language models for text classification, question answering, entity extraction, and part-of-speech tagging.

Example: Text Classification

To demonstrate how AdaptNLP can be used for language model fine-tuning and training, we will fine-tune a pre-trained language model from Transformers for sequence classification, also known as text classification.

Using AdaptNLP starts with a Python pip install.

pip install adaptnlp

First, we import EasySequenceClassifier, which abstracts the sequence classification task to its most basic components such as data preprocessing, inference, and training. We can then instantiate the EasySequenceClassifier class object to start training our own custom sequence classification model.

from adaptnlp import EasySequenceClassifier

classifier = EasySequenceClassifier()

To train a sequence classification model with our `classifier`, we need to prepare our data and our training hyperparameters.

AdaptNLP is tightly integrated with Hugging Face’s nlp library, so we will import nlp and load in the “ag_news” dataset. The AG News dataset is a collection of news articles labeled as one of four classes: world, sports, business, or sci/tech. This makes it a perfect multi-class dataset for us to train our classifier on. If you’d like, explore the dataset on Hugging Face’s nlp Viewer UI and try out the amazing nlp library in general.

Note: The classifier can be trained with CSV data file path inputs as well as nlp.Dataset inputs.

from datasets import load_dataset

train_dataset, eval_dataset = load_dataset('ag_news', split=['train[:10%]', 'test'])

Now that we have our train and evaluation/test datasets, we can now create the training arguments object from the transformers library, TrainingArguments. This lets us specify parameters and hyperparameters for training the classifier such as output paths, epochs, batch size, and weight decay. Wonderful, extensive documentation on TrainingArguments can be found on Hugging Face’s documentation site.

From transformers import TrainingArguments

 
training_args = TrainingArguments(

output_dir='./models',

num_train_epochs=1,

per_device_train_batch_size=16,

per_device_eval_batch_size=16,

warmup_steps=500,

weight_decay=0.01,

evaluate_during_training=True,

logging_dir='./logs',

save_steps=100

)

We can then start training by running the classifier’s built-in `train()` method, which takes in the train and eval datasets and the `training_args` variable we created. Besides specifying the text and label column names, you will now specify the pre-trained language model to fine-tune.

classifier.train(

training_args=training_args,

train_dataset=train_dataset,

eval_dataset=eval_dataset,

model_name_or_path="bert-base-cased",

text_col_nm="text",

label_col_nm="label",

)

Important Note: In this example, we will use the “bert-base-cased” pre-trained language model with a sequence classification head. However, you can use nearly any pre-trained language model. Try out a pre-trained DistilBert or an Electra model,  a custom fine-tuned model, or any model in Hugging Face’s model repository.

After training is completed, all artifacts and metadata such as checkpoints, model files, configs, and logs will be located in the directory paths specified in your `training_args` for `output_dir` and `logging_dir`. In this example, they are in “./models” and “./logs”.

You will then run a final evaluation with the built-in `evaluate() method to see how well your model performs by calculating metrics on the eval/test dataset.

classifier.evaluate()

Outputs:

{'epoch': 1.0,

 'eval_accuracy': 0.9019736842105263,

 'eval_f1': array([0.90401969, 0.9692994 , 0.85683646, 0.87806097]),

 'eval_loss': 0.295024262882377,

 'eval_precision': array([0.9408082 , 0.96650968, 0.87322404, 0.8358706 ]),

 'eval_recall': array([0.87      , 0.97210526, 0.84105263, 0.92473684])}

Great! You’ve successfully fine-tuned and trained your own sequence classifier for the AG_News dataset. Now let’s explore the data and model objects in more detail

The EasySequenceClassifier object can dynamically load and run mini-batch inference on nearly any Transformers model, including the one you just trained. You can load the model and run mini-batch inference with the built-in `tag_text` method.

text = [

"The batter up went for the run and scored a touch down.",

     "The engineer designed rocket fuel that can take us to mars.",

     "The president of the United States and the prime minister of Britain talked.",

     "The stock market went down as the economy took a hit from stuff."

]

 

results = classifier.tag_text(

text=text,

model_name_or_path = "./models",

mini_batch_size=2

)

print(results)

Outputs:

[Sentence: “The batter up went for the run and scored a touch down .”   [− Tokens: 13  − Sentence-Labels: {‘sc’: [World (0.0421), Sports (0.9516), Business (0.003), Sci/Tech (0.0033)]}], 

Sentence: “The engineer designed rocket fuel that can take us to mars .”   [− Tokens: 12  − Sentence-Labels: {‘sc’: [World (0.1011), Sports (0.0295), Business (0.0767), Sci/Tech (0.7928)]}],

Sentence: “The president of the United States and the prime minister of Britain talked .”   [− Tokens: 14  − Sentence-Labels: {‘sc’: [World (0.9544), Sports (0.003), Business (0.0335), Sci/Tech (0.0091)]}],

Sentence: “The stock market went down as the economy took a hit from stuff .”   [− Tokens: 14  − Sentence-Labels: {‘sc’: [World (0.0243), Sports (0.0013), Business (0.9655), Sci/Tech (0.0089)]}]]

While you’re at it, you can try to run `tag_text()` with a different model fine-tuned on AG_News from Hugging Face’s model repository to see how your custom trained model fares.

Fine-Tuning Language Models

To go beyond only fine-tuning a classifier from general-domain language models, you can use AdaptNLP’s `LMFineTuner` to fine-tune a language model on your target task data. Data from your target task will typically have a different distribution or topic domain from a general-domain language model, so fine-tuning a language model on your target task data can help it “adapt” to your data.

For more information on these techniques, and AdaptNLP in general, visit our documentation site for tutorials, guides, class reference documentation, and more.

A fine-tuned language model can be trained and easily be integrated into user-built systems by providing state-of-the-art text-based classifications. By standardizing the input and output data and function calls, developers can easily use NLP algorithms regardless of which model is used in the backend. Before AdaptNLP, we integrated each version of the latest released model and pre-trained weights, then reiterated through a build for an NLP task pipeline. AdaptNLP streamlined this process to help us leverage new models in existing workflows without having to overhaul code.

Using the latest transformer embeddings, AdaptNLP makes it easy to fine-tune and train state-of-the-art token classification (NER, POS, Chunk, Frame Tagging), sentiment classification, and question-answering models. We will be giving a hands-on workshop on using AdaptNLP with state-of-the-art models at ODSC Europe 2020.

Follow us at @AdaptNLP and give us a star at www.github.com/novetta/adaptnlp!

About the author/ODSC Europe speakers:

Advances in natural language processing have allowed to quantify the intuitive yet elusive notion of sentiment expressed in text and to test its predictive power in relation to changes in social systems.

Studies in cognitive sciences as well as economics have found that unsettling narrative preceded events such as the Great Depression in the 1920s and the Global Financial Crisis in 2008, suggesting that news sentiment is a means of forecasting the economy (1,2). ECB President Mario Draghi’s “whatever it takes” speech from July 2012 is a great example of a narrative that has impacted markets and the economy. These three words marked the turnaround of the euro crisis, purely achieved by Draghi’s verbal intervention.

Newspapers are a proven means for both individuals and institutions to share and distribute information. Most publications have an online presence and generate large amounts of data. This data includes information in the shape of sentiment and opinions about the economy, which is not yet reflected in macro-economic indicators.

The Global Database of Events, Language, and Tone (GDELT) (3) is a research collaboration that monitors the world’s newspapers from a multitude of perspectives, extracting items such as themes, emotions, events, mentions of organizations, and persons and locations for every news article analyzed almost in real-time.

News sentiment and the economic recovery following COVID-19

As the coronavirus spread around the world, governments in many countries were forced to impose strict lockdowns and temporarily close businesses. As a result, a lot of companies are struggling to survive. Many had to lay off employees, leading to a spike in unemployment.

The chart shows the average tone, financial uncertainty, and confidence indices based on emotions from GDELT, with news items filtered thematically for “economic growth.”

Net sentiment (i.e. positive minus negative tone) from global newspapers has improved notably since bottoming in April this year.

Levels of financial uncertainty peaked at the height of the outbreak in early spring. Financial uncertainty has since been declining but remains at elevated levels.

Confidence increased until February, perhaps reflecting confidence in local governments’ ability to contain the virus. The index plunged in March as lockdowns were imposed in countries around the world. Confidence moderately recovered in April but moved mostly sideways in May and June as the longer-term economic repercussions from the COVID-19 outbreak became apparent.

According to net sentiment, recovery is well on underway. However, the two other indices tell a somewhat different story. The coronavirus is likely to have a powerful impact on confidence as consumers are staying at home for a prolonged period, possibly feeling pessimistic about the future. With heightened levels of financial uncertainty and worsening financial conditions, household consumption typically falls as savings go up, weighing on economic growth.

Thoughts and conclusions

Net sentiment lacks the insights that more specific emotions can convey about the economic recovery, as it merely indicates whether positive outweighs negative sentiment. More specific emotions from news narratives can help form a clearer view of the present state of the economy. Both confidence and financial uncertainty suggest that the global economy is only at the beginning of a recovery from this year’s COVID-19 outbreak. The shape and speed of the recovery depend on factors like the economic impact of physical distancing, the effectiveness of government support packages, as well as the pace of the easing of lockdown restrictions and the occurrence of a second wave.

Traditional GDP forecasts are unreliable during normal market conditions and become even more challenging at present when the virus’s trajectory is unknown. It is difficult to capture the impact of COVID-19 on consumer and business behavior in a timely manner and thus to estimate a likely recovery path.

To find out more about forecasting the economy with news, narrative, and emotions, join me for my talk at ODSC Europe, “Forecasting the Economy with Fifty Shades of Emotions.”

(1) Robert J. Shiller. NARRATIVE ECONOMICS. Jan. 2017
(2) David Tuckett et al. “Bringing Social Psychological Variables into Economic Modeling: Uncertainty, Animal Spirits and the Recovery from the Great Recession”. In: IEA World Conference, Jordan (2014)
(3) gdeltproject.org

More on the author/speaker:

Sonja Tilly is a PhD candidate at UCL. Her research focuses on forecasting macro-economic variables and stock market movements using narrative and emotions from global newspapers.

Sonja has over a decade of experience working in asset management, most recently at Quoniam Asset Management, where she contributed to the development of trading strategies using media sentiment. Prior to that, she was a Quantitative Analyst at Hiscox. Sonja is a CFA Charterholder.