Tools Used:

Introduction

As a trainee at Gulf Air, I worked on a project to help the airline handle the huge number of customer emails they receive every day. These emails cover all sorts of topics like baggage, bookings, schedule changes, and more. Before, staff had to read and sort each email manually, which took a lot of time and sometimes led to mistakes.

Project Overview

To address this challenge, I designed and implemented an AI-powered email classification system. The core of the solution uses natural language processing (NLP) techniques using Python and the Hugging Face Transformers library. For the model architecture, I fine-tuned the DeBERTa transformer, which is known for its strong performance on language understanding tasks.

Data Collection & Augmentation

To ensure the integrity and privacy of the data, I did not use historical company emails. Instead, I collected around 30 real, labeled emails and carefully censored any sensitive information. Recognizing that this was a small dataset, I used data augmentation techniques to expand it:

• Synthetic Generation: I used the GPT-4o-mini model to generate additional emails based on the 30 examples, creating realistic and diverse samples for each category.
• Paraphrasing: To further increase the dataset size and variability, I used a fine-tuned version of the T5 model to paraphrase the generated emails. This iterative process continued until I had a dataset of approximately 9,000 labeled emails.

This approach allowed me to build a robust and diverse dataset while maintaining data privacy and integrity.

Model Training & Evaluation

The final dataset was preprocessed and analyzed using Pandas and Jupyter Notebook for exploratory data analysis and feature engineering. After extensive experimentation and hyperparameter tuning, the fine-tuned DeBERTa model achieved an impressive accuracy of 98.9% on the validation set!

Confusion Matrix

Here’s a confusion matrix showing what the model got right and what it got wrong on the validation data:

Dashboard

To help visualize all the email data, I built a Power BI dashboard that shows important patterns at a glance. First, I created a semantic model in Power BI to organize the data in a logical way - linking email categories and sending dates together.

The dashboard has several helpful views that let staff see:

• Which types of emails are most common
• How email volumes change throughout the week or month

Here is a screenshot of the dashboard:

Note: The data in this image is synthetic data and is not real.

Impact

Using the solution I developed could help the company respond to customers faster by automatically routing emails to the right department. It reduces the workload on support teams and lowers operational costs by eliminating the need for manual sorting. But most importantly, it allows Gulf Air to quickly identify common customer concerns and complaints, helping the company improve its services based on real-time feedback.