In an era when email the most common way of official communication, managing email threads efficiently is essential. Lengthy email conversations can become overwhelming, making it challenging to extract the key information or sentiments from them. To address this issue, we embarked on a journey to create an intelligent solution using AWS services and fine-tunning open-source LLMs. In this blog post, we'll walk you through our project, step by step, showcasing how it optimizes email comprehension efficiency with the power of AI.
AWS SageMaker Model Registry: Where Magic Happens
We chose to fine-tune the T5 (Text-to-Text Transfer Transformer) model, as this was the best open-source encoder-decoder LLM from HuggingFace , to ensure it could understand the nuances of email content and produce accurate and coherent summaries.
After training our T5 model, the next challenge was to deploy it in a testing environment. We chose AWS SageMaker Model Registry as our go-to platform for model deployment. To ensure the quality of the model, before deploying the model, it had to go through an approval process. Initially, the model is registered with a "pending for approval" status.
Automation with EventBridge and Lambda
To streamline the model deployment process and make it as user-friendly as possible, we implemented an EventBridge rule. This rule would monitor the model version's status on the AWS SageMaker Model Registry. When the model version status changed to "Approved," it triggers a Lambda function.
This Lambda function starts a GitHub Action to create or update a SageMaker endpoint. These endpoints allowed us to use our T5 model to infer email summarization. The result of this inference a concise summary of the email thread.
Data Routing with AWS SQS
We use SQS to separate model monitoring from the inference. To gather more data to gain insights into the effectiveness of our T5 model. So, we used AWS Simple Queue Service (SQS) to collect the email thread, the T5 model's prediction, and a prediction generated by OpenAI.
The collected data was routed to another Lambda function that served two purposes. Firstly, it logged the email thread, T5 model's prediction. This logging ensured that we could keep track of the performance of the model over time.
Measuring Toxicity and Sentiment
The Lambda function, focused on monitoring, is equipped with the capability to measure toxicity and sentiments within the email thread. This additional information allows us to assess the overall tone and potential issues within the conversation. We wrote more about this at this blog.
Scheduled Data Archiving
To maintain a record of our predictions and insights, we implemented yet another Lambda function that got triggered by an EventBridge scheduler every 24 hours. This function is responsible for collecting the predictions logged in CloudWatch and storing them in an AWS S3 bucket. This archival process ensures that we keep a historical record of email thread summaries and analysis, facilitating long-term analysis and reporting.
User Interaction: Streamlit App on AWS ECS
To make our solution accessible to users, we deployed a Streamlit web application on AWS Elastic Container Service (ECS). Users could visit this app and select a subject whose email thread they wanted to summarize. The app communicated with our backend system, triggering the summarization process, and presented the summary to the user in a user-friendly format. The app is integrated with Gmail and Outlook.
Streamlit App View
Conclusion
In summary, our project demonstrates the power of AI in optimizing email efficiency. By automating email thread summarization, we help users save time and make more informed decisions. Furthermore, with comprehensive logging, sentiment analysis, and scheduled data archiving, this solution provides valuable insights into email communication dynamics.
As email continues to be a cornerstone of professional communication, projects like ours show how technology can enhance productivity and effectiveness in the digital workplace. We hope this project inspires you to explore new ways of using AI to streamline your work and improve your communication.
Whether you're managing a busy inbox or building AI-powered solutions, there's a world of possibilities to explore in the realm of email efficiency.