Automated AI Model Deployment Pipeline for Healthcare

Client Background

The client is a forward-thinking healthtech institution dedicated to advancing medical imaging through the implementation of cutting-edge technologies. Recognizing the potential of artificial intelligence in healthcare, the client sought to leverage the Monai framework to optimize their deep learning model training, deployment, interface unification and evaluation processes. They aspired to create a robust workflow for their healthcare data, striving for a more systematic and efficient approach to model development & deployment.

Problem Statement

Despite the advantages offered by Monai, the client faced challenges when it came to serving their models in a production environment. They needed a solution that could not only streamline the integration of pre-trained models and domain-specific operations but also handle critical aspects of production such as authentication, monitoring, and auto-scaling — all within less than a day’s work, instead of taking a whole week. The varying requirements and infrastructure of their organization demanded a flexible yet effective production-ready setup.

Solution

To address these challenges, the client collaborated with Antematter to develop a micro-framework tailored around Monai's capabilities and their unique requirements. This bespoke framework added the necessary features to support HTTP requests, authentication, logging, visualization, security ,and auto-scaling, amongst others – all while integrating seamlessly with Monai.

Utilizing a high-level architecture involving AWS API Gateway, EKS, and load balancing, the team ensured authenticated client requests were efficiently routed to the appropriate model endpoints. To automate model deployment and resource provisioning, an infrastructure-as-code approach was implemented using Terraform, complemented by a CI/CD pipeline using Github actions. This systematic orchestration facilitated a smooth transition from model training to production, providing agility and reliability throughout the process.

Architecture

Results & Highlights

• The custom micro-framework elegantly wrapped Monai, enabling standardized interfaces and processes across all models.
• Adopting infrastructure-as-code with Terraform ensured repeatable, documented, and version-controlled deployments, reducing room for errors dramatically.
• The CI/CD pipeline streamlined and automated pushing trained models to production, minimizing manual intervention, and accelerating deployment cycles.
• The ELK stack integration for logging and visualization enhanced the client’s capability to monitor model performance and system health effectively.
• Auto-scaling features assured that the system could handle varying loads, optimizing resource utilization while maintaining performance.
• The resulting pipeline reduced testing to production time from around 40 man-hours to just 6, hence greatly enhancing the ability of the team to iterate and test.