Consume and Serve AI Using Caikit and ModelMesh
Not currently scheduled.
Presented by
-
Rafael Vasquez
@https://twitter
https://rafvasq.github.io/talks
Rafael Vasquez is a software developer on the Open Technology team at IBM. He previously completed an MASc. working on self-driving car research and transitioned from a data scientist role in the retail field to his current role where he continues to grow his passion for MLOps and open source. Most recently he has been dedicated to open source projects such as KServe ModelMesh.
Rafael Vasquez
@https://twitter
https://rafvasq.github.io/talks
Abstract
In this talk, both Caikit and ModelMesh will be introduced along with a demo showing various features of both projects in the context of multi-model serving.
Caikit is an open source AI toolkit that enables users to manage models through a set of developer friendly APIs. Caikit streamlines the management of AI models for application usage which enables developers to write applications that consume AI models who may not understand the intricate details of the AI models that they use.
ModelMesh is a model serving management and routing layer, optimized for high volume, high density, and frequently changing model use cases. It intelligently loads and unloads models to and from memory to strike a balance between responsiveness and compute. Used in production for several years at IBM, it was contributed to the open source community as part of KServe and has served as the backbone for most Watson services, such as Watson NLU and watsonx Assistant.
While Caikit provides an abstraction layer for developers to consume AI models through APIs, ModelMesh provides a serving layer for multi-model use cases.
In this talk, both Caikit and ModelMesh will be introduced along with a demo showing various features of both projects in the context of multi-model serving. Caikit is an open source AI toolkit that enables users to manage models through a set of developer friendly APIs. Caikit streamlines the management of AI models for application usage which enables developers to write applications that consume AI models who may not understand the intricate details of the AI models that they use. ModelMesh is a model serving management and routing layer, optimized for high volume, high density, and frequently changing model use cases. It intelligently loads and unloads models to and from memory to strike a balance between responsiveness and compute. Used in production for several years at IBM, it was contributed to the open source community as part of KServe and has served as the backbone for most Watson services, such as Watson NLU and watsonx Assistant. While Caikit provides an abstraction layer for developers to consume AI models through APIs, ModelMesh provides a serving layer for multi-model use cases.