Building a Domain-specific Guardrail Model in Production
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916345794265088 |
|---|---|
| author | Niknazar, Mohammad Haley, Paul V Ramanan, Latha Truong, Sang T. Shrinivasan, Yedendra Bhowmick, Ayan Kumar Dey, Prasenjit Jagmohan, Ashish Maheshwari, Hema Ponoth, Shom Smith, Robert Vempaty, Aditya Haber, Nick Koyejo, Sanmi Sundararajan, Sharad |
| author_facet | Niknazar, Mohammad Haley, Paul V Ramanan, Latha Truong, Sang T. Shrinivasan, Yedendra Bhowmick, Ayan Kumar Dey, Prasenjit Jagmohan, Ashish Maheshwari, Hema Ponoth, Shom Smith, Robert Vempaty, Aditya Haber, Nick Koyejo, Sanmi Sundararajan, Sharad |
| contents | Generative AI holds the promise of enabling a range of sought-after capabilities and revolutionizing workflows in various consumer and enterprise verticals. However, putting a model in production involves much more than just generating an output. It involves ensuring the model is reliable, safe, performant and also adheres to the policy of operation in a particular domain. Guardrails as a necessity for models has evolved around the need to enforce appropriate behavior of models, especially when they are in production. In this paper, we use education as a use case, given its stringent requirements of the appropriateness of content in the domain, to demonstrate how a guardrail model can be trained and deployed in production. Specifically, we describe our experience in building a production-grade guardrail model for a K-12 educational platform. We begin by formulating the requirements for deployment to this sensitive domain. We then describe the training and benchmarking of our domain-specific guardrail model, which outperforms competing open- and closed- instruction-tuned models of similar and larger size, on proprietary education-related benchmarks and public benchmarks related to general aspects of safety. Finally, we detail the choices we made on architecture and the optimizations for deploying this service in production; these range across the stack from the hardware infrastructure to the serving layer to language model inference optimizations. We hope this paper will be instructive to other practitioners looking to create production-grade domain-specific services based on generative AI and large language models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_01452 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Building a Domain-specific Guardrail Model in Production Niknazar, Mohammad Haley, Paul V Ramanan, Latha Truong, Sang T. Shrinivasan, Yedendra Bhowmick, Ayan Kumar Dey, Prasenjit Jagmohan, Ashish Maheshwari, Hema Ponoth, Shom Smith, Robert Vempaty, Aditya Haber, Nick Koyejo, Sanmi Sundararajan, Sharad Computers and Society Artificial Intelligence Machine Learning Generative AI holds the promise of enabling a range of sought-after capabilities and revolutionizing workflows in various consumer and enterprise verticals. However, putting a model in production involves much more than just generating an output. It involves ensuring the model is reliable, safe, performant and also adheres to the policy of operation in a particular domain. Guardrails as a necessity for models has evolved around the need to enforce appropriate behavior of models, especially when they are in production. In this paper, we use education as a use case, given its stringent requirements of the appropriateness of content in the domain, to demonstrate how a guardrail model can be trained and deployed in production. Specifically, we describe our experience in building a production-grade guardrail model for a K-12 educational platform. We begin by formulating the requirements for deployment to this sensitive domain. We then describe the training and benchmarking of our domain-specific guardrail model, which outperforms competing open- and closed- instruction-tuned models of similar and larger size, on proprietary education-related benchmarks and public benchmarks related to general aspects of safety. Finally, we detail the choices we made on architecture and the optimizations for deploying this service in production; these range across the stack from the hardware infrastructure to the serving layer to language model inference optimizations. We hope this paper will be instructive to other practitioners looking to create production-grade domain-specific services based on generative AI and large language models. |
| title | Building a Domain-specific Guardrail Model in Production |
| topic | Computers and Society Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2408.01452 |