Uber Eats for Data Science (ModelOps)
Data science is taking off and failing at the same time. NewVantage Partners reports that 92% of companies are accelerating their investment in data science, up 40% year over year. Yet just 12% have deployed AI at scale—that’s down from 15%. (1)
So firms are investing more in data science and putting less of it into production. It’s like buying a Ferrari and leaving it in your garage out of fear.
What’s going on?
Sure, there’s cause to be careful with AI. Security, bias, and privacy, to name a few. But some IT teams go too far—they hand-curate algorithms and re-write them from scratch. I recently met a firm that deploys just one model a year as a result.
There’s a better way. Model operationalization, or ModelOps, is a new class of tools that helps firms deploy data science safely, reliably, and transparently. IT is in the loop, but they don’t have to rewrite everything. Once data scientists develop algorithms, they publish them to a ModelOps repository. Business teams can search models and discuss which ones might work. AI ops teams manage algorithm deployment via containers, Python notebooks, a BI tool, or within a data fabric. All of this happens in hours or days rather than months or years.
As for forensics, ModelOps tracks every step. This helps evaluate model performance, identify bias and spot drift (2). In some industries, like pharmaceuticals, this transparency is mandated.
ModelOps is like Uber Eats for data science. Uber Eats lets chefs focus on cooking. ModelOps helps data scientists focus on science. Let someone else worry about delivery. It’s helping firms get more from their AI investments, not by doing more data science, but by making it easier to travel the last mile.
(1) New Vantage Partners 2021 (and 2020) survey on data science— forward by Tom Davenport and Randy Bean.
(2) Model Drift (also known as model decay) refers to the degradation of a model's prediction efficacy due to changes in the environment, and thus the relationships between variables. For example, predictions of what type of clothes a customer might buy fluctuates with the weather. Periodic re-training is required to current factors are considered.
