From my experience, on the daily the split looks something around 90% software engineering and 10% math. But you absolutely must know the math.
There was this diagram presented in thekeynote (well, originally, ) which summarizes the components of an ML production system nicely:
All of the gray boxes here are engineering tasks, while the orange “ML code” box involves a mix of engineering and mathematical thinking.
Machine learning has been called “the high interest credit card of technical debt”. For example:
- What are your business needs for a prediction API? What type of throughput/SLA are you expecting?
- What monitoring tools do you have in place? Especially if you’re training online, the distribution of inputs encountered may change. Are you prepared to handle that?
- What type of machines do you have running? You might not need a cluster of Tesla V100 GPUs if you’re a startup, you might only need one or two dedicated GPUs.
So where does the math come in?
There are times when you need to develop a new model, rather than making an existing model production-ready. Many ML engineers also wear another hat: that of the data scientist.
Often, you’re given a task such as “build a recommendation system”. This task has a pretty enormous range of possible solutions, so math is useful for model creation, selection, and evaluation. To make this more concrete:
- Based on the data distribution / problem definition, identify which models would work well with it.
- Perhaps you understand that a given cost function might not make sense for a given problem. Understanding cost functions is extremely important, since they bring about very different solutions.
- Model evaluation and fairness is a very important area for making sure your product is providing a positive experience to all of your users. For example, even if you have a 99% accuracy for a given task, it could be the case that for a given segment of the user base, the algorithm performs very poorly.
It’s also fairly common for ML engineers to write papers based on the work that they do, since they need to extend state-of-the-art models to fit their use case. It’s important to keep up to date, since the ML field moves fast.
This is why I love ML engineering: you can simultaneously build mathematical insight, engineering rigor, and product thinking in one job.