In my conversation last week with Manuel Bruscas, he asked what I think the keys are to successful data science management. With a cringe, I thought back to my first-ever interview for a management role, when one of the data scientists asked a simple and obvious question:

What’s your management philosophy?

I realized that despite years of accumulating thoughts on the subject, I had never bothered to distill a coherent philosophy. I floundered but the interviewer and team were forgiving and I got the job. And now, thanks to Manuel’s reminder, I’ve put my thoughts down in writing. I gave him the summary, here’s the expanded version.

Data science management is different

Much ink has been spilled on effective management in general. For engineering management in particular there are some very well-known resources¹ but I don’t see a corresponding body of advice for data science.

My guess is that this gap is because data science management is hard and that most of us prefer to write and read technical articles. The problem is that data science is a unique animal, which presents unique challenges for data science management. How is data science different? Let me count the ways…

It’s part engineering, part craft.
We often don’t know if a new project will succeed before we start. Much of our work is exploratory.
There’s so much hype, which leads non-data science partners to one of two extremes: either it’s all BS or wild overestimates of what our resource-constrained team can accomplish.
The field moves very quickly. Methods, tools, best practices—all of it changes from one month to the next.
Data scientists bounce frequently from one company to another.
Everybody calls themselves a data scientist. There can be hundreds of applicants for an open position, and it’s hard to figure out what each person knows.

Six keys to successful data science management

So let’s talk about the things you can do to be a successful data science manager. First, digest your favorite general advice about good management. Then, consider the following areas as well.

1. Make the work matter

The role of data scientist can be a precarious one. For many companies, the data science team is the first place to look when layoffs are needed; owning business-critical components equals job security. It’s also more fun; it’s a total drag to get excited about a cool idea just to watch it wither on the vine. All in all, when the work doesn’t matter, morale drops quickly and people head for the exits.

What does it mean for the work to matter? For analytics and experimentation, it means business leaders make decisions based on the results. For machine learning, it means getting the thing into production. Both of these sound obvious, but they can be hard to achieve in less dynamic or quantitative business cultures.

You need to be tenacious about this and find creative workarounds and compromises to overcome obstacles. Celebrate occasions when your team has an impact. Ask decision-makers to explicitly acknowledge the role of analytics and experimentation. For ML systems, be sure to measure the business impact and broadcast the results.

2. Understand stakeholder goals and concerns

Spend time learning about the goals and concerns of other teams, especially the business units.

It can be hard to pull ourselves away from the looming mountain of data science work, I know. But building bridges with other teams is important, for several reasons.

It’s more likely that our projects will target the things that matter, i.e. business objectives.
We can set expectations about what data science projects can realistically achieve, given our resources.
We build empathy and pre-empt resentment and contempt, in both directions. I can’t overstate the importance of this; inter-team resentment seems unusually common in data science and it’s deeply corrosive.

So what does this look like in practice?

Be sure to meet with your collaborators on an informal, regular basis, maybe for coffee or lunch. Don’t let those meetings drop when the modeling or engineering work gets busy. When you meet, don’t daydream about how the latest amazing ML research could revolutionize their field, ask questions about how their own work is going and what their plans are.
Before starting a new project, shop a proposal around to all potential stakeholders. Listen to their concerns, come up with a risk mitigation plan, and circle back with your partners to make sure the plan is feasible and sufficient.
When you hear resentment start to bubble up, either from collaborators or from within your data science team, nip it in the bud immediately.

3. Balance the fun stuff with delivering value

Data science—more than any other functionality I think—has a big discrepancy between what we want to be doing and what brings value to our employers.

This is especially true for ML, where data scientists want to experiment with the latest and greatest tools and models.² It’s true for analytics and experimentation as well (to a lesser extent), where practitioners are keen to play with advanced models for causal inference and experiment analysis.

The activities that usually bring the most value, however, are things like data cleaning, more robust data pipelines, building systems to monitor, inspect, debug, and refresh models in production, documenting experiment protocols and making sure treatments are applied correctly, understanding how decision-makers consume and use dashboard content, etc.

One management approach to the discrepancy is to insist that data scientists do the boring stuff. I don’t like this; I think it’s much better to strike a real balance that gives data scientists space to experiment with some of the fun things.

Why? Mostly because careers are long and tenure at any one company tends to be short. Data scientists don’t just want to play with advanced models and tools because it’s fun; they know those things look better on their CVs. No, it’s not ideal, but it’s the reality, and part of our jobs as managers is to help our ICs with career growth. Plus, now and then, the fun experiments turn out to yield a lot of business value.

How to strike the balance?

Most importantly, give data scientists a chance to show off their work to other teams. Other engineering and business units will appreciate conclusions and results more than fancy models or experiment designs, which will energize the team to focus on the results.
Minimize process that exaggerates the tedious parts of the job. On-call rotations, Jira tickets, daily standups, etc. may be necessary, but don’t overdo it. Too much process is a quick way to make your data scientists dread coming to work.
Look for projects where the latest and greatest models can bring value. The customer support team very likely has a lot of text data to play with, for example.
Organize a study group. I hesitate to recommend this because study groups can suck up a lot of time and energy, but they can be productive if the topic directly impacts a team’s work. One of my favorite topics is practical causal inference for data analytics.

4. Hire well

Data science hiring is tricky, for at least two reasons:

A successful data science org needs to cover a huge amount of ground, in terms of technical knowledge and skills. Statistics, modeling, experiments, SQL, Python or R, data architecture, MLOps, etc.
On the flip side, it seems like everybody calls themselves a data scientist or ML engineer these days, which makes it hard to figure out what skills each candidate has. On top of that, there’s often an avalanche of applications for an open seat.

A lot has been written about the best way to interview data scientists³ and I don’t have any new silver bullets for you. I do have some high-level advice that I learned the hard way.

Be honest with yourself and your company about your requirements and your ability to pay. Do you truly need that ex-DeepMind researcher who knows how to beat StarCraft with reinforcement learning? Probably not—they’re going to be very expensive and they may not be super interested in getting their models into production. On the flip side, don’t hire somebody straight out of an academic data science program unless they’ve demonstrated the ability to work in a commercial setting or you have the capacity to train them.
Do a lot of interviews. The real world is never a straightforward secretary problem ⁴, but the principle applies: you need a lot of practice before you know both what your options are and that your assessments are calibrated. I interviewed around 20 people for a Director of Data Science position and only by the end did I have confidence in my evaluations. Doing a lot of interviews also helps to clarify what skills you’re really looking for.
Take the partnership with recruiters seriously. Just as we need to persuade our ICs to do some of the boring work to productionize their analytics and models, we need to bite the bullet and do some of the boring, repetitive work of recruiting. If nothing else, spend some time doing top-of-funnel scanning and screening yourself, to help recruiters calibrate.

5. Stay technical and hands-on

After I left my first data science management position, I heard from my team that my replacement was not “hands-on” like I had been. This surprised me because I had interviewed my successor and knew they were technically capable; they must have chosen not to do hands-on technical work.

There is tremendous pressure as a manager to focus on non-technical work, but I think it’s a mistake for a data science manager to abandon technical work entirely. At the end of the day, you manage people with technical jobs and you’re responsible for the output of a technical team, so you need to stay immersed in that world.

Of course, you probably shouldn’t spend much time writing SQL from scratch, debugging data pipelines, or tinkering with model training experiments. So what does it mean to be technical as a data science manager?

Code review and technical coaching for your direct reports. Strict code review as a formal process is not always a good fit for a data science team, but I’ve never regretted taking the time to read a data scientist’s work in detail.
Testing. I once reported to A VP of engineering who didn’t write a single line of code but was a fearsome tester. Testing is doubly productive for a manager because it improves the data science quality and builds empathy for downstream consumers.
Systems-oriented thinking and architecture. Work with other engineering and business teams to build a holistic perspective. This is absolutely a requirement for ML but is equally important for analytics data quality and trustworthy experimentation.
Hiring. See above. It takes technical experience and thoughtfulness to describe the technical skills you’re looking for in a job candidate and to evaluate those skills in an interview.

Incidentally, this is an area where I think data science and engineering management overlap; Camille Fournier has a similar perspective in The Manager’s Path:

…even though you may stop writing code, your job will require that you guide technical decision making…you have the job of holding [architects and senior technical staff] accountable for their decisions, of making sure that the decisions pass the technical smell test, and have been balanced against the overall context of the team and the business…if you truly wish to command the respect of an engineering team, they must see you as technically credible…if you don’t stay in the code, you risk making yourself technically obsolete too early in your career.

6. Accept that it’s a hard job

So that’s it, that’s all it takes: do all of the regular management stuff, then add all of these things on top of it. Piece of cake!

I kid, of course. The last and most important key to success is to accept that data science management is a very hard job. Make peace with the fact that none of your responsibilities will ever be done or perfect. The key thing is to keep learning and improving, individually and as a team. Humility goes a long way.

References

Listing by Xavi Cabrera on Unsplash.

Footnotes

My favorite is Camille Fournier’s The Manager’s Path.↩︎
As of this writing, this would be things like large language models, text-to-image with stable diffusion, applying transformers to applications other than language, etc.↩︎
Some of my favorites:
- Trey Causey, Hiring data scientists
- Erik Bernhardsson, How to hire smarter than the market: a toy model
- Randy Au, We should phase the “SQL Interview” out
↩︎
In short, if you had to make a decision about each candidate immediately after interviewing them, reject the first few, then accept the first candidate you see who’s better than everybody you’ve interviewed so far.↩︎