The art of outpacing change: how the role of data scientists is evolving
If there is one thing that is certain in life it is change. In analytics, the pace of change now feels faster than ever, with new challenges and opportunities arising every day. When change happens this rapidly, it is very easy to feel that you are constantly on the back foot, reacting to events.
It is therefore important to stop every now and then, and consider potential change in a more strategic way. With the democratization of artificial intelligence (AI), especially after the boom of generative AI like ChatGPT, it feels valuable to look ahead. We need to consider how data science, and data scientists, can help businesses to ‘outpace’ change: to get ahead of the curve and be proactive instead of reactive.
The challenges facing data science
Data scientists face many challenges in businesses today. Many organizations are still developing their data and analytics literacy. They may have an ambition to be data-driven, but the reality is that many people still do not trust the outputs of analytical models, or even understand the implications data quality can have on the results and decisions that are made with them. The rise of AI has also led to concerns about data ethics, and how to manage the ethical issues linked to analytical models. But at the same time, many business teams are wondering why they can’t just forge ahead using ChatGPT and the like.
I think one of the biggest challenges is the balance between accuracy and interpretability. It may be possible to build a very accurate model that includes every possible parameter—but can you ever explain why it produces a particular answer? This is made more of a challenge by the need to explain this issue to business users, who want accurate answers, but not at the cost of time or understandability. They may find it hard to grasp the payoffs inherent in developing any analytical model.
All these challenges are amplified by the pace of change. As data scientists, we need to take the time to stay up to date with changes in our field, including new tools and technologies. Sometimes, this requirement can feel almost overwhelming. However, one day of training now can save many hours of work in the future. Staying updated is the only way to get value from what is available.
The role of teamwork
The ‘red thread’ running through all these challenges is teamwork and trust: between data scientists and the business, and within teams of data scientists. Business users are more likely to trust analytical models and their outputs if they trust the data scientists who developed those models. It is therefore essential to build strong relationships with business teams.
The best way you can do it is showing how you can add value to business teams and their work. This needs skills like storytelling, to make results understandable, and the ability to understand business needs and translate them into technical outputs (and vice versa). With time and patience, business teams come to see that data science adds value, and then to rely on that value.
There is, however, a key assumption in all this. You can only add value if you can build models that generate accurate, reliable results, and then interpret those results for the business. This requires good data quality, of course, and an understanding of data. However, with AI in particular, it also means having an awareness of your own biases, and actively looking for ways to expose and reduce them.
Diversity and teamwork
This is where the data science team comes in. There is a tendency to assume that there is a single ‘ideal’ list of skills that every data scientist should have. This would lead to teams consisting of a series of near-clones, or at least people with very similar backgrounds and levels of experience. I don't think it's an overstatement to say that you would lose potential value and your competitive advantage vs other competitors.
Any team—especially in data science—needs diversity. The best teams have a range of experience, backgrounds, and opinions, brought together in a way that enables constructive challenge and criticism, and learning from each other. Hiring managers need to be actively seeking to expand team diversity by looking at the gaps in the data science team.
Multidisciplinary teams and even multidisciplinary profiles, coupled with curiosity and great communication skills, will provide the strongest results. These will give businesses the best chance of outpacing change. Data scientists play a key role in helping to achieve this—but they cannot do it alone.
