Data Scientists Should Be Design Thinkers

World Airline Routes

Every company is looking for that cool data scientist who will come equipped with all the knowledge of data, domain expertise, and algorithms to turn around their business. The inconvenient truth is there are no such data scientists. Mike Loukides discusses the overfocus on tech skills and cites DJ Patil:

But as DJ Patil said in “Building Data Science Teams,” the best data scientists are not statisticians; they come from a wide range of scientific disciplines, including (but not limited to) physics, biology, medicine, and meteorology. Data science teams are full of physicists. The chief scientist of Kaggle, Jeremy Howard, has a degree in philosophy. The key job requirement in data science (as it is in many technical fields) isn’t demonstrated expertise in some narrow set of tools, but curiousity, flexibility, and willingness to learn. And the key obligation of the employer is to give its new hires the tools they need to succeed.

I do agree there's a skill gap, but it is that of "data science" and not of "data scientists." What concerns me more about this skill gap is not the gap itself but the misunderstanding around how to fill it.

There will always be a skill gap when we encounter a new domain or rapidly changing technology that has a promise to help people do something radically different. You can't just create data scientists out of thin air, but if you look at the problem a little differently — perhaps educating people on what the data scientists are actually required to do and have them follow the data science behind it — the solution may not be that far-fetched as it appears to be.

Data scientists, the ones that I am proposing who would practice "data science" should be design thinkers, the ones who practice design thinking. This is why:

Multidisciplinary approach

Design thinking encourages people to work in a multidisciplinary team where each individual team member champions his or her domain to ensure a holistic approach to a solution. To be economically viable, technologically feasible, and desirable by end users summarizes the philosophy behind this approach. Without an effective participation from a broader set of disciplines the data scientists are not likely to be that effective solving the problems they are hired and expected to solve.

Outside-in thinking and encouraging wild ideas

As I have argued before, the data external to a company is far more valuable than the one they internally have since Big Data is an amalgamation of a few trends - data growth of a magnitude or two, external data more valuable than internal data, and shift in computing business models. Big Data is about redefining (yet another design thinking element, referred to as "reframing the problem") what data actually means to you and its power resides in combining and correlating these two data sets.

In my experience in working with customers, this is the biggest challenge. You can't solve a problem with a constrained and an inside-out mindset. This is where we need to encourage wild ideas and help people stretch their imagination without worrying about underlying technical constraints that have created data silos, invariably resulting into organization silos. A multidisciplinary team, by its virtue of people from different domains, is well-suited for this purpose.

What do you do once you have plenty of ideas and a vision of where you want to go? That brings me to this last point.

Rapid prototyping

Rapid prototyping is at the heart of design thinking. One of the common beliefs I often challenge is the overemphasis on perfecting an algorithm. Data is more important than algorithms; getting to an algorithm should be the core focus and not fixating on finding the algorithm. Using the power of technology and design thinking mindset, iterating rapidly on multiple data sets, you are much likely to discover insights based on a good-enough algorithm. This does sound counterintuitive to the people that are trained in designing, perfecting, and practicing complex algorithms, but the underlying technology and tools have shifted the dynamics.