How will Increasing Automation Impact Data Science Jobs?
The answer probably isn’t what you think it is
People tend to assume that using automated systems and robots — in any industry — means cutting back on jobs that were previously held by humans. Although it’s true that automation does have an effect on jobs, it isn’t as clear cut as it may seem. In fact, increasing the use of automation might do the opposite of what people think.
Automated systems don’t just create themselves. Engineers, programmers, and data scientists are all needed to create those systems in the first place. That means there will actually be more data science jobs opening up in the next few decades as businesses automate more parts of their operations.
What Machines Can’t Do
It might seem like today’s artificial intelligence (AI) systems and data processing algorithms can do anything. While they are very impressive, most automated systems are only good at doing one thing or one type of thing.
That’s because using AI to solve problems isn’t that simple. You can just set an algorithm loose with a bunch of data and expect it to come up with the solution you’re looking for. Instead, much of the work that goes into data science occurs before writing a single line of code.
For instance, data scientists need to ponder questions like what type of data to analyze, how to find that data, and determine what problem the data solves. Automated systems can’t figure those things out on their own.
People outside of the data science field tend to ignore this fact. That’s because it doesn’t include flashy algorithms or quick solutions.
Take one of Target’s automated systems, for example. It is able to determine if a customer is pregnant based on their shopping patterns. People generally assume that “the AI did it” and that this system is foolproof.
Target’s analysis tools are certainly powerful. However, the end result doesn’t portray the whole picture. Before the algorithms that make it possible were written, a team of data scientists was figuring out why it matters if customers are pregnant. They knew that this specific customer demographic is often open to choosing a new primary retailer. Meanwhile, the AI system only knew that they were pregnant.
On top of this, the human data scientists were able to draw on decades of previous research and understanding of consumer behaviors. This allows them to apply the results from the algorithm and turn them into profitable changes to Target’s business model. Without those human workers, the AI would, essentially, be a pointless form of pregnancy test.
Instances like this help show why data science jobs aren’t going away. No company wants a number-crunching algorithm only for it to arrive at a meaningless solution. They do want experienced minds that can think critically, create smart algorithms, and then turn that data into actionable guidance.
Munging Away
Aside from relying on past information, data scientists also need to prepare in advance before creating algorithms to solve problems. A large part of this process is called data munging. In other words, data scientists need to clean up their datasets before using them.
Data collected from real-world systems is incredibly messy. It comes with extra tags, unneeded variables, and can be incredibly confusing to interpret. Fortunately, skilled data scientists are able to bridge the gap between real-world data and the answers to the questions they hope to address. Much of that process relies on their contextual knowledge of real-world systems and the business they are working for.
Going back to the Target example, data scientists had to make assumptions for many things. Doing so allowed them to ditch extraneous data, normalize features, and make control groups for accurate comparison.
This sort of work can’t happen without human judgment. Although AI algorithms are powerful, they can’t replace the human mind. Recent instances of machine learning systems introducing bias into research are also particularly concerning.
With that in mind, it becomes clear that removing humans from the data science picture is impossible. Not to mention the fact that data munging and feature engineering make up the majority of the work done in the field.
Learning from the Past
Several years ago, there was a field that caused people to fear for their jobs in nearly the same way that data science does today — software engineering. Many people assumed that once creating new programs got easy enough there wouldn’t be a need for human programmers. Today, everyone knows that this isn’t the case.
To be clear, there is nothing easy about software engineering. If you don’t believe that, feel free to check in with any programmer who pulls countless all-nighters to try and debug their code.
That being said, software engineering is a lot easier than it was a decade ago. Yet, demand for highly-skilled programmers has only gone up. Today’s software engineers are paid tremendously and work at some of the coolest companies on the planet. Instead of easier software engineering creating less demand in the field, there is more than ever.
This paradox serves as a sort of parallel to the field of data science. As the use of automation increases in every sector, the demand for data science positions will increase exponentially. Human workers will always be necessary.
Automation, by nature, will actually create new data science jobs — not eliminate existing ones. We’ve seen this happen with software engineering and the trend will repeat itself with data science.
Those looking for a profitable, growing field to enter should consider data science. Not only will there be jobs available in the years to come; those job openings will allow applicants to change the world with automation.
Originally published at https://www.theburnin.com on September 04, 2020.
