The Growing Trend of Data Science in Japan

Google searches for data science, and all its related buzzwords like “Big Data” “Machine Learning” and “AI” has roughly reached the same level of frequency and interest and the searches for climate change, in addition to become integrated into daily vernacular for large swaths of internet users. And yet, people don’t seem to have a full understanding on what this field is about. However because of the huge hype about the potentially large cost saving measures data science can bring, people look to implement forms of analytics prematurely and without a good understanding of what they want the implementation of this technology to achieve, other than “save money.” So what actually is data science? It can be summarized as: solving complex problems through analyzing data. The buzzwords that everyone is searching on Google are the examples of the various ways that is achieved.  

Companies in Japan are no different than anywhere else in the world; they too are looking to ride the data science trend by employing data scientists. If we target the above graph to just to Japan we can clearly see that interest in the current data science trend of machine learning is twice as popular as all the other topics (including climate change!).

However, according to a 2015 survey conducted by Nikkei Asia, 62% of companies say that their biggest challenge is finding talent. The majority of the companies surveyed rely on educating internal employees to handle analysis needs and 39% of companies are using tools (e.g. computer software) that can easily be managed by employees without any specialized skills.

In order to implement data analytics into company practices, the biggest challenge for organizations both in Japan and the United States is attracting and retaining talented data scientists. Therefore, internal education and using tools that employees are familiar with is the current (inefficient) solution to answer data science questions. An easy way to incorporate data science is to promote Microsoft Excel because most people have a basic understanding of how to operate Microsoft Excel and the company doesn’t need to make large investments in expensive software or talented data scientists. According to a KD Nuggets survey, excel makes up about 40% of tools used in data science.

Microsoft Excel itself is a very powerful tool because without a lot of training 2-dimensional data can be processed to create an output that is “good enough”. It’s very easy to put some data into the tool, format it, and then generate some graphical output for that afternoon’s meeting in Microsoft PowerPoint. It allows people to express a simple data idea in the form of easily digestible content, however as data structures grow in quantity and complexity the limitations of Microsoft Excel become apparent.

In businesses, those that don’t work commonly with programming tend to store many complex data sheets together that result in huge datasets to reference, or they create a long tedious processes of data entry within excel because it’s what they know how to do. This is not the type of work excel was intended for, although Microsoft is working on solutions to enhance performance with these problems. These are just some of the problems that the field of data science tries to solve. For example:

There are of course other ways to solve these problems, but these are some of the basic approaches and each problem has its own set of circumstances that must be met (mainly time and money). What seems to be common in Japan at the moment is that these simple solutions are often not the focus when it comes to answering a data science problem. Rather the approach is to allocate more time to a problem than to come up a better solution.

It comes down to requiring more people within a company that are versed in knowing what is available to solve a given problem. The ability to understand, articulate, and create a solution is the reason why data scientists are so sought after. They provide a buffer between the business and raw data, breaking down the problem and offering solution options for the business to pursue.

While LinkedIn may not be the default job recruitment tool in Japan, it provides an indication that there aren’t many job posts for data scientists in Japan. According to the site’s job postings: Japan shows 0.2% (several hundred) of posts are for data scientists, while in the US it’s about 1.1% (tens of thousands) of listings. For a growing field and one that is being emphasized by Japan’s CXX, the search is a bit more challenging.

For someone looking to break into the data science field in Japan and not already within the country there are a couple routes to try in order to go abroad:

Ultimately, Japan provides a unique working and social experience that can help someone grow personally and professionally. There is plenty of room to improve processes in any business and Japan recognizes that data science is a necessity in order to be competitive. For someone looking to succeed in this growing market, it may not be easy, but it’s definitely worth a try – good luck!