Machine learning is fast on the rise. Things are looking quite interesting for the ML landscape today and organisations are becoming increasingly fond of the innovations and advancements that this technology can offer them.
And that’s why it’s become more important than ever for us to explore this groundbreaking technology. Educating ourselves to take full advantage of its many uses will become our greatest competitive advantage.
Better yet, it will come to positively reshape the way that all of us approach the future.
But what does the current landscape look like for ML?
We talked to our machine learning veterans and this is what they had to say:
A Landscape Split In Two
Being that artificial intelligence, machine learning, and other related technologies are still not entirely mainstream, it hasn’t all been smooth sailing.
Although the potential is clear and there exists an abundance of use cases, the landscape is still split.
One side is all in. The other side is simply late to the party.
“I would say that it’s binary. What I mean by that is that there are two camps, the one camp is very advanced with their ML and they have ML models in production that are actually being used for something. The others are nowhere.” said ML Engineer Dr. Dominic Kafka.
“You do have some businesses in between, but they seem to be few and far between. They manage to bridge the gap fairly quickly once they get going, but getting from one to the other is very difficult”.
With many organisations either struggling to understand contemporary technology, or being ill-equipped and unprepared to embrace it, more and more businesses are going to end up losing out.
“Many have not started and those that have started are kicking ass. Think social media platforms, Google, Uber and businesses in between… Whereas more of the larger enterprises are far more conservative and prefer to avoid anything high risk/high demand. So conservative businesses are going to fall a lot further behind, they’re going to take things a lot slower,” said Dr. Kafka.
3 Major Areas Preventing ML Adoption
There are a number of factors that prevent businesses from bridging the gap between ML.
These factors create unnecessary obstacles to adoption and make it difficult to successfully implement the technology.
A reason for failure is that enterprises pass it off as a project, rather than an investment. So the misconception is that ML is either too expensive, confusing or unnecessary.
“ML is only considered a project of innovation. And because it’s only thought of as a project, there are still some major hindrances that are stopping it from being fully fledged or fully adopted”, says Brett St Clair, CEO and Co-Founder of Teraflow.ai.
So for businesses to fully embrace ML, it’s important that we consider these three problem areas:
Poor Access to Data
Sample data can only take you so far. Data scientists have limited access to clean and accessible data to create models with, so they often rely on sample data, which diminishes accuracy.
“70% of the reason why ML models fail is because the right datasets and the right
types of data are not being fed into the model for it to be able to train itself. Because of this inaccessibility of data, the models are failing. And what we’re seeing is that data scientists are just using sample data at the moment. While they might get a little bit of an increment, they’re struggling to make it work. What you need is a large volume of organised, structured and accessible data to be training those models.” said St Clair.
This becomes a data engineering problem. It’s the data engineers that build out the pipelines for the healthy flow of clean, structured data.
Deploying Models at Scale
ML doesn’t only require your data scientists to run the math behind these models. It requires a well-designed framework to approach planning, processing, deploying and evaluating them in real-time.
“…The ability to deploy those models into production is a massive challenge. So you’ve got code everywhere and nothing is organised.” Said St Clair.
“You’ve got to retrain models in the world of hyperparameter tuning, where a) you need a lot of compute power, and b) you need to test many different variables simultaneously. You need to understand those results and it needs to be in a very structured way to the point that you release. So the world of CI/CD in ML is very immature”.
On-Site Hardware
Having done its job for many years, on-site hardware has now run its course.
Other than the massive savings that a cloud-based infrastructure affords you, running ML models in a functional and effective way requires extensive computing capabilities.
St Clair explains that “You cannot run models on site. You can run models on sample data on a machine. But throw in hyperparameter tuning and there’s your first problem. You often need neural nets to tune these models and these require vast amounts of compute”.
“The other challenge with ML is that it’s not an always-on compute. It spikes. When you’re retraining models, it spikes. When you’re ingesting data, it spikes. So you can only do this on the cloud. The amount of compute power you need is vast, so you should be paying for only what you need. This is the ultimate, pure use-case for a cloud computing environment”.
Regulations and Ethics
Being that algorithmic biases exist, we need certain regulations or ethical boundaries to ensure the best interest of the humanity.
“While the current focus on the regulatory aspects of ML might not seem “sexy” and could give the impression that they hamper progress, they are very important to get right, right now, before humanity can embark on the journey towards a fully ML-driven society,” said Machine Learning Engineer, Christiaan Viljoen.
“Having appropriate guardrails in place will prevent all the “scary”, but entirely possible, dystopian scenarios, such as the widespread use of autonomous weaponry, from arising”.
ML is an extremely powerful technology. So we need to consider the output and purpose driving the models that we release. Does it serve the greater good? Or does it serve a sinister purpose?
Either Way, It’s Looking Great for ML!
Regardless of the obstacles and slow adoption in the ML landscape, ML is here to stay. And it isn’t slowing down any time soon.
This fascinating technology is introducing us to a whole new reality. One where innovation is abundant, where workflows are seamless and where our lives see constant improvements through AI.
“Machine Learning and Cloud technologies are two key aspects of the modern digital enterprise that really go hand-in-hand in allowing businesses to remain competitive in an evolving landscape today and, as such, Cloud adoption and ML adoption are driving one another forward very rapidly right now,” says Viljoen.
“All the major Cloud providers now have fully managed ML platforms which automate various mundane ML tasks and allow one to get models into production much faster than was possible in the past, with best practices also becoming much easier to implement due to the managed nature of these tools. Also, these Cloud ML platforms allow candidate ML use-cases to be rapidly prototyped and experimented with, without having to make any long-term up-front commitments”.
This Way For More on Machine Learning
Feel free to give Dom, Brett or Christiaan some love on LinkedIn!
