Machine Learning is the latest and most prominent buzzword to have found its way into pretty much every technology presentation I have attended in the last six months. However, the most common question asked is – what is Machine Learning and when should I use it? In this post, I aim to shed some light on this.
When it comes to Machine Learning we can break up a problem into the following different types: Rules Engine, Machine Learning and Deep Learning.
A Rules Engine is the more traditional form of computing when a developer can write explicit logical functions that consume, parse and output the data. A Rules Engine behaves exactly as it is written by the developers.
Machine Learning is where we have algorithms that parse the data, learn from the data and then output predictions based on what they have learnt. This approach does require guidance so is often referred to as “Supervised Learning” because the Machine Learning algorithms are provided with example cases where predictions have been made correctly. What this means is that when you are training Machine Learning, you need to provide a set of example outcomes. So, if you are using Machine Learning to determine when a photograph of a cat is submitted, you need to also provide images of cats and images that aren’t of cats – grouped into folders, so the algorithms know which is which. But over time, Machine Learning gets better and better at the task – as it learns more from data with guidance from users.
Deep Learning is actually a subset of Machine Learning, but it requires no guidance as the algorithms have the ability to validate the accuracy of its own predictions. If we have a case where Machine Learning predicts and makes a mistake, a developer or engineer is required to correct this – but in Deep Learning this is not needed. Deep Learning also has more potential to evolve to more complicated decisions such as text analysis where it can understand cues from text more deeply than traditional Machine Learning.
How do I decide if Machine Learning & Deep Learning works for my use case?
Is the problem solved by someone’s gut feeling or intuition? If people are answering the problem your application is trying to help with using gut feeling, then this is a strong indication that there is a case for Machine Learning – the senses of the human brain are what Machine Learning tries to replicate. A gut feeling usually means that a person has a range of inputs, and then makes a decision based on a correlation their brain is aware of.
Is there a statistically sound dataset that underpins the use case? Machine Learning works well if you have a clean dataset. It is very much a case of if you put data in that has no correlation or is noisy, it is likely that the Machine Learning model will learn the noise. Good data in equals good results out.
Is the data accessible and usable? Machine Learning needs the ability to access information easily – but it’s worth considering if the data can be accessed and if there are any conditions over the data, such as GDPR compliance or commercially sensitive information.
Does the problem have a large number of variables or does it follow an easy logic? Machine Learning can provide huge benefit predictions that require input from many parameters or unclear logical processes. If the problem can be broken down to a logical flow where parameters in the dataset are not too numerous and direct correlations can be made, then maybe a more traditional rules-based engine is a better fit. However, Machine Learning can be used on top or as a phase two to provide some real deep insight.
Do I have a large dataset? Machine Learning works better the larger the dataset and the more detail you can provide it whereas small datasets will show inherent bias and may have inconsistent results. The actual size of the dataset required depends on the use case and application. For instance, if you want products to be recognised then 50 images of each product are required. If I want to recognise sentiment, then you may need thousands of different statements for each classifier.
Is accuracy important for the use case? A traditional rules-based system is very quick to set up. Machine Learning is also becoming more and more commoditised with the ability of cloud-based services and training – although Deep Learning will take longer to deploy, as refining the process can take a while.
Will the use case evolve over time? Machine Learning can improve itself over time, so if there are variations in the dataset and evolution of the use case then Machine Learning can also keep up with the process. If the use case is a one-off report, then maybe a data scientist engagement would be best suited.
The more yes answers to the above questions mean the more Machine Learning and even Deep Learning can provide an answer and insight into your use case. Machine Learning has huge potential to make massive differences and to make applications smarter and more engaging. At Keytree we have a wealth of Machine Learning expertise and have found it to be a rapidly growing area of business. If you would like to find out more, get in touch so we can support and advise on your use case.