When using or deploying any technology, we always face the decision about which tools we should use for the job of Deep Learning. Similar to the approach taken by big players such as Google and Facebook when choosing JavaScript libraries or Java frameworks – as they each have competing open source libraries.

Which Machine Learning libraries will give you the biggest benefit and the best future proofing – considering we want something that is easy to implement and also something that is well supported with a good community. There is quite an extensive list of Machine Learning libraries, so here is my comparison of some of the big names.

Will Powell

Will Powell

Head of Innovation

Caffe – In my humble opinion, this is the grandfather of Machine Learning libraries. Here we have a python Machine Learning library written in C++ that includes some of the broadest levels of support for convolutional neural networks and computer vision. I first came across Caffe when using OpenCV – the widely used computer vision library which added Deep Learning models using Caffe. If you are using OpenCV then Caffe is the obvious choice.

SAP Leonardo Machine Learning Foundation – This is a full suite of business-focused Machine Learning capabilities, and it runs within the SAP Cloud Platform environment. It is a very powerful offering with Image Processing, NLP and brings its own model capabilities. The key benefits are first, you can start using the service without any ML knowledge or expertise as there are out of the box examples and a simple API to get access, but you can go lower down the stack to implement custom models. Secondly, SAP Cloud Platform is easy to integrate into the rest of your enterprise landscape and access the data the Machine Learning depends on using for training. Overall this is a strong contender for any Machine Learning considerations.

PyTorch – Developed by Facebook’s Artificial Intelligence team and built on Torch – this is a python open source Machine Learning framework and dynamic graph, which is very suitable for use cases in language processing. PyTorch has been growing in popularity, especially within the academic field, due to that fact that it is incredibly easy to use, implement and work with. Although it was slightly held back by the lack of production scalability deployment mechanisms, this has now been answered by its merge with Caffe2. Overall this is a great framework for rapid prototyping.

TensorFlow – Developed by Google, TensorFlow is probably the most popular Machine Learning framework out there and has over five times the number of stars on GitHub compared to PyTorch. As a framework, it is very strong in distributed training, production scalability and deployable on mobile devices such as Android. Deployment is delivered using TensorFlow Serving which is an incredibly resilient and scalable platform to deploy models on – and has great features like “hot swap” and is even used by Google themselves for their own deployment. Also in the training process, refining the hyperparameters and identifying any issues is made easy with TensorBoard visualisation tools.

Theano – This is a very powerful python library designed for highly optimised executions of mathematical expressions. Theano is more low level than some of the above, but it can achieve great results with a steeper learning curve – the only caveat is that the framework is no longer being developed by MILA, so future proofing is a concern.

Keras – Originally developed off the back of Theano, to enable quicker development and easier understandings, Keras is the wrapper around Theano, which has since been expanded out to Machine Learning technologies (Tensorflow, CNTK and Theano). This wrapper design does put Keras in a particularly strong position, as it unifies underlying technologies. It also removes much of the boilerplate from your development making your development quicker and nimbler. Keras is also really simple to use when it comes to computation on CPU and GPU. I have seen this commonly used as a Keras/TensorFlow combination, as Keras has far less work to develop in, so overall – the community is strong with about a third of TensorFlow’s stars on GitHub.

TuriCreate – Developed by Apple this is a python Machine Learning library that is very simple to use allowing easy creation of some complex use cases like image classification, object detection and sentiment analysis – without really knowing about Machine Learning underneath. TuriCreate uses a lot of transferred learning mechanisms meaning that learning times are very rapid. For instance, its image classifiers use Resnet or Squeeze image references for rapid training, which exports models that are tightly integrated into Apple MacOS and iOS as it leverages CoreML execution and exports as MLModel files.

With this tie to Apple OS, there are elements within the OS that provide optimisations and size of models files are vastly different from the other Machine Learning libraries. For instance, an image classifier on TensorFlow could be 100MB per classifier whereas the CoreML Model file maybe 10KB per classifier, which is a huge benefit for mobile use. TuriCreate is also baked into XCode 10 and can even be used within Playground using MLImageClassifierBuilder for a drag-drop interface to build Machine Learning models.

Caffe2 – Also developed by Facebook this went some way to help PyTorch deliver the scalability and productive deployment methodologies, and was merged into PyTorch.

scikit-learn is a general purpose python library providing a wide range of out-the-box algorithms for clustering, regression, classification, feature selection and evaluation. It has a rich history as the de-facto python Machine Learning framework and is remarkably consistent. This makes swapping algorithms in and out simple with very little code changes, making it ideal for fast prototyping and development. The plug and play approach really helps accelerate the evaluate, tune, deploy workflows common to most Machine Learning projects.

If scikit-learn has an achilles heel it might be its breadth of support for NNs. Perhaps for this reason, and the emergence of TensorFlow in 2015 scikit-flow was developed. SkFlow provides the familiar scikit-learn fit/predict and learn/test programming model on top of TensorFlow graphs and sessions. SkFlow ultimately became TensorFlow, which is possibly why scikit-learn is still so popular – the framework can be used to work with the traditional scikit-learn algorithms as well as Tensorflow NNs and higher level frameworks like Keras.

So, in summary, there are a lot of libraries out in the machine learning world. If you are prototyping then Keras, TensorFlow or PyTorch are very strong and powerful.

For me, TensorFlow still has the edge on production deployment but it is a very tightly run thing – with TensorFlow and TuriCreate if you are looking at Machine Learning on Android or iOS respectively.