Image may be NSFW.
Clik here to view.
Being two popular machine learning frameworks, TensorFlow and Theano are used extensively by researchers in the deep learning domain, and more often than not, are compared for their popularity, ease of use, technological benefits and much more. While TensorFlow was developed by researchers and engineers working on Google Brain Team a year ago, Theano is headed by Yoshua Bengio, University of Montreal’s MILA and has been in existence since 2007. Though Theano is considered effectively dead with the team announcing in September 2017 that they would cease all its major development after the release of 1.0 version, many experts in the area of deep learning still depend on Theano.
An open source software library to carry out numerical computation using data flow graphs, the base language for TensorFlow is C++ or Python, whereas Theano is completely Python based library that allows user to define, optimize and evaluate mathematical expressions evolving multi-dimensional arrays efficiently, as per their website.
Existing for over a year in the industry, TensorFlow has become one of the most widely adopted open source library for performing fast gradient based machine learning on GPUs, and has a flexible architecture allowing users to deploy computation to one or more CPUs and GPUs in a desktop or mobile.
Theano on the other hand can be deployed on single GPU and allows transparent use of GPU to perform data intensive computations. It integrates with Numpy allowing efficient symbolic differentiation. It has been powering large scale computationally intensive investigations quiet efficiently.
Having sifted through some of the key points defining these two, lets understand the differences between them, and which is the one that researchers are more likely to use over the other.
a) Popularity:
TensorFlow, hands down gains a point here, as it is better in terms of marketing and its recent surge in popularity has made it one of the most popularly used libraries, defeating the long time players of the open-source market such as Theano and Torch. It boasts components that are visually appealing with many users believing that TensorFlow has better computational graph visualisations, whereas Theano may not be as fun to look at. However, for visualising convolutional filters, images, and graphs, Theano is just as good.
What makes TensorFlow win a brownie point is the fact that it has support from one of the leading tech giants, Google, whereas up until now Theano was being actively developed and maintained by University of Montreal’s MILA, with more than 50 members, contributing their time to improve it constantly.
b) Technology:
Many experts believe that TensorFlow is like a complete re-implementation of Theano and that Google created TensorFlow to replace Theano. Even then the latter is considered faster than TensorFlow in many respects. Thenao also supports wider range of operations and while TF has shown promise in this aspect, there is still time before it reaches at par with Theano. The two libraries are in fact quite similar with some of the creators of Theano, such as Ian Goodfellow, went on to create Tensorflow at Google before leaving for OpenAI.
Both of them generate a computational graph and perform automatic differentiation. This prevents the need to have to hand code new variation of backpropagation every time a researcher is experimenting with a new arrangement of neural networks. However, unlike other frameworks in deep learning, Theano computes the gradient when determining the backpropagation error by deriving an analytical expression. This eliminates accumulation of error during successive derivative calculations.
TensorFlow is offering users a great amount of documentation for installation and learning materials aimed at helping beginners understand the theoretical aspects of neural networks and help in setting it up. Tensorflow also has the ability to do partial sub graph computation, which is not offered in other frameworks. It is worth noting that one of the Theano frameworks, Keras, supports TensorFlow.
Advantages of TensorFlow:
- It supports reinforcement learning and other algorithms
- Offers computational graph abstraction
- Has a faster compile time than Theano
- Offers TensorBoard for visualisation
- Offers data and model parallelism
- It can be deployed on multiple CPUs and GPUs
Drawbacks of TensorFlow:
- It doesn’t support matrix operations, making copying these large matrices a costly affair
- It runs dramatically slower than other frameworks
- It doesn’t have pertained models
- Computational graph can be slow
- It is not commercially supported
- Drops out to Python to load each new training batch
- Not very toolable
- Dynamic typing is error-prone on large software projects
Advantages of Theano:
- Open-source deep-libraries such as Keras, Lasagne and Blocks have been built on top of Theano
- Computational graph is nice abstraction
- Raw Theano is somewhat low-level
- It has some high level wrappers such Keras, Lasagne which increases it usability
Drawbacks of Theano:
- It can be troublesome on AWS
- It can be deployed on single GPU
- Large models can demand long compile times
- Error messages doesn’t help much in debugging
- Much fatter than Torch
On a concluding note:
It can be said that both the APIs interfaces are more-or-less similar and share some design characteristics, however TensorFlow’s API is easier to use and read. The latter also provides helpful operators, monitoring tools and debugging tools, which lacks in Theano. When it comes to usability and speed, Theano is simpler to use and faster than TensorFlow, while TF is better for deployment. Users also believe that TensorFlow is more consistent than Theano, but TF being a recent introduction, users have to be extensively dependent on papers and presentations to help them better understand its usage.
The post Tensorflow Vs. Theano: What Do Researchers Prefer As An Artificial Intelligence Framework appeared first on Analytics India Magazine.