With Machine Learning (ML) advancing its frontiers day by day, developing base algorithms to fulfill the needs of self-learning in machines is a challenging task. In meta-learning, as the algorithms encounter a horde of data, they become more prone to providing useful output rather than giving out incomprehensible output. One such algorithm developed by OpenAI, a non-profit artificial intelligence (AI) research company, is Reptile— a meta-learning algorithm designed to perform over a wide array of tasks.
Why Meta-learning is necessary?
In ML, the learner algorithm assumes a set of statements that determines output based on input. If the input digresses from the usual and finds something new or strange, then the output returned is vague. This is called learning or inductive bias since the algorithm is not tweaked into incorporating many data types or range. For example, if a neural network is built over a specific data range, the algorithm produces meaningful output that falls within the data range or type. It may not function if a different type of input is fed.
This is where meta-learning comes into play. The level of learning is dynamically improved with the increase in number and quality of input data with rigorous variations. For instance, one may increase factors such as training-set size, sample distribution among other things to achieve an efficient learning algorithm. Ultimately, the aim of meta-learning is to decide how components of algorithms interact with each other and then act on improving it. Also, meta-learning helps new algorithms go beyond the scope of existing algorithms.
Meta-learning is nothing but learning to learn. It is the process to choose the best strategy among the assortment of tasks present in an algorithm. The performance of the algorithm will improve over time with meta-learning.
How Reptile works?
Reptile is slightly based on another algorithm called Model-Agnostic Meta-learning(MAML) developed by researchers at University of California, Berkeley in collaboration with OpenAI. MAML uses step-by-step approach of training parameters for a gradient descent associated model. This way every step will serve as a training data for any new task encountered in the model. Reptile differs in this aspect. It utilises the stochastic gradient descent to initialise the parameters of the model instead of performing several computations which are resource-consuming, which is evident in MAML. This significantly reduces the dependency of higher computational, hardware requirements if implemented in a ML project. SGD is an improved form of the standard gradient descent.
The difference among the former and the latter lies in the update made in the parameters. The equations are given below:
where,
= parameters for the project
= learning rate
= gradient descent with respect to parameters
E = Error
= Output presented for the parameters
The stochastic variation is presented like this:
Reptile follows this variation for its working. Although, the code for the algorithm may look a bit different, nevertheless the logic remains the same. The initialisation is performed either using Omniglot (for one-shot learning) and Mini-Image.Net (for few-shot learning).The algorithm is also tested using the Taylor Series. The reason being the product between the gradients related to similar tasks shows improvement in generalisation.
Reptile is compared for its usefulness with another algorithm called the First-Order Model-Agnostic Meta-Learning (FOMAML). It is mathematically similar to Reptile except that it uses the standard gradient descent and the update has a gradient which stays constant throughout iterations.The key contrasting feature between these two algorithms is the reduction in variance shown below.
The above image shows the different gradients — represented by the letter ‘g’ obtained using Omniglot. They are combined to form meta-gradients to facilitate quicker learning by the algorithm. For demo purposes, the platform used by OpenAI to showcase Reptile is Tensorflow, owing to its flexibility in terms of ML and mathematical functions associated to build a ML project.
Another important aspect is that Reptile is built using ‘shortest descent’ method of minimizing obstruction when learning anything new. This is possible by implementing a technique which incorporates memory points — known as fast weights and slow weights. The former stores long-term data while the latter stores temporary and pivotal new data. This forms a connectionist model where the learning is based on, and is similar to human cognition and behavior. With this, the fast weights compensate for the slow weights which sometimes will be a hindrance to the functioning of the algorithm.
Even though Reptile and FOMAML have similarities in computing algorithm updates, they have a different take when it comes to addressing a problem. One such example is one-shot and few-shot techniques for optimizing random parameters during the process.
The post Understanding Reptile: A Scalable Meta-learning Algorithm By OpenAI appeared first on Analytics India Magazine.