Memristor-based Hardware Neural Networks Modelling Review and Framework Concept

This paper is a report of a study in progress that considers development of a framework and environment for modelling hardware memristor-based neural networks. An extensive review of the domain has been performed and partly reported in this work. Fundamental papers on memristors and memristor related technologies have been given attention. Various physical implementations of memristors have mentioned together with several mathematical models of the metal-dioxide memristor group. One of the latter has been given a closer look in the paper by briefly describing model’s mechanisms and some of the important observations. The paper also considers a recently proposed architecture of memristor-based neural networks and suggests enhancing it by replacing the utilized memristor model with a more accurate one. Based on this review, a number of development requirements was derived and formally specified. Ontological and functional models of the domain at hand have been proposed to foster understanding of the corresponding field from different points of view. Ontological model is supposed to shed light onto the object-oriented structure of memristor-based neural network, whereas the functional model exposes the underlying behavior of network’s components which is described in terms of mathematical equations. Finally, the paper shortly speculates about the development platform for the framework and its prospects.


Introduction
Until 1970-s the world has been aware of only three passive elements of electrical circuitry: resistors, capacitors and inductors.The three stated elements coupled with natural relationships provide five connections for four basic notions of electrical circuit theory (voltage, charge, current and flux).Mathematics, however, claims that four things can be mutually interconnected in six different ways.Indeed, the relation between charge and flux was not present.It wasn't until 1971 that the discordance has been formulated and solved.A new element -memristor -has been proposed by Leon Chua in his paper in IEEE Transactions on Circuit Theory completing the mathematical symmetry of circuit theory.It took nearly 40 years for memristor to transform from a purely theoretic concept into feasible implementation.In 2008 a group of scientists from Hewlett-Packard Labs lead by Stan Williams has finally built working memristors [1].One of the most promising domains of memristor application, seem to be artificial neural networks [2].These often come in either software or hardware implementations, sometimes in a combination of both.While digital neural networks simulate the data processing mechanism of biological neural networks, hardware ones strive to emulate it.It is worth mentioning that since most of computer architectures conform to the von Neumann architecture, neural network simulation becomes a challenging task because of the paradigm mismatch.Instead of simulating the ways of nature, hardware neural networks try to directly replicate them, creating non-von-Neumann architectures.In comparison with digitally simulated networks, hardware ones can achieve better speed, less power consumption and chip space.On the other hand, hardware networks often prove to be far less accurate that their software counterparts, due to the nonuniformity of analog components [3].Another disadvantage of modern hardware neural networks, which they actually share with the software ones, is the volatile storage of synaptic weights.There are ways to achieve the nonvolatile weight storage within hardware networks, but usually such weights are either static (cannot be changed once manufactured), quickly digress (require frequent updating) or are rather hard to program [4].The emergence of memristor, however, seems to have opened new possibilities in addressing the stated problems.Memristors seem to be a perfect match for synapses, making hardware implementations of neural networks more reliable and greatly increasing productivity of neural computations [5].Nevertheless, memristors are still scarcely availably and lack industrial-grade production.Being such a new technology, they are often hard and expensive to acquire for experimentation, but a large variety of memristor models has already been produced, making it possible to model memristor-based devices.Thus, considering the domain of artificial intelligence, a need in profound and correct model of artificial memristor-based feedforward neural network arises.Such model would be of great help in assessing the qualities of modeled system: computation performance, time and energy expenses, material costs, etc.Consequently, the goal of the research is to develop a framework for modelling artificial memristor-based neural networks.

Theoretical Memristor
The concept of memristor has been recognized since 1971, when Leon Chua has proposed for the first time in a well-organized and mathematically described way [6].The 1971 Chua's paper in IEEE Transactions on Circuit Theory is considered to be the pioneer work in the corresponding field of research.Although, the concept of memristor-like devices has been suggested earlier in 1960 by Bernard Widrow, Leon Chua was the first one not only to provide a feasible foundation for memristor's existence, but also to estimate and mathematically describe its' supposed behavior and properties.Memristor fulfills the mathematical symmetry of relationships between major circuit notions.The relationship created by a memristor, according to Chua, is expressed as follows: , where M(q(t)) is the memristance defined as The definition of memristance may be represented in a more convenient form by substituting flux and charge with their integral definitions: The similarity of memristor to the remainder of classical circuit elements can be better reflected by expressing their definitions via differential equations as it is done in fable 1.The first important property of memristors, which commonly is referred to as memristance and stands for the ability to change its resistance gradually via a controlled mechanism (e.g.memory of device's history of charge).
The second significant attribute of memristors, figured out by Chua, is the nonvolatility property, which stands for the absence of internal power supply.In other words, Chua proposed that memristor is able to store the value of own resistance without the need to be connected to a power source.
In 1976, Leon Chua and his fellow colleague Sung Kang proceeded exploring the mathematical and physical properties of the memristor [7].
They had come to an understanding, that since memristor is a dynamic device, one equation is not enough to describe it, henceforth memristor's behavior is represented by following equations for current-controlled memristor , , ! " , , and for voltage-controlled one , !, !" , !,

Fig. 1. Pinched hysteresis loop in the i-v curve
where v and i denote the input voltage and current respectively and x stands for the internal state of the device.In their paper, Chua and Kang also provided a more generalized concept of memristive systems with no specific reference to particular physical variables.One noteworthy peculiarity derived from these equations is that regardless of the state x (which implements the memory effect), the output voltage is equal to zero whenever input voltage or current are equal to zero as well.This zero-crossing property, Chua and Kang write, manifests itself vividly in the form of a Lissajous figure, which always passes through the origin.Thus, they extended the definition of memristor that is now to encompass any system able to demonstrate a Lissajous figure (later called pinched hysteresis loop by Chua) in the i-v curve, which is presented on fig. 1.

Memristor Models
However, the true interest has been sparked by the notable work of Richard Stanley Williams' group of researchers at Hewlett-Packard laboratories.Despite this fact, the idea of memristors not being a purely theoretical concept has captivated minds of many researchers around the world, resulting in more than 120 publications about memristors and memristive systems by 2011.[8].
After the concept of memristor was brought back to the public's sight, several implementations of memristors and memristive systems have been proposed.Different implementations of memristor rely on various physical and chemical reactions that give rise to both memristance and nonvolatility, properties essentially constituting the definition of memristor.There have been reported polymeric [9,10], spintronic [11], ferroelectric [12] and layered [13] implementations of memristor, but titanium dioxide memristors remain the most well studied group.During this research four models were closely considered, namely linear ion drift model [1], nonlinear ion drift model [14], Simmons tunnel barrier mode [15], and threshold adaptive memristor model (TEAM) [16].Unfortunately, due to the paper size considerations only the last one of them will be reported.

Memristor Bridge Neural Network
This paper considers the neural network architecture proposed by Adhikari et al. in 2012 [4].The architecture is based on the memristor-bridge synapse [17] and aims to solve the issue of nonvolatile synaptic weight storage and implement a newly proposed hardware learning method.

Memristor Bridge Synapse
Memristor bridge synapse architecture was first proposed in [17], it is a Wheatstonebridge-like circuit that consists of four identical memristors of opposite polarities.When positive or negative strong pulse vin (t) is applied at the input, the memristance of each memristor is increased or decreased depending upon its polarity.Kim et al. write, that if input pulse voltage is equal to vin, voltages at memristors can be calculated according to "voltage-divider formula".Then given memristances M1, M2, M3, and M4 stand for the corresponding memristors at time t, the output voltage is reported to be equal to the voltage difference between terminals A and B:

Memristor Bridge Neuron
In artificial neural networks neurons are required to sum a set of input postsynaptic signal and, according to the activation function, propagate (or not propagate) the signal further on to the next layer of the network.The neuron is then required to sum the input postsynaptic signals.Kim et al. point out, that the signal summing operation is easier to be performed in current mode: postsynaptic signals should be connected to a single node, so that the following neuron would receive the sum of currents via Kirchhoff's current law.In order to achieve current summation, the memristor bridge synapse has to be modified because it provides voltage output.Kim et al. suggest combining the memristor bridge with differential amplifier.The latter converts postbridge negative and positive voltage into corresponding currents.Hence, for a set of synapses there exist two nodes: one for positive postsynaptic current and one for negative postsynaptic current.These nodes sum the output currents of each individual synapse in the set.Neuron itself is then comprised of the summation nodes, but also of the active load circuit that implements the activation function as in fig. 2. The sum of all postsynaptic currents is converted back to voltage (presynaptic signal for next layer of neural network) by the active load circuit according to the activation function.
In their paper, Adhikari et al. also provide rigorous mathematical explanation of the suggested architecture behavior.

Neural Network Training
A composition of an arbitrary number of neurons connected via memristor-bridge synapses therefore constitutes the artificial network.

Framework Concept
As one can see, plenty of research has been carried out in the field of memristors and memristor-based neural networks.Multiple approaches to both creating and modelling memristors have been mentioned in previous sections.
It is needed to create a reliable framework for simulating memristor-based neural networks.So far, rather abundant overview of the domain has been presented.Despite the vast variety of works mentioned, the domain at hand lacks general integrity and is not formalized enough to start composing the framework at least in its basic form.Hence, the domain must be formalized to a certain extent.In order to derive this degree of formalization, the requirements for the stated framework are to be determined.This will enable framework to be designed properly and will ensure it complies with the needs and wants of its users.Requirements are decided to include four major points: accuracy, performance, flexibility, and explicitness.Accuracy stands for reliability of framework and if its output data can be trusted.Performance reflects how quick does the simulation proceed.Flexibility corresponds to how easy it is to swap components and models within framework.Finally, explicitness is determined by the overall convenience of the framework and how well does it represent results of the simulation.Insights into these requirements can be better revealed according to the SMART criteria (a project management technique for elaborating objectives), which is done in Table 2.The requirements described above help determine what is to be expected from the framework, what kind of formalization for the domain is required, and set guidelines for further process of design and development.The domain may be formalized by representing it as a graphical scheme, henceforward called ontological model.The reason for such naming is that this model encompasses relevant entities of the domain under discourse, as well as reflects their major properties and interrelations, which in turn roughly corresponds to the definition of ontology.This model will limit the complexity of the field of memristor-based neural networks and expose the intrinsic connections between the notions at hand.First, let us derive a set of entities to be found within this model.At the very core of every network there are neurons and synapses.Flexibility must be ensured from the very beginning of the development.
Visualization may be introduced after the basis of the framework is complete.
Both neurons and synapses of hardware neural networks are implemented through Both neurons and synapses of hardware neural networks are implemented through circuits.Circuit design may vary from one implementation to another, therefore, the general concept of neurons and synapses should be decoupled from its' particular hardware implementation to ensure flexibility.This will enable the framework to safely switch between specific circuit implementations of neurons and synapses, but will also ensure framework's operability.The framework must as well be able to switch between different realizations of memristor, namely, memristor models.Hence, the latter should be considered a separate entity, which is contently used as a component in synapse circuitry.For the time being only the metal dioxide class of memristors is considered to limit already reasonable complexity of the framework.
Finally, the network must should be able to employ different learning techniques.Despite the fact that this work considers only chip-in-the-loop method, the framework should be designed being able to implement various ways of network training.Here it is necessary to take into account not only the learning algorithm, but also how this algorithm is applied to hardware circuit components of the network.
The ontological model is depicted on fig. 3. Solid border circles correspond to the entities of the domain; dashed border circles stand for the properties (attributes) of certain entities; filled arrows represent association relation between entities; empty arrows reflect inheritance (or, possibly, interface implementation); finally, dashed lines reflect attribution connections.It must be noticed, that the ontological model is likely to be changed in the following works and presented version is not final.Some of the anticipated issues include particular implementations of learning techniques, for instance, chip-in-the-loop does not require auxiliary circuitry, whereas spike timing-dependent plasticity usually does.Another bottleneck to be expected relates to the circuit implementations of neurons and synapses.The latter may consist of multiple circuits that should be represented as separate entities in order to preserve flexibility of the system, yet should conform to the same interface for the sake of integrity.
In this way we shed light onto the structural peculiarities of the future framework.This model is to help composing the classes to be implemented as well as their interrelations.Let us now consider the other side of the developed system, namely, its functional requirements.In this paper, the latter refer to a certain number of capabilities expected by users from the framework.
Framework under development strives to model memristor-based neural network suggested by Adhikari et al., which is described in the previous section.It is also expected to make possible modeling with better level of preciseness by enabling swappable memristor models.For instance, employing TEAM memristor model may significantly raise the relevance of proposed hardware neural network model through fostering the accuracy of memristor's physical model.
The functional scope of the framework may be represented as a set of intertwined mathematical equations that describe various parts of the network model.Each entity of the framework can be characterized with equations that have adjustable parameters, which are usually derived by the authors of corresponding models from experimental data analysis.These equations are extracted from relevant models and are bound in such way, that one equation's output usually corresponds to input of the other equation.This set of equations is depicted on fig. 4.

Conclusion and Prospects
In this paper, a range of memristor models has been reviewed together with some of the fundamental papers on memristor-related technologies.Based on this review, a concept of framework for modeling memristor-based hardware neural networks has been proposed.This framework represents an implementation of neural network architecture considered in the paper, but implies ability to swap memristor models in order to increase the overall flexibility and, possibly, relevance of models generated with the help of proposed framework.The ability to switch between model is also expected to help comparing suggested implementations.In the process of framework structure discovery a set of criteria has been formulated to assess the future software product, domain of memristor-based neural networks has been formalized to a certain extent, and, finally, the framework has been given a functional structure strictly defining its' capabilities.Specific platform for framework implementation is yet to be chosen.As of current state of affairs, Unity engine is expected to be the most favorable candidate.Its architecture perfectly fits the nature of soft simulation (which the framework ultimately represents), providing some software patterns that greatly alleviate the development.Considered engine is also able to realize extensive visualization of models as well as equip them with user-friendly interface to further enhance model explicitness and facilitate employment of the future framework for academic purposes.Finally, implementing a circuit simulation framework in Unity also pursues an exploration goal, since such attempts have not been previously well studied.
Adhikari et al. intend to useChip-in-the-Loop technique for training the network of proposed architecture.They, however, suggest modifying this technique slightly in order to take into account peculiar properties of memristor-based circuits.This technique is a viable choice since it provides a way to deal with memristor bridge non-idealities without explicitly modelling these nonidealities.According to this technique, the circuit performs the forward computation of the network, whereas back-propagation and weight update is done on the software side.The hardware circuit network is reproduced by a software clone, which is used to process the training data.After the computer network has processed all the training data, synaptic weights of each individual synapse circuit are programmed by direct application of strong voltage pulses in order to match with the weights from computer network's weight matrix.Hence, the whole of the hardware network is treated as it consists of a set of simple single-layer networks.Each one of those single layer networks is trained separately, according to the weight matrix.Because of the nature of memristor bridge synapses, the need in additional circuitry is eliminated.

Table 1 . Differential equations of basic circuit elements Device Electronic Symbol Unit Differential equation
report the results of comparison between the fitted TEAM and Simmons tunnel barrier model.The feasible preciseness of TEAM model was proved by the average discrepancy between models' state variable difference of only 0.2%.The maximum difference of this value constituted 12.77%, however the run time of the model was nearly halfed (47.5%)Kvatinsky et al. had been also able to fit the model with different types of physical memristor models, namely STT-MRAM and Spintronic memristors.
This model, however, was decided to be further utilized throughout the work.TEAM model, proposed by Kvatinsky et al., incorporates advantages of ion drift models' explicitness and Simmons tunnel barrier accuracy, yet manages to preserve relatively high computational performance and generalizability.TEAM model is based on the same physical behavior as Simmons tunnel barrier model.But it manages to convey it with simpler mathematical functions.The model introduces several assumptions for the sake of analytical simplicity: state variable does not change below a certain threshold and exponential dependence is replaced with a polynomial one.Detailed mathematical foundation of the model may be found in the corresponding paper.A major advantage of such a relation is the explicitness of current and voltage relationship as opposed to the Simmons tunnel barrier model.Nevertheless, Kvatinsky et al. were able to perform a fitting procedure forcing TEAM model to match the latter with reasonable and sufficient accuracy.In their paper, authors of 248 TEAM model also

Table 2 .
These three notions (neural network, synapse and neuron) constitute the heart of designed model as well.Multilayer network usually distinguishes between input layer neurons, output layer neurons and hidden layer neurons, which may slightly differ.Input neurons should be able to receive input signals, which may not necessarily coincide with how the signals are conveyed within the network.Similarly, output neurons must provide output signals.Consequently, input and output program modules should be introduced, in order to convert electrical output signals into human-comprehensible format and vice versa for the input signals.Framework requirements according to SMART