http://iet.metastore.ingenta.com
1887

Enhanced memristor-based MNNs performance on noisy dataset resulting from memristive stochasticity

Enhanced memristor-based MNNs performance on noisy dataset resulting from memristive stochasticity

For access to this article, please select a purchase option:

Buy article PDF
$19.95
(plus tax if applicable)
Buy Knowledge Pack
10 articles for $120.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Name:*
Email:*
Your details
Name:*
Email:*
Department:*
Why are you recommending this title?
Select reason:
 
 
 
 
 
IET Circuits, Devices & Systems — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

Multilayer neural networks (MNNs) have achieved excellent performance in machine-learning domain. Memristors are a possible device for implementing MNNs in hardware with efficiency and limited area. In this work, a simple model of stochastic memristors was presented first. Then, an MNN architecture based on proposed memristor model was presented. The simulation processes on stochastic memristors were elaborated. The simulation demonstrates that the MNN classification accuracy based on stochastic memristors is usually higher than that based on deterministic memristors when the dataset noise is low. The results have significant meaning to develop analogue memristive devices or memristive chips for MNN applications.

References

    1. 1)
      • 1. LeCun, Y., Bengio, Y., Hinton, G.: ‘Deep learning’, Nature, 2015, 521, (7553), pp. 436444.
    2. 2)
      • 2. Jouppi, N.P., Young, C., Patil, N., et al: ‘In-datacenter performance analysis of a tensor processing unit’. Proc. 44th Annual Int. Symp. Computer Architecture, Toronto, Canada, June, 2017, pp. 112.
    3. 3)
      • 3. Chua, L.: ‘Memristor-the missing circuit element’, IEEE Trans. Circuit Theory, 1971, 18, (5), pp. 507519.
    4. 4)
      • 4. Strukov, D.B., Snider, G.S., Stewart, D.R., et al: ‘The missing memristor found’, Nature, 2008, 453, (7191), p. 80.
    5. 5)
      • 5. Yang, J.J., Zhang, M.X., Strachan, J.P., et al: ‘High switching endurance in TaOx memristive devices’, Appl. Phys. Lett., 2010, 97, (23), p. 232102.
    6. 6)
      • 6. Soudry, D., Di Castro, D., Gal, A., et al: ‘Memristor-based multilayer neural networks with online gradient descent training’, IEEE Trans. Neural Netw. Learn. Syst., 2015, 26, (10), pp. 24082421.
    7. 7)
      • 7. Borkar, S., Chien, A.A.: ‘The future of microprocessors’, Commun. ACM, 2011, 54, (5), pp. 6777.
    8. 8)
      • 8. Hasan, R., Taha, T.M., Yakopcic, C.: ‘On-chip training of memristor crossbar based multi-layer neural networks’, Microelectron. J., 2017, 66, pp. 3140.
    9. 9)
      • 9. Zhang, Y., Wang, X., Friedman, E.G.: ‘Memristor-based circuit design for multilayer neural networks’, IEEE Trans. Circuits Syst. I-Regul. Pap., 2018, 65, (2), pp. 677686.
    10. 10)
      • 10. Yu, S., Guan, X., Wong, H.S.P.: ‘On the switching parameter variation of metal oxide RRAM–part II: model corroboration and device design strategy’, IEEE Trans. Electron. Devices, 2012, 59, (4), pp. 11831188.
    11. 11)
      • 11. Gaba, S., Sheridan, P., Zhou, J., et al: ‘Stochastic memristive devices for computing and neuromorphic applications’, Nanoscale, 2013, 5, (13), pp. 58725878.
    12. 12)
      • 12. Medeiros-Ribeiro, G., Perner, F., Carter, R., et al: ‘Lognormal switching times for titanium dioxide bipolar memristors: origin and resolution’, Nanotechnology, 2011, 22, (9), p. 095702.
    13. 13)
      • 13. Ambrogio, S., Balatti, S., Cubeta, A., et al: ‘Statistical fluctuations in HfOx resistive-switching memory: part I-set/reset variability’, IEEE Trans. Electron. Devices, 2014, 61, (8), pp. 29122919.
    14. 14)
      • 14. Hu, M., Wang, Y., Wen, W., et al: ‘Leveraging stochastic memristor devices in neuromorphic hardware systems’, IEEE Trans. Emerging Sel. Top. Circuits Syst., 2016, 6, (2), pp. 235246.
    15. 15)
      • 15. Al Shedivat, M., Naous, R., Cauwenberghs, G., et al: ‘Memristors empower spiking neurons with stochasticity’, IEEE Trans. Emerging Sel. Top. Circuits Syst., 2015, 5, (2), pp. 242253.
    16. 16)
      • 16. Naous, R., Al Shedivat, M., Salama, K.N.: ‘Stochasticity modeling in memristors’, IEEE Trans. Nanotechnol., 2016, 15, (1), pp. 1528.
    17. 17)
      • 17. Tuma, T., Pantazi, A., Le Gallo, M., et al: ‘Stochastic phase-change neurons’, Nat. Nanotechnol., 2016, 11, (8), pp. 693699.
    18. 18)
      • 18. Averbeck, B.B., Latham, P.E., Pouget, A.: ‘Neural correlations, population coding and computation’, Nat. Rev. Neurosci., 2006, 7, (5), p. 358.
    19. 19)
      • 19. Knag, P., Lu, W., Zhang, Z.: ‘A native stochastic computing architecture enabled by memristors’, IEEE Trans. Nanotechnol., 2014, 13, (2), pp. 283293.
    20. 20)
      • 20. Jiang, H., Belkin, D., Savel'ev, S.E., et al: ‘A novel true random number generator based on a stochastic diffusive memristor’, Nat. Commun., 2017, 8, (1), p. 882.
    21. 21)
      • 21. UCI machine learning repository’. Available at http://archive.ics.uci.edu/ml, accessed January 2019.
    22. 22)
      • 22. Chua, L.O., Kang, S.M.: ‘Memristive devices and systems’, Proc. IEEE, 1976, 64, (2), pp. 209223.
    23. 23)
      • 23. Kvatinsky, S., Friedman, E.G., Kolodny, A., et al: ‘TEAM: threshold adaptive memristor model’, IEEE Trans. Circuits Syst. I-Regul. Pap., 2013, 60, (1), pp. 211221.
    24. 24)
      • 24. Kvatinsky, S., Ramadan, M., Friedman, E.G., et al: ‘VTEAM: A general model for voltage-controlled memristors’, IEEE Trans. Circuits Syst. II-Express Briefs, 2015, 62, (8), pp. 786790.
    25. 25)
      • 25. Zhang, Y., Wang, X., Li, Y., et al: ‘Memristive model for synaptic circuits’, IEEE Trans. Circuits Syst. II-Express Briefs, 2017, 64, (7), pp. 767771.
    26. 26)
      • 26. Li, C., Hu, M., Li, Y., et al: ‘Analogue signal and image processing with large memristor crossbars’, Nat. Electron., 2018, 1, (1), p. 52.
    27. 27)
      • 27. Bae, W., Yoon, K.J., Hwang, C.S., et al: ‘A crossbar resistance switching memory readout scheme with sneak current cancellation based on a two-port current-mode sensing’, Nanotechnology, 2016, 27, (48), p. 485201.
    28. 28)
      • 28. LeCun, Y.A., Bottou, L., Orr, G.B., et al: ‘Efficient backprop’ (Springer, Berlin, Heidelberg, 2012).
    29. 29)
      • 29. Ruder, S.: ‘An overview of gradient descent optimization algorithms’, arXiv preprint arXiv:160904747, 2016.
    30. 30)
      • 30. Simard, P.Y., Steinkraus, D., Platt, J.C., et al: ‘Best practices for convolutional neural networks applied to visual document analysis’. Proc. Document Analysis and Recognition, Edinburgh, UK, 2003, pp. 958963.
    31. 31)
      • 31. Nielsen, M.A.: ‘Neural networks and deep learning’ (Determination Press, San Francisco, CA, USA, 2015).
    32. 32)
      • 32. Çalişkan, A., Cevik, U.: ‘An efficient noisy pixels detection model for CT images using extreme learning machines’, Tehnčki vjesnik, 2018, 25, (3), pp. 679686.
    33. 33)
      • 33. Pedregosa, F., Varoquaux, G., Gramfort, A., et al: ‘Scikit-learn: machine learning in python’, J. Mach. Learn. Res., 2011, 12, pp. 28252830.
    34. 34)
      • 34. Murray, A.F., Edwards, P.J.: ‘Enhanced MLP performance and fault tolerance resulting from synaptic weight noise during training’, IEEE Trans. Neural Netw., 1994, 5, (5), pp. 792802.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cds.2018.5532
Loading

Related content

content/journals/10.1049/iet-cds.2018.5532
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
This is a required field
Please enter a valid email address