access icon openaccess Discrete-level memristive circuits for HTM-based spatiotemporal data classification system

The authors propose a discrete-level memristive memory design for analogue data processing in hardware implementations of hierarchical temporal memory (HTM). In this study, memristors were set to ternary and quaternary states in a sub-cell by application of different write voltage levels through a resistive network configuration. Simulations of the proposed circuit show that the highest number of discrete output levels of the memory was achieved using quaternary logic. However overall, using the same number of sub-cells and ternary logic exhibits the lowest relative error rate. For data classification purposes, the proposed discrete-level memristive cells are incorporated into the TM of HTM architecture, and its hardware circuit is presented for pattern recognition. They report improved results of face recognition using AR, ORL and UFI databases, and TIMIT database for speech recognition. These results are compared with the earlier design of HTM having only the spatial pooler (SP). Accuracy of the HTM architecture incorporating both SP and TM with discrete-level memristive cells for face recognition increased from 76.5 to 83.5% for AR database and speech recognition accuracy is improved from 73.3 to 93.3%.

Inspec keywords: memory architecture; memristors; memristor circuits; pattern classification

Other keywords: hardware implementations; analogue data processing; write voltage levels; discrete-level memristive memory design; quaternary states; pattern recognition; HTM-based spatiotemporal data classification system; hierarchical temporal memory; face recognition; speech recognition; quaternary logic; discrete-level memristive circuits; discrete-level memristive cells; HTM architecture; hardware circuit; spatial pooler; resistive network configuration

Subjects: Resistors; Data handling techniques; Memory circuits; Storage system design

References

    1. 1)
      • 45. Csapo, A.B., Baranyi, P., Tikk, D.: ‘Object categorization using VFA-generated nodemaps and hierarchical temporal memories’. IEEE Int. Conf. Computational Cybernetics 2007 ICCC 2007, October 2007, pp. 257262.
    2. 2)
      • 41. Zyarah, A.M.: ‘Design and analysis of a reconfigurable hierarchical temporal memory architecture’. Master's thesis, 2015.
    3. 3)
      • 57. Hermansky, H., Morgan, N.: ‘Rasta processing of speech’, IEEE Trans. Speech Audio Process., 1994, 2, (4), pp. 578589.
    4. 4)
      • 10. Rabbani, P., Dehghani, R., Shahpari, N.: ‘A multilevel memristor–CMOS memory cell as an ReRAM’, Microelectron. J., 2015, 46, (12), pp. 12831290.
    5. 5)
      • 9. Duan, S.K., Hu, X.F., Wang, L.D., et al: ‘Analog memristive memory with applications in audio signal processing’, Sci. China Inf. Sci., 2014, 57, (4), pp. 115.
    6. 6)
      • 25. Kannan, S., Karimi, N., Karri, R., et al: ‘Modeling, detection, and diagnosis of faults in multilevel memristor memories’, IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst., 2015, 34, (5), pp. 822834.
    7. 7)
      • 40. Hpl.hp.com HP Memristor FAQ.
    8. 8)
      • 43. Farahmand, N., Dezfoulian, M.H., GhiasiRad, H., et al: ‘Online temporal pattern learning’. 2009 Int. Joint Conf. Neural Networks, June 2009, pp. 797802.
    9. 9)
      • 36. Kim, H., Sah, M.P., Yang, C., et al: ‘Memristor based multilevel memory’. 2010 12th Int. Workshop on Cellular Nanoscale Networks and their Applications (CNNA 2010), 2010, pp. 16.
    10. 10)
      • 33. Biolek, D., Kolka, Z., Biolkova, V., et al: ‘Memristor models for spice simulation of extremely large memristive networks’. 2016 IEEE Int. Symp. Circuits and Systems (ISCAS), 2016, pp. 389392.
    11. 11)
      • 26. Maan, A.K., Jayadevi, D.A., James, A.P.: ‘A survey of memristive threshold logic circuits’, IEEE Trans. Neural Netw. Learn. Syst., 2016, PP, (99), pp. 113.
    12. 12)
      • 22. Eryilmaz, S.B., Kuzum, D., Jeyasingh, R., et al: ‘Brain-like associative learning using a nanoscale non-volatile phase change synaptic device array’, arXiv preprint arXiv:1406.4951, 2014.
    13. 13)
      • 18. Khan, S.: ‘The divided flash memory device for implementing neurons and neural networks’. , 2013 Int. Conf. Informatics, Electronics & Vision (ICIEV), 2013, pp. 15.
    14. 14)
      • 29. Hawkins, J., Blakeslee, S.: ‘On intelligence’ (Macmillan, 2007).
    15. 15)
      • 51. Cui, Y., Ahmad, S., Hawkins, J.: ‘Continuous online sequence learning with an unsupervised neural network model’, Neural Comput., 2016, (28), pp. 24742504.
    16. 16)
      • 53. Samaria, F.S., Harter, A.C.: ‘Parameterisation of a stochastic model for human face identification’. Proc. Second IEEE Workshop on Applications of Computer Vision 1994, 1994, pp. 138142.
    17. 17)
      • 21. Lee, B.C., Zhou, P., Yang, J., et al: ‘Phase-change technology and the future of main memory’, IEEE Micro, 2010, 30, (1), pp. 131141.
    18. 18)
      • 30. Hawkins, J., Ahmad, S., Dubinsky, D.: ‘Hierarchical temporal memory including HTM cortical learning algorithms’. Technical Report, Numenta, Inc., Palto Alto, 2010. Available at http://www.numenta.com/htmoverview/education/HTM_CorticalLearningAlgorithms.pdf, accessed 16/03/2017.
    19. 19)
      • 39. Duarte, J.C., Martins, E.V., Alves, L.N.: ‘Frequency characterization of memristive devices’. 2013 European Conf. Circuit Theory and Design (ECCTD), 2013, pp. 14.
    20. 20)
      • 49. Cui, Y., Ahmad, S., Hawkins, J.: ‘The HTM spatial pooler: a neo cortical algorithm for online sparse distributed coding’, bioRxiv, 2017, p. 085035.
    21. 21)
      • 4. Haller, G.M., Wooley, B.A.: ‘An analog memory integrated circuit for waveform acquisition up to 900 MHz’. 1993 IEEE Nuclear Science Symp. Medical Imaging Conf., San Francisco, CA, 2–5 November 1993, vol. 1, pp. 25.
    22. 22)
      • 48. Ibrayev, T., James, A.P., Merkel, C., et al: ‘A design of HTM spatial pooler for face recognition using memristor–CMOS hybrid circuits’. 2016 IEEE Int. Symp. Circuits and Systems (ISCAS), May 2016, pp. 12541257.
    23. 23)
      • 27. Versace, M., Chandler, B.: ‘The brain of a new machine’, IEEE Spectr., 2010, 47, (12), pp. 3037.
    24. 24)
      • 52. Martınez, A., Benavente, R.: ‘The AR face database’. Rapport Technique 24, 1998.
    25. 25)
      • 11. James, A.P., Fedorova, I., Ibrayev, T., et al: ‘HTM spatial pooler with memristor crossbar circuits for sparse biometric recognition’, IEEE Trans. Biomed. Circuits Syst., 2017, PP, (99), pp. 112.
    26. 26)
      • 47. Fan, D., Sharad, M., Sengupta, A., et al: ‘Hierarchical temporal memory based on spin-neurons and resistive memory for energy-efficient brain-inspired computing’, IEEE Trans. Neural Netw. Learn. Syst., 2016, 27, (9), pp. 19071919.
    27. 27)
      • 20. Simpson, R.E., Fons, P., Kolobov, A.V., et al: ‘Interfacial phase-change memory’, Nat. Nanotechnol., 2011, 6, (8), pp. 501505.
    28. 28)
      • 37. Manem, H., Rose, G.S., He, X., et al: ‘Design considerations for variation tolerant multilevel CMOS/nano memristor memory’. Proc. 20th Symp. Great lakes Symp. VLSI, 2010, pp. 287292.
    29. 29)
      • 24. Mostafa, H., Ismail, Y.: ‘Process variation aware design of multi-valued spintronic memristor-based memory arrays’, IEEE Trans. Semicond. Manuf., 2016, 29, (2), pp. 145152.
    30. 30)
      • 38. Manem, H., Rose, G.S.: ‘A read-monitored write circuit for 1T1M multi-level memristor memories’. 2011 IEEE Int. Symp. Circuits and systems (ISCAS), 2011, pp. 29382941.
    31. 31)
      • 55. Garofolo, J.S., Lamel, L.F., Fisher, W.M., et al: ‘DARPA TIMIT acoustic–phonetic continuous speech corpus CD-ROM. nist speech disc 1-1.1’. NASA STI/Recon Technical Report N 93, 1993.
    32. 32)
      • 19. Strukov, D., Merrikh-Bayat, F., Prezioso, M., et al: ‘Memory technologies for neural networks’. 2015 IEEE Int. Memory Workshop (IMW), 2015, pp. 14.
    33. 33)
      • 2. Rojas, R.: ‘Neural networks: a systematic introduction’ (Springer Science & Business Media, 2013).
    34. 34)
      • 3. Fujita, O., Amemiya, Y.: ‘A floating-gate analog memory device for neural networks’, IEEE Trans. Electron Devices, 1993, 40, (11), pp. 20292035.
    35. 35)
      • 42. Mnatzaganian, J., Fokoué, E., Kudithipudi, D.: ‘A mathematical formalization of hierarchical temporal memory's spatial pooler’, arXiv preprint arXiv:1601.06116, 2016.
    36. 36)
      • 13. Angell, J.B., Widrow, B., Pierce, W.H.: ‘Birth, life, and death in microelectronic’. Technical Report No. 1552-2/1851-1, May 1961.
    37. 37)
      • 16. Lee, S.-K., Park, D.-H.: ‘Multi level flash memory device and program method’. US Patent 7,054,199, May 2006.
    38. 38)
      • 54. Lenc, L., Král, P.: ‘Unconstrained facial images: database for face recognition under real-world conditions’. 14th Mexican Int. Conf. Artificial Intelligence (MICAI 2015), Cuernavaca, Mexico, 25–31 October 2015.
    39. 39)
      • 56. Ellis, D.P.W.: PLP and RASTA (and MFCC, and inversion) in MATLAB, 2005.
    40. 40)
      • 6. Diorio, C., Mahajan, S., Hasler, P., et al: ‘A high-resolution non-volatile analog memory cell’. 1995 IEEE Int. Symp. Circuits and Systems 1995 ISCAS'95, 1995, vol. 3, pp. 22332236.
    41. 41)
      • 7. Chen, Y., Jung, G.-Y., Ohlberg, D.A.A., et al: ‘Nanoscale molecular-switch crossbar circuits’, Nanotechnology, 2003, 14, (4), p. 462.
    42. 42)
      • 14. Hasler, P., Minch, B., Mead, C., et al: ‘A high-resolution nonvolatile analog memory cell’. Proc. 1995 IEEE Int. Symp. Circuits and Systems, 1995, vol. 3, pp. 22332236.
    43. 43)
      • 12. Anderson, J.A., Rosenfeld, E.: ‘Talking nets: an oral history of neural networks’ (MIT Press, 2000).
    44. 44)
      • 17. Lee, J.-W.S.: ‘Multilevel phase change memory’. US Patent 7,488,968, 10 February 2009.
    45. 45)
      • 35. Rose, G.S.: ‘Overview: memristive devices, circuits and systems’. Proc. 2010 IEEE Int. Symp. Circuits and Systems (ISCAS), 2010, pp. 19551958.
    46. 46)
      • 8. Hu, X.F., Duan, S.K., Wang, L.D., et al: ‘Memristive crossbar array with applications in image processing’, Sci. China Inf. Sci., 2012, 55, (2), pp. 461472.
    47. 47)
      • 5. O'Halloran, M., Sarpeshkar, R.: ‘A 10 nW 12 bit accurate analog storage cell with 10 aa leakage’, IEEE J. Solid-State Circuits, 2004, 39, (11), pp. 19851996.
    48. 48)
      • 28. George, D., Hawkins, J.: ‘A hierarchical Bayesian model of invariant pattern recognition in the visual cortex’.  Proc. 2005 IEEE Int. Joint Conf. Neural Networks, 2005 IJCNN ‘05, July 2005, vol. 3, pp. 18121817.
    49. 49)
      • 23. Chua, L.: ‘Memristor – the missing circuit element’, IEEE Trans. Circuit Theory, 1971, 18, (5), pp. 507519.
    50. 50)
      • 50. Numenta Inc.: ‘Hierarchical temporal memory including HTM cortical learning algorithms’. Technical Report, 2006.
    51. 51)
      • 46. Melis, W.J.C., Chizuwa, S., Kameyama, M.: ‘Evaluation of the hierarchical temporal memory as soft computing platform and its VLSI architecture’. 39th Int. Symp. Multiple-Valued Logic, 2009, pp. 233238.
    52. 52)
      • 44. Ramli, I., Ortega-Sanchez, C.: ‘Pattern recognition using hierarchical concatenation’. 2015 Int. Conf. Computer, Control, Informatics and its Applications (IC3INA), October 2015, pp. 109113.
    53. 53)
      • 32. Stanley Williams, R.: ‘How we found the missing memristor’, IEEE Spectr., 2008, 45, (12), pp. 2835.
    54. 54)
      • 15. Lee, B.W., Sheu, B.J., Yang, H.: ‘Analog floating-gate synapses for general-purpose VLSI neural computation’, IEEE Trans. Circuits Syst., 1991, 38, (6), pp. 654658.
    55. 55)
      • 31. Hawkins, J., Ahmad, S., Purdy, S., et al: ‘Biological and machine intelligence (BAMI)’, Initial online release 0.4, 2016.
    56. 56)
      • 1. Dubois, J., Mattavelli, M.: ‘Embedded co-processor architecture for CMOS based image acquisition’. Proc. 2003 Int. Conf. on Image Processing 2003 ICIP 2003, 2003, vol. 2, p. II591.
    57. 57)
      • 34. Pickett, M.D., Strukov, D.B., Borghetti, J.L., et al: ‘Switching dynamics in titanium dioxide memristive devices’, J. Appl. Phys., 2009, 106, (7), p. 074508.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cps.2017.0053
Loading

Related content

content/journals/10.1049/iet-cps.2017.0053
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading