The authors introduce a continuous stochastic generative model that can model continuous data, with a simple and reliable training algorithm. The architecture is a continuous restricted Boltzmann machine, with one step of Gibbs sampling, to minimise contrastive divergence, replacing a time-consuming relaxation search. With a small approximation, the training algorithm requires only addition and multiplication and is thus computationally inexpensive in both software and hardware. The capabilities of the model are demonstrated and explored with both artificial and real data.
References
-
-
1)
-
G.E. Hinton ,
T.J. Sejnowski
.
(1986)
Learning and re-learning in Boltzmann machines, Parallel distributed processing: explorations in the microstructure of cognition.
-
2)
-
P. Smolensky
.
(1986)
Information processing in dynamical systems: foundations of harmony theory, Parallel distributed processing: explorations in the microstructure of cognition.
-
3)
-
Fleury, P., Murray, A.F., Reekie, M.: `High-accuracy mixed-signal VLSI for weight modification in contrastive divergence learning', Proceedings of 12th Int. Conf. on Artificial neural networks (ICANN2002), August 2002, Madrid, Spain, p. 426–431.
-
4)
-
Hinton, G.E.: `Training products of experts by minimizing contrastive divergence', Technical, 2000.
-
5)
-
Marks, T.K., Movellan, J.R.: `Diffusion networks, products of experts, and factor analysis', 2001.02, Technical, 2001.
-
6)
-
L. Tarassenko ,
G. Clifford
.
Detection of ectopic beats in the electrocardiogram using an autoassociative neural network.
Neural Process. Lett.
,
15 -
25
-
7)
-
Y.W. Teh ,
G.E. Hinton
.
(2001)
Rate-coded restricted Boltzmann machine for face recognition, Advances in neural information processing system.
-
8)
-
Hinton, G.E.: `Products of experts', Proceedings of 9th Int. Conf. on Artificial neural networks, (ICANN'99), Sept. 1999, Edinburgh, Scotland, UK, p. 1–6.
-
9)
-
A.F. Murray
.
Novelty detection using products of simple experts—a potential architecture for embedded systems.
Neural Netw.
,
1257 -
1264
-
10)
-
Fleury, P., Woodburn, R.J., Murray, A.F.: `Matching analogue hardware with applications using the products of experts algorithm', Proceedings of IEEE European Symp. on Artificial neural networks, 2001, p. 63–64.
-
11)
-
Chen, H., Murray, A.F.: `A continuous restricted Boltzmann machine with hardware-amenable learning algorithm', Proceedings of 12th Int. Conf. on Artificial neural networks (ICANN2002), Aug. 2002, Madrid, Spain, p. 358–363.
-
12)
-
J. Alspector ,
J.W. Gannett ,
S. Haber ,
M.B. Parker ,
R. Chu
.
A VLSI-efficient technique for generating multiple uncorrelated noise sources and its application to stochastic neural networks.
IEEE Trans. Circuits Syst.
,
109 -
123
-
13)
-
J.J. Hopfield
.
Neurons with graded response have collective computational properties like those of two-state neurons.
Proc. Natl. Acad. Sci. USA
,
3088 -
3092
-
14)
-
Woodburn, R.J., Astaras, A.A., Dalzell, R.W., Murray, A.F., McNeill, D.K.: `Computing with uncertainty in probabilistic neural networks on silicon', Proceedings 2nd Int. ICSC Symp. on Neural computation, 1999, p. 470–476.
-
15)
-
J.R. Movellan ,
P. Mineiro ,
R.J. Williams
.
A Monte-Carlo EM approach for partially observable diffusion processes: theory and applications to neural networks.
Neural Comput.
,
1501 -
1544
-
16)
-
J.R. Movellan
.
A learning theorem for networks at detailed stochastic equilibrium.
Neural Comput.
,
1157 -
1178
-
17)
-
B.J. Frey
.
(1997)
Continuous sigmoidal belief networks trained using slice sampling, Advances in neural information processing systems.
-
18)
-
Y. Wang ,
Y.S. Zhu ,
N.V. Thakor ,
Y.H. Xu
.
A short-time multifractal approach for arrhythmia detection based on fuzzy neural network.
IEEE Trans. Biomed. Eng.
,
989 -
995
http://iet.metastore.ingenta.com/content/journals/10.1049/ip-vis_20030362
Related content
content/journals/10.1049/ip-vis_20030362
pub_keyword,iet_inspecKeyword,pub_concept
6
6