Multimodal medical image registration via common representations learning and differentiable geometric constraints

Multimodal medical image registration via common representations learning and differentiable geometric constraints

For access to this article, please select a purchase option:

Buy article PDF
(plus tax if applicable)
Buy Knowledge Pack
10 articles for $120.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Your details
Why are you recommending this title?
Select reason:
Electronics Letters — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

Multimodal medical image registration remains a challenging problem when strong appearance variations and imprecise alignment exist in images. Previous deep network approaches cannot handle such a high degree of variability and prohibit the use of strong geometric constraints. The authors introduce a novel deep architecture that not only produces image representations that are well-suited for this challenging task, but also leverages knowledge of the geometry constraints for robust registration. By enforcing the representations of different modalities living in a common semantic space, they obtain convolutional features tending to respond to object parts consistently across modality. This yields a unique description for all object fragments and allows the end user to know the model's decision process, whereas most existing models remain unclear and difficult to explain. By using the differentiable spatial transformer to compensate transform, they integrate geometric consensus into the cost function to enable end-to-end model optimisation which has not yet been exploited before. The authors evaluate their method on a very challenging medical image dataset. Experiments demonstrate that the proposed method provides a plausible representation and outperforms state-of-the-art approaches by a significant margin.


    1. 1)
    2. 2)
      • 2. Simonovsky, M., Gutiérrez-Becker, B., Mateus, D., et al: ‘A deep metric for multimodal registration’. Medical Image Computing and Computer-Assisted Intervention (MICCAI), Athens, Greece, 2016, pp. 1018.
    3. 3)
    4. 4)
      • 4. Liao, R., Miao, S., de Tournemire, P., et al: ‘An artificial agent for robust image registration’. Association for the Advancement of Artificial Intelligence (AAAI), San Francisco, USA, 2017, pp. 41684175.
    5. 5)
      • 5. Jaderberg, M., Simonyan, K., Zisserman, A.: ‘Spatial transformer networks’. Neural Information Processing Systems (NIPS), Montréal, Canada, 2015, pp. 20172025.
    6. 6)
      • 6. Chollet, F.: ‘Xception: deep learning with depthwise separable convolutions’, arXiv preprint, 2017, 1610-02357.
    7. 7)

Related content

This is a required field
Please enter a valid email address