Develop a machine learning-based registration framework that makes use of the Diffeomorphic Registration implemented in DIPY. The goal is to train a CNN model that can compute the deformation field in an unsupervised setting. This project will also involve leveraging the reconstruction module in DIPY to perform image fusion via inter-modality registration.
Understand Diffeomorphic Registration. Look at dipy.align module and DIPY tutorial.
Understand advanced Image Reconstruction Models such as Free-Water-DTI, DKI, etc. in DIPY
Implement a Deep Neural Net (e.g. a 2D/ 3D CNN) or a Multivariate Model for Co-learning
Difficulty: High
Skills required: Python, Deep Learning, Tensorflow, Registration, strong math skills.
Hi @malin, @bramsh, and Shreyas. I am interested in this project. I have experience with Deep Learning and Tensorflow and have been working with deep neural net architectures. Can you help me where would be the best starting point for this project?
Hi @bramsh and @ShreyasFadnavis . I am Saksham, a sophomore at IIT Delhi. I am interested in this project. I have worked with tensorflow and deep neural networks particularly in computer vision tasks. I have a strong mathematical background and a good understanding on deep learning algorithms. How can I get started with this project building upto my proposal?
You can start by making a small enhancement/bugfix/documentation fix/etc to DIPY. It can help you get some idea about how things would work during the GSoC. The fix does not need to be related to your proposal.
Thank you for your interest! As per @bramsh 's suggestion, make sure to make a PR on DIPY’s GitHub… This way, you will also get familiarized with DIPY’s API. I would also suggest to read up some literature on thin-plate splines, diffeomorphic registration (see: https://dipy.org/documentation/1.1.1./examples_built/syn_registration_3d/ and https://www.youtube.com/watch?v=knuyM6bEQvs) to get an idea of how it works. To get some basics in place, see: http://user.engineering.uiowa.edu/~aip/papers/bookstein-89.pdf. As a part of this project, we will first start with a 2D case and then move to 3D. Feel free to get creative with the deep learning architecture! Deep learning is not compulsory to learn this spline, you can also implement any other method that you feel can do the job It might be simple to think of this problem in a supervised setting first, where you simulate the ground truth by adding a known deformation field… Does this help?
Hi,I am Shubham Shaswat,currently in final year pursuing Computer Science at HIT Kolkata. I would like to contribute to the project.Any suggestion to get started would be much appreciated.
also are there any chat room for discussing the project?
Thanks
Hi,Going through the links and tutorials as listed,I got some rough idea about the objective of the topics,still I need clarifications.
This is the brief idea about what I have understood:
Image Registration is matching points from one image (source) to another image(target) correspond to the same physical regions in both.
In our case we have a static image and moving image,our goal is to find deformation field.
So,If we going to build a Deep Learning model,does this mean our input will be these two images and the target would be finding a deformation field?
Hi @ShubhamShaswat! Yes, the tutorial is useful and you do have the correct idea. The goal is to do a non-rigid registration using thin-plate splines representing the deformation field.
The Deep Learning phase would involve something (like you guessed correctly) similar to: Cao. et.al. “Deformable Image Registration Based on Similarity-Steered CNN Regression”, MICCAI, 2017. Does this help?
Thanks,I will look into this paper.Also,I think it would be better if I share an architecture of our model including input pipeline so that we know we are on a right track.
Hi @ShreyasFadnavis and @bramsh
I am Kunal Mehta currently in my Final Year of Computer Engineering at K.J Somaiya.I have previously worked on various Machine Learning, Deep Learning and would like to contribute as much as I can towards this project.I have gone through previous discussions as well as resources and documentation which @ShreyasFadnavis mentioned.I have also worked on basic Dipy tutorial to improve my understanding towards this topic.
Here is the jupyter notebook on colab which I was working on following the tutorial to improve my knowledge on Dipy. https://colab.research.google.com/drive/16f3xoWaeqnGGcyxUQWO-hPq0Sk4XiZjv
Could you please help identify issues that are related to our topic
Thank You
Kunal Mehta
The author used Similarity Guidance to bridge the gap between two input images. In order to obtain the very accurate deformation field of each image pair, SyN is first applied on the intensity images with careful parameter tuning, and then Diffeomorphic Demons.The Deformation field after these 2 stages, is used as Ground Truth for model Training.
They paper mentioned that they don’t need deformation field in training.The model is optimized by maximizing image similarity with smoothness regularization of deformation fields.
I am looking at these two papers as reference in our project.So,if there anything more you want to add or point out any mistakes will be much appreciated.
I have also question regarding the use of current Dipy modules in these models.
Hi @KUNAL_MEHTA! Thank you for your interest. I see that you have created a replica of some DIPY tutorials, which is good. I am not sure if you mean GitHub issues here, but, generally speaking, they are not very beginner friendly. However, you can take a look at different issues related to registration such as: https://github.com/dipy/dipy/issues/1838
I would recommend understanding the Registration API of DIPY. We also have extensive examples of different registration methods which can be found here: 
It might be simple to think of this problem in a supervised setting first, where you simulate the ground truth by adding a known deformation field… Does this help?
