Current eye-trackers generally rely on previous-generation computer-vision algorithms and the best ones are also expensive and closed-source. Recent work has reported that it is possible to obtain very good performance using simple convolutional neural networks (CNN) running off a mobile phone camera. This project primarily involves implementing and extending these CNNs to evaluate and improve their performance. Potential extensions include: connecting the CNN to a mobile-phone app’s camera stream, incorporating head-position (pose) estimates for eye-in-head measurements, and incorporating time-series filtering to improve the eye-position estimation.
Planned effort: 350 hours.
Skill level: Intermediate, Advanced.
Pre-requisite skills: At least comfortable with Pytorch/Tensorflow. Have experience with image/video processing and computer vision. Alternatively, comfortable with Android/iOS app development.
@Dhwanit_Balwani Check out the project page from last year, go through the project requirements mentioned in the description, and prepare your proposal as per the GSoC recommendations. And submit it. If you send us a draft early enough, we can give you comments.
@suresh.krishna I am interested in contributing to this problem statement. Is there any slack channel or group which I should join to know more about the same?
No, not really. But if you would like to contribute to this project or similar projects, outside of the GSoC period, feel free to get in touch with me. For the GSoC period, each applicant is supposed to write up their own proposal…
I read the project description and am extremely interested to participate in GSoC with this.
I wanted to know if you would accept people who are not currently students (I graduated in 2020 and have left my job recently so I have been working independently for a few months).
I have had various jobs and research internships where I worked on using ML to solve different types of problem statements, so I am very interested in interdisciplinary research. Also, although I have developed a curiosity for computational neuroscience, I don’t have any past experience in neuroscience.
Let me know what steps I need to start with if I am someone you consider eligible. Cheers!
Hi,
I am really interested in this project, I feel I have all those mentioned skills to implement this project from Android development (basic to intermediate) to Tensorflow framework deep learning algorithm implementation. Is there any way I can discuss this project and my relevant skills and experience through e-mail?
I am interested in this project. I have gone through your comments on GSoC-21 on the same project also, skimmed through “eye-tracking” papers from nature communication and CVPR-16.
basically implementing a deep CNN with augmentation and hyperparameter tuning on a large-scale gaze-capture dataset to predict Gaze using a smartphone camera.
I will read those papers fully and is there any way I can interact with you through e-mail?
yes, this year onwards, google summer of code is open to those who are not students as well, as long as you can commit to the project for the gsoc period. and experience in computational neuroscience is not required at all. i encourage you to apply ! take a look at the source paper, the project definition above, the requirements for your project proposal, and start preparing the proposal ! you can also take a look at last year’s work on this project -
you can email me if needed by searching for my name on google, but you can also send me private messages here if you want to talk privately. welcome aboard.
As an update, and as I believe the program description or instructions to applicants states, if you send us a draft in time, we will be able to give you feedback. I believe there is an option to do this through the portal, or there was one last year. Please update if you find out the process for this year.
The proposal itself, along with your CV, will show how well you have grasped the topic, can research the field, can work independently, can demonstrate your skills and are most likely to push the project forward. The ranking of applicants will be made based on that.
My name is Umang Pandey and I am a senior undergraduate student at IIT
Kanpur. I have a decent research experience in ML/AI and I would love to
be a part of the GSoC this year and make some meaningful contributions to
this project. Please let me know if there are any such
opportunities or tasks available that might make increase my
chances of working with you. Also, what do you look for in an ideal
candidate? Any advice on how to get started would be
greatly appreciated.
Hi @malin
My name is Muhammed Abdullah and I am an undergraduate from VNIT, Nagpur, India. I am interested in ML and Computer Vision and have done few projects like Face Recognition project
I found the problem statement very intriguing and wish to collaborate. I do have quite good experience with python, PyTorch, and open-source contributions, as well as I, have some experience with Android based projects.(eg: this, this)
I am eager to learn new skills needed for the project. Any suggestion on how to get started would be a great help. Is there anything that I need to prepare prior to proposals?
Looking forward to hearing back from you.
Thank You.
