GSoC 2022 Project Idea 6.2: Efficient app-based measurement of visual functions in infants and young children (350 h)

GSoC
,
1

Accurate and efficient measurement of visual function is difficult in infants and young children because of limited cooperation, inability to provide cognitive verbal responses and lack of efficient behavioural methods. This is important in the clinical and research context where detection and treatment of eye conditions in infancy is dependent on measurement of visual function. Visual deprivation in infants disrupts normal visual development and affects multiple visual functions that are important in brain-based visually guided behaviors in everyday life such as contrast sensitivity, motion perception, contour integration, and face recognition. At present there are no reliable automated objective methods for measuring visual functions in infants and young children below the age of 3 years.

This new project will address these limitations. It involves the development of an application with a suite of visual stimuli to probe multiple visual functions. The application will employ an adaptive staircase with a preferential looking behavioral paradigm and eye tracking. The application will measure the sensory threshold of each visual function and the response path to the threshold, such as uncertainty, providing important additional indicators creating an individual and disease specific profile of visual loss. This application will extend to establish visual function norms, profile visual loss allowing targeted intervention therapy and monitor the effects of treatment.

Planned effort: 350 hours

Skill level: Intermediate/Advanced

Pre-requisite skills: Comfortable with 1 language (e.g. Python). Experience with app development and basic image/video processing. Ideally, comfortable with Android/iOS app development.

Mentors: Arvind Chandna @arvindchandna (lead) and Suresh Krishna @suresh.krishna (co-mentor)

Tech keywords: App development, iOS/Android, Python/equivalent

1 Like
2

Hello @arvindchandna and @suresh.krishna . I’m Sanjay, a third year student pursuing my B.Tech(CSE) at Amrita School of Engineering. I find this project very intriguing and I also feel that my skillset best-suits the project. Is there some preliminary tasks / contributions that I should make, to be eligible for the project ?

2 Likes
3

@sanjay-thiyagarajan Welcome aboard and thanks for your interest. No, there are no preliminary tasks or contributions. You are welcome to ask questions to understand the desired app better. As I believe the program description or instructions to applicants states, if you send us a draft in time, we will be able to give you feedback. I believe there is an option to do this through the portal, or there was one last year. Please update if you find out the process for this year.

Your proposal itself, along with your CV, will show how well you have grasped the topic, can research the field, can work independently, can demonstrate your skills and are most likely to push the project forward. The ranking of applicants will be made based on that.

Hope that helps

1 Like
4

Hi everyone. This looks like a super-interesting project, I think applications open from the 4th of April onwards.

I am really happy to submit a proposal at the time, to get your feedback. Meanwhile, I would like to ask a few questions and shortly introduce myself and learn more about the project to better grasp the ideas behind it.

My name is Ioannis Valasakis and I am a PhD student in King’s College London. The field of my research is autism spectrum disorder identification with a focus on brain functional connectivity, graph theory metrics and deep learning. We have neonatal datasets which involve eye metrics and eye motion characteristics, so I am extremely interested in developing further this area and use as a combination for my research. I have previous experience in the industry as a software engineer in the fields of embedded programming and software engineering.

I would like to better understand the deliverables and the path that will be followed.

  • Is there any preference on iOS vs Android? I expect a compatible framework to be used or is native development preferred? (I do have experience with both Swift and Java and ML using Google’s ML kit for image classification)
  • Do you expect traditional eye tracking measurements or employ a deep learning network as well?
  • What outcomes of visual function measurements do you expect? i.e. Gaze responses, fixation duration, etc?
  • Is there any proposed literature to use as a background for the study and build on it?
  • Finally, is a publication expected out of this work? What is the open source license to be used and the copyright owners?

Can you give a few more details about the proposed identification of visual stimuli? I had a look on Leek, M.R. Adaptive procedures in psychophysical research. Perception & Psychophysics 63, 1279–1292 (2001). https://doi.org/10.3758/BF03194543 and on Early Screening of Visual Processing Dysfunctions in Children Born Very or Extremely Preterm Frontiers in Human Neuroscience, 15, 729080 - November 2021 for some proposed methods for visual identification in infants and young children.

I am very happy working with you and feel free to point me to the right direction for the proposal.

Best wishes,
Ioannis

1 Like
5

@wizofe Thank you very much for your interest and detailed message, Ioannis. We will get back to you very soon.

1 Like
6

@wizofe

Is there any preference on iOS vs Android? I expect a compatible framework to be used or is native development preferred? (I do have experience with both Swift and Java and ML using Google’s ML kit for image classification)

This is a strategic decision based on getting something completed versus having the GSoC mentee learn a new compatible framework and also how well things work with the cross-platform framework… something to be discussed with the team once the selection process is done. Your proposal can just assume either native development or a cross-platform framework and provide evidence of ability to handle it…

Do you expect traditional eye tracking measurements or employ a deep learning network as well?

First iteration, whatever works and using whichever libraries are available. Just enough to reliably detect gaze changes, time of gaze change, fixation duration etc. Future iterations, can make everything more fancy and improve quality of gaze-tracking etc, experiment with different methods etc.

What outcomes of visual function measurements do you expect? i.e. Gaze responses, fixation duration, etc?
Is there any proposed literature to use as a background for the study and build on it?

tinyurl.com/yeyn7nr6 is a list of example publications

There will be a bunch of measures calculated for each task that is used - example with grating acuity, vernier acuity, contour integration etc… one would characterize the psychometric function with measures like % correct, slope and other parameters of the psychometric function, no of reversals etc.

Finally, is a publication expected out of this work? What is the open source license to be used and the copyright owners?

Publication will always be nice once the product is finalized and all contributors will of course be authors. I am not sure of the open source license and have contacted INCF for guidelines. Copyright should be shared between all contributors, is my first guess, but happy to discuss further.

Hope that helps !

1 Like
7

When agreeing to the program terms, the contributor grants Google “a non-exclusive, worldwide, perpetual, irrevocable, free license (with right to sublicense) to reproduce, prepare derivative works of, distribute, perform, display, and otherwise use your Submissions for the purpose of administering the Program and promoting Google Summer of Code.”

1 Like
8

Hi @suresh.krishna thank you for your helpful guidance on that. My question was more about the future of the project and its perspective use, as I would feel it’s really beneficial to keep my involvement but also make it part of an open-source community!

9

Thank you. Those are very helpful points indeed to steer my research further and also investigate for algorithmic approaches and respective frameworks to implement them.

10

@wizofe I hope you put in a proposal and we very much look forward to receiving it :slight_smile:

1 Like
11

As you mention above, we can also give you feedback on a draft before the actual submission if you get it to us in time.

1 Like
12

That sounds great. I am going to work on it over the weekend and send it through! Would you like to share your emails by PM? :slight_smile: Thanks!

13

You are welcome to Google my email and write to me if needed or send me the draft directly, but I believe that the GSoC portal, once it opens on April 4th, will allow you to formally submit a draft for feedback through the portal.

You can also PM me here…

1 Like
14

Hi Suresh! Oh that makes sense, I wasn’t aware I could send a draft proposal through the GSoC portal, let’s do it the official way then. I’ll ask if I have any questions while creating the proposal though. Thanks for your time and availability :slight_smile:

2 Likes
15

Hi loannis @wizofe,
Nice to meet you. I have a question I am hoping you can answer.

As far as I understand, autism spectrum disorder can have external clues (e.g. eye motion) which is one the characteristics you mentioned. I believe that eye motion can’t be used as a characteristic in identifying bipolar disorder. I don’t know whether deep learning, brain functional connectivity, and graph theory metrics are involved bipolar disorder identification or not, but assuming it is, can you point me out where to research about that?

I am aware that’s not your field of research, but for sure you have more experience in research than me since I am not a PhD student.
Thanks in advance.

1 Like
16

Hi Selim. Unfortunately, I am not very familiar with bipolar disorder as a research subject. I think you will be able to find more information by looking at online resources (e.g. Google Scholar) or just a search engine of your choice!

Use the keywords of your choice and use “reviews” as a good reference, especially if they are recent. For example, if I quickly search for your terms I get this paper, which may be interesting. Always look at the bottom to the rest of the papers cited. Good luck!

Syan SK, Smith M, Frey BN, et al. Resting-state functional connectivity in individuals with bipolar disorder during clinical remission: a systematic review. J Psychiatry Neurosci . 2018;43(5):298-316. doi:10.1503/jpn.170175

1 Like
17

@malin @suresh.krishna
I have a couple of questions regarding the project description and proposal writing. I am comfortable enough with Python and Android development and I hope the answers to these questions help me propose the libraries to be used.

Project description:
I believe the description is clear to what the outcome should be. However, what I find ambiguous is the project scope. Does this project span only 16 weeks? If not, are there milestones to achieve one by one or at least the next milestone?*

As I understand from Chapter 4 of Visual Impairments, the fact that evidence of normal or abnormal visual status at one age is not necessarily predictive of what the visual status will be at a later age. Thus, I am not sure how a disease-specific profile should be created! Also, shouldn’t the application be built first for data collection since the normative data should be gathered by the same tool of measurement?

Accurate and efficient measurement of visual function is difficult in infants and young children because of limited cooperation, inability to provide cognitive verbal responses and lack of efficient behavioural methods.

Are there any feedback sources that can be gathered from infants other than eyes and head?

Should project 6.1 be used as eye-tracking method in this one?

Proposal writing:
Is the domain logic already defined or not? If yes, where can I find it? If no, should it be defined prior to writing proposal phase or after?

It involves the development of an application with a suite of visual stimuli to probe multiple visual functions.

Are all of those visual functions defined or to be defined?

I am still a beginner to the open-source community. It would help me a lot if you let me know how it works. I am imagining it as discussion between members of an open-source community regarding a specific problem or project. Then, each member contributes which what he can do. What I don’t understand here, is how to write a timeline of what I will do if we are still in the designing or discussion phase? (assuming that my understanding is correct and applies to this project)

18

This defiantly helps. :smiley:

I was able to get the same search results that you got, but I don’t know where to find the reviews of papers in search results.

19

  1. The GSoC part of the project is 350 hours in scope – it is a large project in GSoC terms. The progress made here will hopefully be continued - you should plan your milestones based on what you are able to achieve realistically, starting from scratch.

  2. Yes, the tool can be used for data collection, norm development and correlation with other diaagnostic methods to identify measurement ranges that are abnormal. i.e. Current gold standard methods of diagnosis will serve as the reference.

  3. in principle, yes, there are potentially many data sources, but this particular app aims at eye and head tracking for measuring visual function. if you have other ideas, you can incorporate them.

  4. no, not necessary to use any specific eyetracker… including the other project, which is not yet functional.

  5. if you submit a first draft, we can help you along with the domain logic and visual functions.

https://tinyurl.com/y7bshs3k ](https://tinyurl.com/y7bshs3k are useful references to look at. There will be a bunch of measures calculated for each task that is used - example with grating acuity, vernier acuity, contour integration etc… one would characterize the psychometric function with measures like % correct, slope and other parameters of the psychometric function, no of reversals etc. Read up on psychometric function if needed. Where the infant’s eyes go will be used as the response measure… for exampmle, choose higher contrast stimulus of two… if infant’s eye naturally goes to higher contrast, we know visual system is able to detect it.

  1. it is partly an open-source community project, and partly a proposal used for selecting contributor(s) via GSoC. these are partly overlapping goals. the proposal is really for selecting one or more contributors… what we eventually do will be based on the team. those not selected via GSoC will also be most welcome to contirbute if they want to join - it is just that they will not formally be part of GSoC, will nto receive a stipend, and will not have to submit a report.

hth

1 Like