Handling missing data for REMoDNaV

1

Hello,

I am using REMoDNaV 1.0 for eye event classification in data obtained from EyeLink Plus. Missing data (mostly blinks) are represented using NaNs (explicitly changed from the default output). When I run REMoDNaV on this data, I get an (expected?) error because of the Savitzky-Golay filter.

numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares

As far as I understand it, any data with NaN values should give this error.

Eg:

x = np.array([2, 2, 5, 2, 1, np.nan, 1, 4, 9])
print(savgol_filter(x, 5, 2))

So, right now, it looks like the algorithm does not work if there is missing data. Or am I missing something? I ran the algorithm without the Savitzky-Golay filter (with custom filenames/paths):

remodnav "/FHLPilot_P002_eye4.csv" "//FHLPilot_P002_eye4.tsv" 0.024878919425550945 500 --savgol-length 0.0 --median-filter 0.05

It runs but I do get quite a few velocity warnings:

WARNING:Computed velocity exceeds threshold. Inappropriate filter setup?

My question is: how does the algorithm treat missing data and how should we represent data loss from blinks in the input data file? I understand the algorithm does not classify blinks and treats blinks also like data loss. But, in its current form it looks like we cannot use data with loss. Also, from the elaborate output figure that is generated, it looks like quite a few smooth pursuit events around blinks (missing data). I have attached the generated figure (

FHLPilot_P002_eye4
FHLPilot_P002_eye417791×314 565 KB
).

I use REMoDNaV 1.0 on Python 3.9.6 on MacOS (Big Sur). This is the first time I am using REMoDNaV.

Thanks!

2

Hi,

I can confirm that remodnav does work in the presence of missing data-- in fact, I don’t think I have ever seen an eyetracking dataset without missing data :wink:

Please let me know, if the warning message is unclear. The threshold it is referring to is a parameter of the algorithm (--max-vel), you can set it to whatever value you believe to be reasonable.

I gave an appropriate description of the missing data treatment. There is no classification of blinks vs just-missing-data: Missing data is just that.

It looks like quite a few smooth pursuit events around blinks

For the algorithm, a smooth pursuit is a non-saccade event that is also not a fixation. Given that it has no concept of a blink, it also cannot be aware of pre/post-blink smooth-pursuit classification.

Best

3

numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares

This is unlikley to be related to remodnav directly, but a more general numpy issue. See here, for example: python - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares on first run only - Stack Overflow

It suggests that simply running things again can “fix” it. Not a real solution, but maybe something that get’s you going for now.