Using reproducible ephys preprocessing quality controls

t0mo7 · June 28, 2024, 3:44pm

Hi there,

I am trying to use the reproducible ephys quality controls to check my NP2 data, processed through spikeinterface.

From the appendix reproducible ephys paper, it seems like the noise level in the AP and LFP band, plus the LFP derivative, are computed on the raw data, before any processing (e.g. destriping etc for AP, or low cut filtering, subsample shifting etc for LFP). Is that correct? For NP2 recordings, does the IBL have specific code for extracting the LFP band? I am currently using CATGT.

For the LFP metrics, what is the order of operations? I assume one first gets the power at each frequency for each channel. For the ‘noise level’, I assume one then, for each channel, crops the data to be between 20 and 80Hz, and then works out the inter-frequency derivative. Then I assume one calculates the median of each of these single-channel derivatives. For ‘LFP derivative’, I assume one averages the power over some frequencies (maybe 20-80Hz), and then works out the derivative across channels. Is this correct?

Can I also confirm that the visually assessments for noisy channels and artefacts are also done on the raw data, before any processing?

Finally, can I ask how the raster plots for checking drift and epileptiform activity are generated? I assume this is a simpler spike detection algorithm than kilosort, and I was hoping to use spike interface’s detectpeaks() for this. Does preprocessing (sample shift, failing channel triage, destriping) happen prior to the spike detection, or is it, again, run on raw data?

Thanks!

owinter · July 3, 2024, 1:02pm

Hello,

From the appendix reproducible ephys paper, it seems like the noise level in the AP and LFP band, plus the LFP derivative, are computed on the raw data, before any processing (e.g. destriping etc for AP, or low cut filtering, subsample shifting etc for LFP). Is that correct? For NP2 recordings, does the IBL have specific code for extracting the LFP band? I am currently using CATGT.

Yes we have an utility that splits the LFP band from the full band NP2 recordings:

github.com

int-brain-lab/ibl-neuropixel/blob/7e663d1985bab3ae5f16bcb19aa7c9720bbf5bec/src/neuropixel.py#L236


      
              Split the trace header into values for a specific shank. Applicable for NP2.4 probes
              :param h:
              :param shank:
              :return:
              """
              shank_idx = np.where(h["shank"] == shank)[0]
              h_shank = {key: h[key][shank_idx] for key in h.keys()}
              return h_shank
          
          
          class NP2Converter:
              """
              Class used to 1. Extract LFP data from NP2 data and 2. If NP2.4 split the data into
              individual shanks
              """
          
              def __init__(self, ap_file, post_check=True, delete_original=False, compress=True):
                  """
                  :param ap_file: ap.bin spikeglx file to process
                  :param post_check: whether to apply post-check integrity test to ensure split content is
                  identical to original content (only applicable to NP2.4)

    np_conv = neuropixel.NP2Converter(bin_file, compress=True)
    np_conv_status = np_conv.process()
    np_conv_files = np_conv.get_processed_files_NP21()
    np_conv.sr.close()

Can I also confirm that the visually assessments for noisy channels and artefacts are also done on the raw data, before any processing?

It is often useful to apply a low-cut filter to the raw data, even with a very low corner to at least remove the DC offset. The visual control is done by looking before and after destriping.
There is this GUI that allows to visualise a file:

Some other examples can be found here:
https://int-brain-lab.github.io/iblenv/notebooks_external/loading_raw_ephys_data.html

Finally, can I ask how the raster plots for checking drift and epileptiform activity are generated? I assume this is a simpler spike detection algorithm than kilosort, and I was hoping to use spike interface’s detectpeaks() for this. Does preprocessing (sample shift, failing channel triage, destriping) happen prior to the spike detection, or is it, again, run on raw data?
Yes you’ll have to destripe before running spike detection. Then there is a driftmap function to create a matplotlib raster plot using the spikes depths and times:
brainbox.plot.driftmap(spikes['times'], spikes['depths'], ax=axs[1, 0], **kwargs)
An example of the output is here.
Loading SpikeSorting Data — IBL Library documentation

Let me come back in another post about the derivative.

Best Regards