KEDRI - FMRI

KEDRI
AUT

KEDRI
AUT Main Nav
Centre Banner
Main Content

FMRI

Using Deep Learning in Spiking Neural Networks to Make Sense of fMRI Data 

Nikola Kasabov, Lei Zhou, Jie Yang, Maryam Gholami Doborjeh 

 

Evolution of neurons’ activation degrees and the deep learning architecture formed in the SNNcube, when the subject is watching a picture (one trial of Class Pic)

Evolution of neurons’ activation degrees and the deep learning architecture formed in the SNNcube, when the subject is watching a picture (one trial of Class Pic). 

 Evolution of neurons’ activation degrees and the deep learning architecture formed in the SNNcube, when the subject is watching a sentence (one trial of Class Sen).

Evolution of neurons’ activation degrees and the deep learning architecture formed in the SNNcube, when the subject is watching a sentence (one trial of Class Sen). 

 

2D Visualization of the SNNcube activation maps for watching a picture (Class Pic)

Fig. 3. Aa(Class Pic): 2D Visualization of the SNNcube activation maps for watching a picture (Class Pic).

 

2D Visualization of the SNNcube activation maps for a watching a sentence (Class Sen).  

 Fig. 3. Aa(Class Sen): 2D Visualization of the SNNcube activation maps for  a watching a sentence (Class Sen).

 

Probability map estimated by t-test for Class Pic 

  Fig. 3. Ab(Class Pic):Probability map estimated by t-test for Class Pic.

 

Probability map estimated by t-test for Class Sen  

 Fig. 3. Ab(Class Sen): Probability map estimated by t-test for Class Sen.

Locations of activation neurons in the averaged SNNcube for Class Pic Fig. 3. Ba(Class Pic): Locations of activation neurons in the averaged SNNcube for Class Pic.

Locations of activation neurons in the averaged SNNcube for Class SenFig. 3. Ba(Class Sen): Locations of activation neurons in the averaged SNNcube for Class Sen.

 

25 ROIs’ (regions of interest) mapped in the SNNcube 

Fig. 3.C: 25 ROIs’ (regions of interest) mapped in the SNNcube; Abbreviations: CALC - calcarine; DLPFC - left dorsolateral prefrontal cortex; FEF - frontal eye fields; IFG - inferior frontal gyrus; IPL - left inferior parietal lobule; IPS - intraparietal sulcus; IT - inferior temporal gyrus; OPER - pars opercularis; PPREC - posterior precentral gyrus; SGA - supramarginal gyrus; SPL - superior parietal lobule; T - temporal lobule; TRIA - pars triangularis; SMA - supplementary motor area.

 

Neurons’ activation degrees at third snapshot when the subject is watching a picture (one trial of Class Pic)

 Fig. 4. A: Neurons’ activation degrees at 3rd snapshot when the subject is watching a picture (one trial of Class Pic).

 

Neurons’ activation degrees at 9th snapshot when the subject is watching a picture (one trial of Class Pic)

Fig. 4. A: Neurons’ activation degrees at 9th  snapshot when the subject is watching a picture (one trial of Class Pic).

Neurons’ activation degrees at 16th snapshot when the subject is watching a picture (one trial of Class Pic)

Fig. 4. A: Neurons’ activation degrees at 16th  snapshot when the subject is watching a picture (one trial of Class Pic).

 

Neurons’ activation degrees at 3rd snapshot when the subject is watching a sentence (one trial of Class Sen) Fig. 4. A: Neurons’ activation degrees at 3rd  snapshot when the subject is watching a sentence (one trial of Class Sen).

 

Neurons’ activation degrees at 9th snapshot when the subject is watching a sentence (one trial of Class Sen) Fig. 4. A: Neurons’ activation degrees at 9th  snapshot when the subject is watching a sentence (one trial of Class Sen).

 

Neurons’ activation degrees at 16th snapshot when the subject is watching a sentence (one trial of Class Sen)

Fig. 4. A: Neurons’ activation degrees at 16th  snapshot when the subject is watching a sentence (one trial of Class Sen).

 

The 3rd snapshot of the locations of neurons with the top 500 activation degrees for Class Pic Fig. 4. B (Class Pic)The 3rd  snapshot of the locations of neurons with the top 500 activation degrees for Class Pic.

 

The 9th snapshot of the locations of neurons with the top 500 activation degrees for Class Pic

Fig. 4. B (Class Pic)The 9th  snapshot of the locations of neurons with the top 500 activation degrees for Class Pic. 

 

The 13th snapshot of the locations of neurons with the top 500 activation degrees for Class Pic Fig. 4. B (Class Pic)The 13th  snapshot of the locations of neurons with the top 500 activation degrees for Class Pic.

The 16th snapshot of the locations of neurons with the top 500 activation degrees for Class Pic  Fig. 4. B (Class Pic)The 16th snapshot of the locations of neurons with the top 500 activation degrees for Class Pic.

 

The 3rd snapshot of the locations of neurons with the top 500 activation degrees for Class Sen  Fig. 4. B (Class Sen)The 3rd snapshot of the locations of neurons with the top 500 activation degrees for Class Sen.

 

The 9th snapshot of the locations of neurons with the top 500 activation degrees for Class Sen Fig. 4. B (Class Sen):The 9th snapshot of the locations of neurons with the top 500 activation degrees for Class Sen.

 

The 13th snapshot of the locations of neurons with the top 500 activation degrees for Class SenFig. 4. B (Class Sen):The 13th snapshot of the locations of neurons with the top 500 activation degrees for Class Sen.

 

The 16th snapshot of the locations of neurons with the top 500 activation degrees for Class Sen Fig. 4. B (Class Sen): The 16th snapshot of the locations of neurons with the top 500 activation degrees for Class Sen.

 

3D Visualization of typical chains of connections for class Pic  Fig. 4. C (Class Pic): 3D Visualization of typical chains of connections for class Pic.

 

2D Visualization of typical chains of connections for class Pic Fig. 4. C (Class Pic): 2D Visualization of typical chains of connections for class Pic.

 

3D Visualization of typical chains of connections for class Sen Fig. 4. C (Class Sen):3D Visualization of typical chains of connections for class Sen.

 

2D Visualization of typical chains of connections for class Sen

 Fig. 4. C (Class Sen): 2D Visualization of typical chains of connections for class Sen.

 

Modelling and Classification of fMRI data using the NeuCube Spiking neural Network Architecture

Benchmark STAR/PLUS fMRI dataset is available at http://www.cs.cmu.edu/afs/cs.cmu.edu/project/theo-81/www/ ) [1], [2]. During fMRI data acquisition, subjects were shown a picture and a sentence, and instructed to press a button to indicate whether the sentence correctly described the picture. Particular intervals of this data which were related to reading sentences (affirmative or negative sentences) were analysed, using benchmark LIBSVM https://www.csie.ntu.edu.tw/~cjlin/libsvm/ and the data classification results published in [3]. Similarly, this fMRI data was used for illustration of the NeuCube classification of fMRI data corresponding to different sentence polarities. The results are reported as follows:

Method Sessions and selected voxels for classification C1 (affirmative) C2 (negative) Total
NeuCube Session I: 20 voxels selected using SNR 80% 100% 90%
Session II: 20 pre-selected voxels from RDLPFC region 90% 80% 85%
Session III: 20 pre-selected voxels from LDLPFC region 90% 80% 85%
SVM [3] Session I: classification based on the LDLPFC’s voxels 64% 68% 66%
classification based on the RDLPFC’s voxels 65% 69% 67%
 

The feasibility analysis of the NeuCube architecture is not only limited to a higher classification accuracy, but a better visualisation and interpretation of the SNN models trained on the fMRI data as shown in Fig. 1. NeuCube platform is available free at http://www.kedri.aut.ac.nz/neucube.

fmri-NeuCube

Fig 1. The initial (A) and final (B) connectivity of a SNNc after training with two different data sets, related correspondingly to: affirmative sentence versus negative sentence. The final connectivity is also shown as a 2D projection (C). Positive connections are shown in blue and negative – in red.

References

[1] M. Just, “StarPlus fMRI data,” [Online]. Available: http://www.cs.cmu.edu/afs/cs.cmu.edu/project/theo-81/www/. [Accessed 13 07 2014].

[2] F. Pereira, “E-print Network,” 13 02 2002. [Online]. Available: http://www.osti.gov/eprints/topicpages/documents/record/181/3791737.html. [Accessed 2014 07 13].

[3] M. Behroozi and M. R. Daliri, “RDLPFC area of the brain encodes sentence polarity: a study using fMRI,” Brain imaging and behavior, pp. 1-12, 2014. 


Last updated: 17 May 2016 3:15pm

Auckland University of Technology, New Zealand | Copyright © | Privacy | Site map