论文:Deep Learning in Neuroimaging Promises and challenges
笔记:陈亦新
目录
目录
词汇与新坑
introduction
DL for neuroimaging classification and regression
Multilayer perceptron models
Convolutional neural networks and graph convolutional networks
Recurrent neural network
GANs
Attention Modules
Promises and challenges
DL for analysis of dynamic activity and connectivity
modeling spatiotemporal dynamics using DL models
The combination of DL with conventional neuroimaging tools
Promises and challeges
DL for multimodal fusion
DL frameworks for multimodal fusion
Multimodal fusion applications in neuroimaging
Promises and challenges
Visualization and subytype discovery
Network visualization for biomarker discovery
Spectrum and subtype discovery using DL framework
词汇与新坑
FNC:functional network connectivity
Deep neural network with weight sparsity control and pre-training extracts hierarchical features and enhances classification performance: Evidence from whole-brain resting-state functional connectivity patterns of schizophrenia
ICA: independent component analysis
A novel 5D brain parcellation approach based on spatio-temporal encoding or resting fMRI data from deep residual learning 2021
Spatio-temporal dynamics of intrinsic networks in functional magnetic imaging data using recurrent neural netowrks
dFNC:dynamic FNC
Deep chronnectome learning via full bidirectional long short-term memory networks fro MCI diagnosis 2018
In contrast to natural images, which are collected under natural light, neuroimaging data consist mostly of raiological images. Because of this, thie noise distribution of neuroimaging varies depending on the acquisition used.
Rician noise in MRI, quantum noise in computed tomography (CT)
MRI中的Rician噪声,计算机断层扫描(CT)中的量子噪声等等。
❝
As showns in Table 1, neuroimaging data come with many other additional unique aspects, including the number of modalities, high dimensionality, low signal-to-noise ratio, and small sample sizes compared to natural image data.
MRI作为a noninvasive technique with high spatiotemporal resolution, 是当前研究最广泛的neuroimaging modality.
image.png
❝
Advanced neuroimaging analysis approaches are essential for linking brain function and structure to network and behavior.
先进的神经影像学分析方法对于将大脑功能和结构与网络和行为联系起来至关重要。
❝
Linear models and, in particular, flexible matrix decomposition approaches have contributed a lot to our current understanding. For instance, group independent component analysis (ICA), as a purely data-driven algorithm that reveals large-scale networks by making group inferences from funcional MRI (fMRI), is particularly useful for data fusion of multiple modalities, such as genome-wide single-nucleotide polymorphism (SNP) data or event-related potentials.
在这篇review综述中,four interrelated topics are covered:
classification/regression tasks, which are often studied in the context of brain-based biomarker studies, and key DL models.
DL-based dynamic analysis methods, which are useful for leveraging functional information in neuroimaging data.
multimodal fusion methods, which are needed to leverage complementary information amoung the modalities
visualization and subtype discovery, which is crucial for moving to clinical applications and providing clues regarding the underlying biological mechanisms.可视化和亚型发现,这对于转向临床应用和提供有关潜在生物学机制的线索至关重要。
DL for neuroimaging classification and regression
分类和回归是两个被广泛研究的监督学习任务。广义上说,两个任务的目标都是把x(神经成像数据映射到y(诊断,治疗反应和行为)。尽管神经成像数据高度多样化,还是可以分成两大类:structural imaging and functional imaging.
structural neuroimaging data 结构成像,例如structural MRI(sMRI)和diffusion MRI(dMRI弥散MRI),reflect voxel tissue density/volume or structural connectivity.反应了体素组织密度、体积或结构连通性
结构研究的主要目的是为了揭示anatomical relationships解剖关系 in the brain,这也可以用于预测。
Convolutional neural networks and graph convolutional networks
❝
Desprite the great success of CNNs, the non-Euclidean characteristic of graph features such as those obtained from FNC makes the general convolutionand no as well defined as on natural images.
❝
Similarly, a graph convolutional network GCN is a type of neural network architecture that can capture the graph structure and aggregate node information from the neighborhoods in a convolutional fashion with fewer learnable parameters. GCNs are useful in medical or biochemical applications with graph data such as FNC。
Recurrent neural network
❝
Compared to classical linear machine learning models, such as a hidden Markov model, an RNN models the long-term nonlinear mechanisms of the sequential data.
GANs
就是生成模型。
Attention Modules
❝
The use of an attention module was proposed to increase the representation power and improve interpretability by focusing on important brain regions and suppressing unnecessary ones, which is often combined with other DL models for interpretation, allowing the model to dynamically emphasize certain parts of input.
简单的说,就是可解释性。
Promises and challenges
❝
DL models designed for 3D and 4D neuroimaging data often consist of millions of parameters that require many samples for optimization.
DL for analysis of dynamic activity and connectivity
❝
The characterization of brain activity and connectivity dynamics (e.g., the chronnectome) is crucial for out understanding of brain function. However, uncovering relevant transient patterns in brain function is challenging because of the lack of computational tools that can effectively capture nonlinear dynamics from high-dimensional data.
Recent studies show that DL models, especially RNN-based networks, have the potential to capture whole-brain dynamic information and utilize the time-varing functional connectivity state profiles to expand our understanding of brain function and disorder.
此外RNN-ICA提出了结合RNN和ICA使用方案,which can explicitly optimize linear generative models to model temporal dynamics and infer intrinsic networks from time-series observations (the network structure and identified spatial maps are shown in RNN leverages ICA in figure 3)
Despite the variety of available models, most multimodal fusion strategies fall into the following two categories: prefusion and postfusion.
prefusion:concatenates raw features from multiple modalities before sending them to DLs; 这种很简单实现,但是当一个模态的特征维度远比其他的多的时候,或者由于数据结构的heterogeneity异构性,就会不可行。
postfusion: use DLs for learning feature representation of each modality and then concatenated thaem for subsequent tasks.更灵活,但是寻找最佳结构的时候也更费力。
除了基于concatenation-based postfusion,还有更先进的方法,考虑交叉模态的关系。Multimodal reconstruction,deep canonical correlation analysis (DCCA) and knowledge-transfer-based fusion are three popular multimodal fusion methods.
更先进的3中多模态融合的方法:
Multimodal reconstraction
deep canonical correlation analysis DCCA
knowledge-transfer-based fusion
image.png
Multimodal reconstraction:是AE在多模态数据当中的一种方法。
deep canonical correlation analysis DCCA:捕捉跨模态的相关性或者互信息。
这个的参考文献可以学一下:
image.png
knowledge-transfer-based fusion
Multimodal fusion applications in neuroimaging
Promises and challenges
Visualization and subytype discovery
Network visualization for biomarker discovery
流行的可视化方法可以分成四类:
interpretable local surrogates可解释性的局部替代物。两个经典方法是:
Local interpretable model-agnostic explanation LIME