cvpr-kit / nucleiseg-in-histopathology-images Goto Github PK

The model in current form just look at grayscale images and binary labels, missing out on more informavtice details.

While traning loss continues to minimize, the validation loss doesnot after certain epoch, both in UNet3+ and ELUNet

Others

• pretraining with masked-autoencoder / self-supervised learning techniques*

• label soothing

• center loss

• r-drop

• cut-mix

Augmentation*

• color
• elastic demormation
• perlin or simplex noise agumentation

data sampler

https://towardsdatascience.com/demystifying-pytorchs-weightedrandomsampler-by-example-a68aceccb452

data sampler for additional training*

train a model $\rightarrow$ feature extraction $\rightarrow$ clustering (DBSCAN)$\rightarrow$ t-SNE

then, we can sample from cluster centers and boundary points of clusters to fine tunning the model.

we can use DINOv2 instead of training a model to get feature vectors

• https://medium.com/@aipapers/dinov2-from-meta-ai-finally-a-foundational-model-in-computer-vision-c51c89d7956d

DATASET

• annotating tissue images of several patients with tumors of different organs

• H&E staining images: purple for nuclei(핵), pink for cytoplasmic components(세포질)

  • H&E staining: https://www.leicabiosystems.com/en-kr/knowledge-pathway/he-staining-overview-a-guide-to-best-practices/

• the diversity of nuclei appearances across multiple organs and patients, the richness of staining protocols adopted at multiple hospitals

  • diversity of color

    $\rightarrow$ stain normalization

  • diversity of shape

    $\rightarrow$ elastic deformation agumentation

    $\rightarrow$ perlin / simplex agumentation

• Nuclear is Ellipse shape.

  • In my opinion, Neurnal Netowrks should learn elliptical shape

    $\rightarrow$ transformer

    $\rightarrow$ larger kernel than nuclei

    $\rightarrow$ cartoon/shape ( low+high freq.) + texture (mid+high freq.)?

classification model

• nuclei patch (fixed size) / non-nuclei patch classification model
• classify all the patches centered for each pixel
• we can use this result for candidate regions for fine segmentation
• remains scale problem

detection model

• ellipse detection
  • https://arxiv.org/pdf/2001.11584.pdf
  • https://link.springer.com/article/10.1007/s00138-022-01319-5

blob detection

• https://www.nature.com/articles/s41598-019-57223-y
  • Hessian convexity + U-Net

Architecture

• U-Net3++ : our base model*
• maxblurpool : I think it is mandatory for all the segmentation model.*
• cnn+transformer (texture, shape)
• GLU : https://medium.com/deeplearningmadeeasy/glu-gated-linear-unit-21e71cd52081
  • used for giving attention to a word in language model

Regularization

• l1+l2 regularization*
• drop-out (0.2, 0.3)*
• r-drop

Inference

• overlapped patch used in nnU-Net

Loss

• distribution-based loss*
  • class weight ($\alpha$ in focal loss)
    • https://medium.com/@zergtant/use-weighted-loss-function-to-solve-imbalanced-data-classification-problems-749237f38b75
  • focal loss
• Region-based loss*
  • Tversky (weight FP & FN; Dice/IoU/Jaccard)
  • RegionWise
• Boundary-based loss - time
  • HD loss
• Wasserstein distance** between two 2D Gaussian distributions
  • nuclei can be regard as 2-d gaussian distribution
  • https://openaccess.thecvf.com/content/WACV2021/papers/Pan_Ellipse_Detection_and_Localization_With_Applications_to_Knots_in_Sawn_WACV_2021_paper.pdf
  • it is better than KL divergence
  • compute the distance between target elliptical shape and candidate shape
  • Ellipse Proposal Network needed like Region Proposal Network in R-CNN
• my opinion: if we cannot make the NN learn the diversity of nuclei appearances with those losses, we have to add adversarial training strategy like GAN. GAN leans itself a loss somthing effective to a given problem