medigan 1.0.0

A modular package for automated synthetic data generation.

• :x: Problem 1: Data scarcity in medical imaging.

• :x: Problem 2: Scarcity of readily reusable generative models in medical imaging.

• :white_check_mark: Solution: medigan

  1. dataset sharing via generative models :gift:
  2. data augmentation :gift:
  3. domain adaptation :gift:
  4. synthetic data evaluation method testing with multi-model datasets :gift:

medigan provides functions for sharing and re-use of pretrained generative models in medical imaging.

Features:

• Instead of training your own, use one a generative models from medigan to generate synthetic data.

• Search and find a model in medigan using search terms (e.g. "Mammography" or "Endoscopy").

• Contribute your own generative model to medigan to increase its visibility, re-use, and impact.

Available models

Output type	Modality	Model type	Output size	Base dataset	Output examples	model_id	Hosted on	Reference
Breast Calcification	mammography	dcgan	128x128	Inbreast		00001_DCGAN_MMG_CALC_ROI	Zenodo (5187714)
Breast Mass	mammography	dcgan	128x128	Optimam		00002_DCGAN_MMG_MASS_ROI	Zenodo (5188557)	Alyafi et al (2019)
Breast Density Transfer	mammography	cyclegan	1332x800	BCDR		00003_CYCLEGAN_MMG_DENSITY_FULL	Zenodo (5547263)
Breast Mass with Mask	mammography	pix2pix	256x256	BCDR		00004_PIX2PIX_MMG_MASSES_W_MASKS	Zenodo (7093759)
Breast Mass	mammography	dcgan	128x128	BCDR		00005_DCGAN_MMG_MASS_ROI	Zenodo (6555188)	Szafranowska et al (2022)
Breast Mass	mammography	wgan-gp	128x128	BCDR		00006_WGANGP_MMG_MASS_ROI	Zenodo (6554713)	Szafranowska et al (2022)
Brain Tumors on Flair, T1, T1c, T2 with Masks	brain MRI	inpaint GAN	256x256	BRATS 2018		00007_INPAINT_BRAIN_MRI	Zenodo (7041737)	Kim et al (2020)
Breast Mass (Mal/Benign)	mammography	c-dcgan	128x128	CBIS-DDSM		00008_C-DCGAN_MMG_MASSES	Zenodo (6647349)
Polyp with Mask	endoscopy	pggan	256x256	HyperKvasir		00009_PGGAN_POLYP_PATCHES_W_MASKS	Zenodo (6653743)	Thambawita et al (2022)
Polyp with Mask	endoscopy	fastgan	256x256	HyperKvasir		00010_FASTGAN_POLYP_PATCHES_W_MASKS	Zenodo (6660711)	Thambawita et al (2022)
Polyp with Mask	endoscopy	singan	≈250x250	HyperKvasir		00011_SINGAN_POLYP_PATCHES_W_MASKS	Zenodo (6667944)	Thambawita et al (2022)
Breast Mass (Mal/Benign)	mammography	c-dcgan	128x128	BCDR		00012_C-DCGAN_MMG_MASSES	Zenodo (6755693)
Breast Density Transfer MLO	mammography	cyclegan	1332x800	OPTIMAM		00013_CYCLEGAN_MMG_DENSITY_OPTIMAM_MLO	Zenodo (6818095)
Breast Density Transfer CC	mammography	cyclegan	1332x800	OPTIMAM		00014_CYCLEGAN_MMG_DENSITY_OPTIMAM_CC	Zenodo (6818103)
Breast Density Transfer MLO	mammography	cyclegan	1332x800	CSAW		00015_CYCLEGAN_MMG_DENSITY_CSAW_MLO	Zenodo (6818105)
Breast Density Transfer CC	mammography	cyclegan	1332x800	CSAW		00016_CYCLEGAN_MMG_DENSITY_CSAW_CC	Zenodo (6818107)
Lung Nodules	chest x-ray	dcgan	128x128	NODE21		00017_DCGAN_XRAY_LUNG_NODULES	Zenodo (6943691)
Lung Nodules	chest x-ray	wgan-gp	128x128	NODE21		00018_WGANGP_XRAY_LUNG_NODULES	Zenodo (6943761)
Chest Xray Images	chest x-ray	pggan	1024x1024	ChestX-ray14		00019_PGGAN_CHEST_XRAY	Zenodo (6943803)
Chest Xray Images	chest x-ray	pggan	1024x1024	ChestX-ray14		00020_PGGAN_CHEST_XRAY	Zenodo (7046280)	Segal et al (2021)
Brain T1-T2 MRI Modality Transfer	brain MRI	cyclegan	224x192	CrossMoDA 2021		00021_CYCLEGAN_BRAIN_MRI_T1_T2	Zenodo (7074555)	Joshi et al (2022)

Model information can be found in the model documentation and in the global.json model metadata.

Installation

To install the current release, simply run:

pip install medigan

Getting Started

Examples and notebooks are located at examples folder

Documentation is available at medigan.readthedocs.io

Generation example

DCGAN

Create mammography calcification images using DCGAN model

# import medigan and initialize Generators
from medigan import Generators
generators = Generators()

# generate 6 samples with model 1 (00001_DCGAN_MMG_CALC_ROI). 
# Also, auto-install required model dependencies.
generators.generate(model_id=1, num_samples=6, install_dependencies=True)

CYCLEGAN

Create mammograms translated from Low-to-High Breast Density using CYCLEGAN model

from medigan import Generators
generators = Generators()
# model 3 is "00003_CYCLEGAN_MMG_DENSITY_FULL"
generators.generate(model_id=3, num_samples=1)

→

Search Example

Search for a model inside medigan using keywords

# import medigan and initialize Generators
from medigan import Generators
generators = Generators()

# list all models
print(generators.list_models())

# search for models that have specific keywords in their config
keywords = ['DCGAN', 'Mammography', 'BCDR']
results = generators.find_matching_models_by_values(keywords)

Get Model as Dataloader

We can directly receive a torch.utils.data.DataLoader object for any of medigan's generative models.

from medigan import Generators
generators = Generators()
# model 4 is "00004_PIX2PIX_MMG_MASSES_W_MASKS"
dataloader = generators.get_as_torch_dataloader(model_id=4, num_samples=3)

Visualize the contents of the dataloader.

from matplotlib import pyplot as plt
import numpy as np

plt.figure()
# subplot with 2 rows and len(dataloader) columns
f, img_array = plt.subplots(2, len(dataloader)) 

for batch_idx, data_dict in enumerate(dataloader):
    sample = np.squeeze(data_dict.get("sample"))
    mask = np.squeeze(data_dict.get("mask"))
    img_array[0][batch_idx].imshow(sample, interpolation='nearest', cmap='gray')
    img_array[1][batch_idx].imshow(mask, interpolation='nearest', cmap='gray')
plt.show()

Visualize A Model

With our interface, it is possible to generate sample by manually setting the conditional inputs or latent vector values. The sample is updated in realtime, so it's possible to observe how the images changes when the parameters are modified. The visualization is available only for models with accessible input latent vector. Depending on a model, a conditional input may be also available or synthetic segmentation mask.

from medigan import Generators

generators = Generators()
# model 10 is "00010_FASTGAN_POLYP_PATCHES_W_MASKS"
generators.visualize(10)

Contribute A Model

Create an init.py file in your model's root folder.

Next, run the following code to contribute your model to medigan.

• Your model will be stored on Zenodo.

• Also, a Github issue will be created to add your model's metadata to medigan's global.json.

• To do so, please provide a github access token (get one here) and a zenodo access token (get one here), as shown below. After creation, the zenodo access token may take a few minutes before being recognized in zenodo API calls.

from medigan import Generators
generators = Generators()

# Contribute your model
generators.contribute(
    model_id = "00100_YOUR_MODEL", # assign an ID
    init_py_path ="path/ending/with/__init__.py",
    model_weights_name = "10000",
    model_weights_extension = ".pt",
    generate_method_name = "generate", # in __init__.py
    dependencies = ["numpy", "torch"], 
    creator_name = "YOUR_NAME",
    creator_affiliation = "YOUR_AFFILIATION",
    zenodo_access_token = 'ZENODO_ACCESS_TOKEN',
    github_access_token = 'GITHUB_ACCESS_TOKEN',

Thank you for your contribution!

You will soon receive a reply in the Github issue that you created for your model by running generators.contribute().

Contributions in General

We welcome contributions to medigan. Please send us an email or read the contributing guidelines regarding contributing to the medigan project.