diff --git a/notebooks/collections_demos/rms_mutation_prediction/RMS-Mutation-Prediction-Expert-Annotations_exploration.ipynb b/notebooks/collections_demos/rms_mutation_prediction/RMS-Mutation-Prediction-Expert-Annotations_exploration.ipynb index 5d6707c..becd094 100644 --- a/notebooks/collections_demos/rms_mutation_prediction/RMS-Mutation-Prediction-Expert-Annotations_exploration.ipynb +++ b/notebooks/collections_demos/rms_mutation_prediction/RMS-Mutation-Prediction-Expert-Annotations_exploration.ipynb @@ -22,21 +22,24 @@ "id": "USpOiT6MgseH" }, "source": [ - "This tutorial is shared as part of the tutorials prepared by the Imaging Data Commons team and available at https://github.com/ImagingDataCommons/IDC-Tutorials/blob/master/notebooks.\n", + "This tutorial is shared as part of the tutorials prepared by the Imaging Data Commons (IDC) team and available at https://github.com/ImagingDataCommons/IDC-Tutorials/blob/master/notebooks.\n", "\n", "If you are new to IDC and DICOM for digital pathology applications, you may want to check out other introductory tutorials on this topic available here: https://github.com/ImagingDataCommons/IDC-Tutorials/tree/master/notebooks/pathomics.\n", "\n", - "This tutorial is aimed for the users of Imaging Data Commons that are interested to understand how to use annotations of slide microscopy images. You will learn how to:\n", + "This tutorial is aimed at users of the IDC that are interested to understand how to work with image-derived data for slide microscopy images and the ones interested in this specific collection of rhabdomyosarcoma annotations. You will learn how to:\n", "\n", - "select and download specific types of slide annotations, represented as polygons\n", - "parse annotations stored in DICOM Structured Reporting (SR) format\n", - "display and utilize the annotations for further analyses To learn more about the IDC, please visit the [IDC user guide](https://learn.canceridc.dev/).\n", - "If you have any questions, bug reports, or feature requests please feel free to contact us at the IDC discussion forum!\n", + "- select and download a specific type of image-derived data, namely delineations of tissue types, represented as polygons\n", + "- parse the content stored in DICOM Structured Reporting (SR) files\n", + "- display and use the tissue type delineations for further analyses\n", + "\n", + "To learn more about the IDC, please visit the [IDC user guide](https://learn.canceridc.dev/).\n", + "\n", + "If you have any questions, bug reports, or feature requests, please feel free to contact us at the [IDC discussion forum](https://discourse.canceridc.dev/)!\n", "\n", "---\n", "\n", "Initial version: May 2024 \n", - "Updated: Feb 2026" + "Updated: Aug 2026" ] }, { @@ -45,13 +48,22 @@ "source": [ "## Background\n", "\n", - "There are different ways to store annotations in DICOM, depending on the type of the annotation, and the specific DICOM object used. In this notebook we will discuss how to use planar closed contour annotations stored in DICOM **Structured Reporting** (SR) documents, which is also the representation of annotations adopted in the RMS-Mutation-Prediction-Expert-Annotations collection. The RMS-Mutation-Prediction-Expert-Annotations contains **expert-generated region annotations for rhabdomyosarcoma tumour slides** of 95 patients in the RMS-Mutation-Prediction collection released earlier. You can learn more about this collection in the following dataset record:\n", + "There are different ways to store image-derived data in DICOM. In this notebook, we discuss how to use **planar closed contours** stored in DICOM **Structured Reporting (SR)** documents, which is also the representation adopted for the annotations in the `RMS-Mutation-Prediction-Expert-Annotations` collection. This collection contains **expert-generated tissue type delineations for rhabdomyosarcoma tumor slides** of more than 90 patients in the `RMS-Mutation-Prediction` collection released earlier. You can learn more about this collection in the following dataset record:\n", "\n", "> Bridge, C., Brown, G. T., Jung, H., Lisle, C., Clunie, D., Milewski, D., Liu, Y., Collins, J., Linardic, C. M., Hawkins, D. S., Venkatramani, R., Fedorov, A., & Khan, J. (2024). Expert annotations of the tissue types for the RMS-Mutation-Prediction microscopy images [Data set]. Zenodo. https://doi.org/10.5281/zenodo.10462858\n", "\n", - "You can access this annotations collection in the IDC Portal using [this link](https://portal.imaging.datacommons.cancer.gov/explore/filters/?analysis_results_id=RMS-Mutation-Prediction-Expert-Annotations), or you can explore its content using this [custom Google Looker dashboard](https://tinyurl.com/idc-rms-annotations). As is the case with all of the content of IDC, both the images and annotations are publicly available and are free to download!\n", + "You can access this collection in the IDC Portal using [this link](https://portal.imaging.datacommons.cancer.gov/explore/filters/?analysis_results_id=RMS-Mutation-Prediction-Expert-Annotations), or you can explore its content using this [custom Google Looker dashboard](https://tinyurl.com/idc-rms-annotations). As with all content of the IDC, both the images and image-derived data are publicly available and free to download.\n", "\n", - "In the following notebook, the organization of annotations stored in SR objects is explained, and examples of how those annotations can be used and combined with the respective DICOM whole-slide images (WSIs) are provided." + "In the following, the organization of image-derived data stored in SR objects is explained, and examples of how those can be used and combined with the respective DICOM whole slide images (WSIs) are provided." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example\n", + "\n", + "As an example, you can open the following WSI with tissue type delineations, available at [this link](https://viewer.imaging.datacommons.cancer.gov/slim/studies/2.25.155017484756498730492136597238994838876/series/1.2.826.0.1.3680043.10.511.3.97882042930074581218321561123779480)." ] }, { @@ -71,11 +83,11 @@ "source": [ "## Prerequisites\n", "**Installations**\n", - "* **Install highdicom:** Most code in this notebook relies on the Python library [highdicom](https://highdicom.readthedocs.io/en/latest/introduction.html) which was specifically designed to work with DICOM objects holding image-derived information, e.g. annotations and measurements. Further and more detailed information on highdicom's functionality can be found in its [user guide](https://highdicom.readthedocs.io/en/latest/usage.html).\n", + "* **Install highdicom:** Most code in this notebook relies on the Python library [highdicom](https://highdicom.readthedocs.io/en/latest/introduction.html) which was specifically designed to work with DICOM objects holding image-derived information, e.g. annotations and segmentations. Further and more detailed information on `highdicom`'s functionality can be found in its [user guide](https://highdicom.readthedocs.io/en/latest/usage.html).\n", "* **Install wsidicom:** The [wsidicom](https://pypi.org/project/wsidicom/) Python package provides functionality to open and extract image or metadata from WSIs.\n", "* **Install rasterio:** [rasterio](https://rasterio.readthedocs.io/en/stable/installation.html) is a Python package to facilitate working with many types of rasterized data.\n", "* **Install idc-index:** The Python package [idc-index](https://pypi.org/project/idc-index/) facilitates queries of the basic metadata and download of DICOM files hosted by the IDC.\n", - "* **Install DCMTK:** [DCMTK (DICOM Toolkit)](https://dcmtk.org/en/) is an ope-source library that provides some convenient command-line tools for examining DICOM files." + "* **Install DCMTK:** [DCMTK (DICOM Toolkit)](https://dcmtk.org/en/) is an open-source library that provides some convenient command-line tools for examining DICOM files." ] }, { @@ -148,7 +160,7 @@ }, "source": [ "## Accessing DICOM SRs from the IDC\n", - "For the first step - to access and download the SR documents from the RMS collection - we use the Python package idc-index that we installed previously." + "For the first step - to access and download the SR documents from the RMS collection - we use the Python package `idc-index` that we installed previously." ] }, { @@ -217,7 +229,7 @@ "id": "ARf2CopUxG-d" }, "source": [ - "All annotations on its respective slide can also be viewed and explored in detail using the Slim viewer. Click the link generated by the code cell below to see one example. In the Slim viewer's interface switch the slider(s) at the right sidebar to make annotations visible. " + "The tissue type contours can be viewed and explored in detail on the corresponding slide using the Slim viewer. Click on the link generated by the code cell below to see one example. In the Slim viewer's expand `Annotations` at the bottom of the right sidebar, and toggle the slider(s) to display them. " ] }, { @@ -239,11 +251,11 @@ "id": "3TgXKP_AtSm0" }, "source": [ - "## Quick glance on the RMS collection\n", - "The RMS annotation collection contains expert-annotated regions of interest (ROI) for ~90 H&E stained slides of the RMS image collection. The ROIs delineate tissue regions with alveolar or embryonal rhabdomyosarcoma (ARMS or ERMA) as well as necrotic or connective tissue.\n", + "## Quick glance at the RMS collection\n", + "The `RMS-Mutation-Prediction-Expert-Annotations` collection contains expert-annotated regions of interest (ROIs) for more than 90 H&E-stained WSI from the `RMS-Mutation-Prediction` collection. The ROIs delineate tissue regions with alveolar or embryonal rhabdomyosarcoma (ARMS or ERMS) as well as necrotic or connective tissue.\n", "\n", - "Run the following code cell (please be patient for a minute or two) to get a quick overview of what the RMS annotation collection has to offer.\n", - "Running the hidden code cell below will output a dataframe that shows how many annotations of a certain tissue type (column) are available per slide (row). For each slide you also find a link to the Slim viewer to investigate the slide and its annotations visually." + "Run the following code cell (this may take a minute or two) to get a quick overview of what the `RMS-Mutation-Prediction-Expert-Annotations` collection has to offer.\n", + "It outputs a pandas DataFrame that shows how many delineations of a certain tissue type (column) are available per slide (row). Each row also incudes a link to the Slim viewer for visual inspection of the slide and tissue type contours." ] }, { @@ -989,7 +1001,7 @@ "id": "eb8MXftmOmfU" }, "source": [ - "If the RMS annotation collection sparked your interest and / or if you want to learn more about annotations in DICOM SR format, please continue with the following sections. They will show and explain in detail how these annotations can be read, interpreted and prepared for further use in downstream analyses." + "If the `RMS-Mutation-Prediction-Expert-Annotations` collection sparked your interest and / or if you want to learn more about image-derived data in DICOM SR format, please continue with the following sections. They will show and explain in detail how these annotations can be read, interpreted and prepared for further use in downstream analyses." ] }, { @@ -1000,23 +1012,23 @@ "source": [ "## Reading DICOM SRs\n", "\n", - "The following paragraphs provide an overview of the concept of DICOM SRs and explain how the RMS annotation collection is encoded.\n", + "The following paragraphs provide an overview of the concept of DICOM SRs and explain how the `RMS-Mutation-Prediction-Expert-Annotations` collection is encoded.\n", "\n", "Like other DICOM files, DICOM SRs are composed of a document header plus a document body. The document body of an SR can easiest be thought of as a tree of \"content items\". Each \"content item\" is essentially a name-value pair (similar to a DICOM Data Element) and conveys one single piece of information, e.g. a clinical finding.\n", "\n", "The DICOM standard defines multiple templates for SRs, each intended for a particular purpose. In our case, the DICOM SRs follow the [TID1500 \"Measurement Report\"](https://dicom.nema.org/medical/dicom/current/output/chtml/part16/chapter_A.html#sect_TID_1500) template, which is designed to comprise groups of measurements or qualitative evaluations (\"Measurement Group\"). The measurements or evaluations can either refer to an entire image/image frame or to a defined 2D or 3D ROI within an image depending on the type of the \"Measurement Group\".\n", "\n", - "A \"Measurement Group\" can be one of the following three types, however, the types can be mixed and matched within one SR:\n", + "A **\"Measurement Group\"** can be one of the following three types, however, the types can be mixed and matched within one SR:\n", "1. [Measurement and Qualitative Evaluation Group](https://highdicom.readthedocs.io/en/latest/tid1500.html#measurements-and-qualitative-evaluations-group-tid1501): contains measurements or evaluations that refer to one or more entire images or image frames.\n", "2. [Planar ROI Measurements and Qualitative Evaluations](https://highdicom.readthedocs.io/en/latest/tid1500.html#planar-roi-image-measurements-group-tid1410): contains measurements or evaluations and the 2D regions of interest those refer to.\n", "3. [Volumetric ROI Measurements and Qualitative Evaluations](https://highdicom.readthedocs.io/en/latest/tid1500.html#volumetric-roi-image-measurements-group-tid1411): contains measurements or evaluations and the 3D regions those refer to.\n", "\n", - "ROIs can be specified indirectly as a reference to a specific region which is stored in a separate DICOM file or they can be specified explicitly as contour points (e.g. a Circle or a Polygon) along with the measurements and/or findings. Contour points may either be 2D points in the image coordinate system or 3D points in the frame-of-reference coordinate system ([SCOORD vs. SCOORD3D](https://highdicom.readthedocs.io/en/latest/generalsr.html#graphic-data-content-items-scoord-and-scoord3d)). \n", + "ROIs can be specified indirectly as a reference to a specific region which is stored in a separate DICOM file or they can be specified explicitly as contour points (e.g. a circle or a polygon) along with the measurements and/or findings. Contour points may either be 2D points in the image-relative coordinate system or 3D points in the frame-of-reference coordinate system ([SCOORD vs. SCOORD3D](https://highdicom.readthedocs.io/en/latest/generalsr.html#graphic-data-content-items-scoord-and-scoord3d)). \n", "\n", - "The RMS annotations are stored in measurement groups of type \"Planar ROI Measurements and Qualitative Evaluations\", where each finding (e.g. necrotic tissue) is accompanied by the annotated region encoded as a planar SCOORD3D contour.\n", - "The following code uses the python library [highdicom](https://github.com/ImagingDataCommons/highdicom) to extract those findings and contours from a single DICOM SR of the RMS collection.\n", + "The annotations here are stored in measurement groups of type \"Planar ROI Measurements and Qualitative Evaluations\", where each finding (e.g. necrotic tissue) is accompanied by the spatial region encoded as a planar SCOORD3D contour.\n", + "The following code uses the python library `highdicom` to extract those findings and contours from a single DICOM SR of the RMS collection.\n", "\n", - "Explanations and guidance through implementation details in highdicom can be found [here](https://highdicom.readthedocs.io/en/latest/sr.html).\n", + "Explanations and guidance through implementation details in `highdicom` can be found [here](https://highdicom.readthedocs.io/en/latest/sr.html).\n", "A more detailed description of the theory of SR documents can be found in [this book](https://www.dclunie.com/pixelmed/DICOMSR.book.pdf), the [DICOM standard](https://dicom.nema.org/medical/dicom/current/output/chtml/part21/sect_a.3.html) or [this white paper](http://dx.doi.org/10.13140/RG.2.2.34520.62725).\n", "\n" ] @@ -1043,7 +1055,7 @@ "id": "hmfrSH2Bmwy8" }, "source": [ - "The following code snippet uses the previously defined function to access planar measurement groups in a randomly selected RMS SR document and displays one of those measurement groups:" + "The following code snippet uses the previously defined function to access and display one of the planar measurement groups in a randomly selected SR document:" ] }, { @@ -1147,7 +1159,7 @@ "id": "nwEwwoGA9_9x" }, "source": [ - "An alternative way to look at the content of a DICOM SR is by using DCMTK [`dsrdump` command line utility](https://support.dcmtk.org/docs/dsrdump.html), that will have one line for each node of the SR content tree - a lot more condensed and perhaps easier to understand representation than what you see above!" + "An alternative way to look at the content of a DICOM SR is by using DCMTK [`dsrdump` command line utility](https://support.dcmtk.org/docs/dsrdump.html), that will have one line for each node of the SR content tree - a lot more condensed and perhaps easier to understand representation than what you see above." ] }, { @@ -1306,7 +1318,7 @@ "id": "LRYNvxMAEHjA" }, "source": [ - "The following code shows how to access specific attributes such as the **finding type**, the **coordinates of the ROI** or the **FrameOfReferenceUID** those coordinates refer to." + "The following code shows how to access specific attributes such as the **finding type**, the **coordinates of the ROI** or the frame-of-reference (identified via **FrameOfReferenceUID**) those coordinates refer to." ] }, { @@ -1418,7 +1430,7 @@ "id": "cB3skOg7fOa-" }, "source": [ - "Next, we will get an overview on the number of ROI annotations available for the different findings and how they are distributed among the SR documents by creating a dataframe that holds the counts of ROI annotations per finding per SR." + "Next, we will create a pandas DataFrame that holds an overview on the number of ROIs available for the different findings and how they are distributed among the SR documents." ] }, { @@ -2155,7 +2167,7 @@ "id": "JBLfVFnxd6PG" }, "source": [ - "For each SR document (row, including Slim URL to view the corresponding slide), the dataframe lists the number of available ROIs per finding (column). A short look on the dataframes column headers shows that (as we learned before) there are ROIs available for four different findings:\n", + "For each SR document (row; including Slim URL to view the corresponding slide), the pandas DataFrame lists the number of available ROIs per finding (column). A short look on the column headers shows that (as we learned before) there are ROIs available for four different findings:\n", "* Embryonal rhabdomyosarcoma (abbreviated: ERMS)\n", "* Necrosis\n", "* Alveolar rhabdomyosarcoma (abbreviated: ARMS)\n", @@ -2206,7 +2218,7 @@ "id": "CnhU4rIQHHAQ" }, "source": [ - "The following histogram shows for each finding the number of available ROIs per SR document: " + "The following histogram shows the number of available ROIs per SR document for each finding: " ] }, { @@ -2257,9 +2269,9 @@ "id": "RcyoZmfJgVf-" }, "source": [ - "Apparently, most SRs contain one or few ROI annotations per finding (however it also becomes clear that not every SR contains ROIs for all findings). Also, few SR seem to contain a lot (> 25) annotations for a finding.\n", + "Apparently, most SRs contain one or few ROIs per finding (however, it also becomes clear that not every SR contains ROIs for all findings). Also, few SR seem to contain a lot (> 25) ROIs for a finding.\n", "\n", - "Based on this knowledge, we could now, for example, decide to select only SR documents that contain at most 10 ROI annotations across all findings:" + "Based on this knowledge, we could now, for example, decide to select only SR documents that contain at most 10 ROIs across all findings:" ] }, { @@ -2994,9 +3006,9 @@ "id": "HGUeTdchcCRb" }, "source": [ - "Now that we know what annotations are available and how to select the ones of interest, the question arises as to how to use them for analysis, model training or other tasks in combination with the WSI to which they refer.\n", + "Now that we know what annotations are available and how to select those of interest, the question arises as to how to use them for analysis, model training, or other tasks in combination with the WSI to which they refer.\n", "\n", - "To demonstrate this, we select a SR which we know contains exactly one measurement group for Connective tissue, one for Necrosis and one for ERMS. To find the corresponding WSI for a SR document we look for the content of the **ReferencedSeriesSequence**." + "To demonstrate this, we select a SR that we know contains exactly one measurement group each for Connective tissue, Necrosis, and ERMS. To find the corresponding WSI for a SR document, we look for the content of the `ReferencedSeriesSequence`." ] }, { @@ -3025,7 +3037,7 @@ "id": "3yu45z_7kLdp" }, "source": [ - "Using the referenced SeriesInstanceUID, we can now search for the respective DICOM files in the IDC: " + "Using the referenced `SeriesInstanceUID`, we can now search for the respective DICOM files in the IDC: " ] }, { @@ -3147,7 +3159,7 @@ "id": "fspks82oas46" }, "source": [ - "Here, we load (and visualize) the slide using wsidicom and define the slide level we want to work on. For defining the slide level it makes sense to check available levels first, otherwise computations might take longer if the requested level has to be downsampled from the next higher level. To keep the computing time as short as possible, we select the smallest available level, i.e. level 5." + "Here, we load (and visualize) the slide using wsidicom and define the slide level we want to work with. For defining the slide level, it is helpful to check available levels first, otherwise computations might take longer as the requested level has to be downsampled from the next higher level. To keep the computing time as short as possible, we select the smallest available level, i.e., level 5." ] }, { @@ -3285,7 +3297,7 @@ "### Converting a contour to a binary image mask on a certain level\n", "\n", "Mostly, computational pathology algorithms do not work with contours directly, but instead work with binary or label masks to encode the ROIs.\n", - "Thus, the following code provides functionality to generate binary image masks from SCOORD3D contours at a bounding-box level. Label maps on slide-level are discussed in the section after." + "Thus, the following code provides functionality to generate binary image masks from SCOORD3D contours at the bounding box level. Label maps (covering the whole slide) are discussed in the next section." ] }, { @@ -3342,7 +3354,7 @@ "id": "eiKDNtLxjns9" }, "source": [ - "The following code snippet iterates through our ROIs, findings and slide regions and generates plus visualizes a binary image mask for each of them." + "The following code snippet iterates through the ROIs, findings and slide regions and generates and visualizes a binary image mask for each of them." ] }, { @@ -3391,7 +3403,7 @@ "source": [ "### Converting multiple contours to a label map on a certain level\n", "\n", - "For many use cases binary or label maps for the whole slide are required instead of masks limited to the bounding-box area around the ROI.\n", + "For many use cases binary or label maps for the whole slide are required instead of masks limited to the bounding box area around the ROI.\n", "The following code shows how to generate a label map (one label per ROI) for the whole slide. " ] }, @@ -3492,7 +3504,7 @@ "id": "XNGxLX6QMRHa" }, "source": [ - "Lastly, as a lot of computational pathology tools use GeoJSON as their preferred file format for annotations, the following code shows how to easily export annotations captured in DICOM SR as GeoJSON file. This can then be imported into other tools for viewing and analysis, such as for example [QuPath](https://qupath.github.io/)." + "Since currenly, some computational pathology tools use GeoJSON as their preferred file format for annotations, the following code shows how to easily export the tissue type delineations captured in DICOM SRs here as GeoJSON. " ] }, { @@ -3553,11 +3565,6 @@ "\"Example" ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [] - }, { "cell_type": "markdown", "metadata": {