Points
| Timestamp | Points | Contributor | Ad-hoc | References |
|---|---|---|---|---|
| Jan. 26, 2018 | 5 | @vessemer | No | Issues #294 |
| Jan. 26, 2018 | 20 | @vessemer | No |
Issues
#131
PR #298 |
| Jan. 26, 2018 | 6 | @vessemer | No | Issues #230 |
| Jan. 26, 2018 | 5 | @vessemer | No | Issues #295 |
| Jan. 26, 2018 | 3 | @vessemer | No | PR #290 |
| Jan. 22, 2018 | 15 | @vessemer | No |
Issues
#271
PR #292 |
| Jan. 12, 2018 | 2 | @vessemer | No | Issues #283 |
| Jan. 5, 2018 | 2 | @vessemer | No | Issues #271 |
| Jan. 5, 2018 | 2 | @vessemer | No | Issues #268 |
| Dec. 21, 2017 | 2 | @vessemer | No | Issues #145 |
| Dec. 20, 2017 | 3 | @vessemer | No | PR #225 |
| Dec. 12, 2017 | 5 | @vessemer | No |
Issues
#145
PR #233 |
| Dec. 5, 2017 | 45 | @vessemer | No |
Issues
#181
PR #185 |
| Dec. 5, 2017 | 60 | @vessemer | No |
Issues
#182
PR #189 |
| Dec. 5, 2017 | 150 | @vessemer | No |
Issues
#179
PR #188 |
| Nov. 23, 2017 | 3 | @vessemer | No |
Issues
#241
PR #242 |
| Nov. 21, 2017 | 3 | @vessemer | No |
Issues
#241
PR #242 |
| Nov. 15, 2017 | 5 | @vessemer | No |
Issues
#148
PR #197 |
| Oct. 30, 2017 | 5 | @vessemer | No |
Issues
#179
PR #188 |
| Oct. 26, 2017 | 3 | @vessemer | No |
Issues
#181
PR #185 |
| Oct. 26, 2017 | 3 | @vessemer | No |
Issues
#182
PR #189 |
| Oct. 24, 2017 | 3 | @vessemer | No |
Issues
#22
PR #176 |
| Oct. 24, 2017 | 3 | @vessemer | No |
Issues
#164
PR #183 |
| Oct. 24, 2017 | 1 | @vessemer | No |
Issues
#180
PR #184 |
| Oct. 22, 2017 | 3 | @vessemer | No |
Issues
#158
PR #175 |
| Oct. 21, 2017 | 3 | @vessemer | No |
Issues
#157
PR #174 |
| Oct. 21, 2017 | 5 | @vessemer | No |
Issues
#166
PR #172 |
| Oct. 19, 2017 | 5 | @vessemer | No |
Issues
#169
PR #170 |
| Oct. 16, 2017 | 2 | @vessemer | No | Issues #138 |
| Oct. 1, 2017 | 3 | @vessemer | No |
Issues
#143
PR #146 |
| Sept. 30, 2017 | 3 | @vessemer | No |
Issues
#21
PR #141 |
| Sept. 29, 2017 | 3 | @vessemer | No |
Issues
#25
PR #140 |
| Sept. 27, 2017 | 3 | @vessemer | No |
Issues
#26
PR #139 |
| Sept. 25, 2017 | 2 | @vessemer | No | Issues #120 |
| Sept. 25, 2017 | 40 | @vessemer | No |
Issues
#13
PR #78 |
| Sept. 22, 2017 | 5 | @vessemer | No | PR #132 |
| Sept. 19, 2017 | 5 | @vessemer | No |
Issues
#98
PR #119 |
| Sept. 11, 2017 | 2 | @vessemer | No | Issues #98 |
| Sept. 8, 2017 | 5 | @vessemer | No |
Issues
#2
PR #76 |
Activity
11 months, 3 weeks ago
@vessemer commented on PR #306: Reactivate tests and clear models
All rebased after #307, tests have been passed.
2 years ago
@vessemer commented on PR #304: Chose an appropriate DATA_SHAPE
Superceded by PRs #307 and #306
2 years ago
@vessemer commented on PR #307: Fix memory and styling errors
Glad to help :)
2 years ago
@vessemer commented on PR #307: Fix memory and styling errors
@reubano, I'm a bit confused with all those permutations, but it seems to be done :)
2 years ago
@vessemer commented on PR #307: Chose an appropriate DATA_SHAPE
Yes, but this was fixed in PR #306
2 years ago
@vessemer commented on PR #307: Chose an appropriate DATA_SHAPE
```
============================= test session starts ==============================
platform linux -- Python 3.6.3, pytest-3.1.3, py-1.5.2, pluggy-0.4.0
rootdir: /app, inifile:
collected 63 items
src/tests/test_classification.py ...
src/tests/test_classification_3dlrcnn.py ...
src/tests/test_cropping.py ...
src/tests/test_endpoints.py ..x.....
src/tests/test_generators.py .........
src/tests/test_grt123_preprocess.py ...
src/tests/test_identification.py xxx
src/tests/test_improved_segmentation.py ..
src/tests/test_loading.py .......
src/tests/test_preprocess_dicom.py ....
src/tests/test_segment_evaluate.py ........
src/tests/test_segmentation.py ....xxx
src/tests/test_volume_calculation.py ...
=========================== short test summary info ============================
XFAIL src/tests/test_endpoints.py::test_identify
Test was stopped after timeout
XFAIL src/tests/test_identification.py::test_identify_nodules_001
Test was stopped after timeout
XFAIL src/tests/test_identification.py::test_identify_nodules_003
Test was stopped after timeout
XFAIL src/tests/test_identification.py::test_identify_luna
Test was stopped after timeout
XFAIL src/tests/test_segmentation.py::test_nodule_segmentation
Test was stopped after timeout
XFAIL src/tests/test_segmentation.py::test_lung_segmentation
Test was stopped after timeout
XFAIL src/tests/test_segmentation.py::test_stop_timeout
Test was stopped after timeout
=============================== warnings summary ===============================
src/tests/test_endpoints.py::test_identify
/app/src/tests/../../src/preprocess/extract_lungs.py:34: RuntimeWarning: invalid value encountered in less
truncate=2.0) < intensity_th
src/tests/test_identification.py::test_identify_nodules_001
/app/src/tests/../../src/preprocess/extract_lungs.py:34: RuntimeWarning: invalid value encountered in less
truncate=2.0) < intensity_th
src/tests/test_identification.py::test_identify_nodules_003
/app/src/tests/../../src/preprocess/extract_lungs.py:34: RuntimeWarning: invalid value encountered in less
truncate=2.0) < intensity_th
-- Docs: http://doc.pytest.org/en/latest/warnings.html
============== 56 passed, 7 xfailed, 3 warnings in 991.87 seconds ==============
```
2 years ago
@vessemer commented on PR #307: Chose an appropriate DATA_SHAPE
@reubano, done, but I've added all code from #308, 'cause in other cases, it'll lead to wrong tests for a new `calculate_volumes` which now works with real-world units.
2 years ago
@vessemer commented on PR #304: Chose an appropriate DATA_SHAPE
@reubano, done by [`Rfactoring`](https://github.com/concept-to-clinic/concept-to-clinic/pull/306), [`Chose an appropriate DATA_SHAPE`](https://github.com/concept-to-clinic/concept-to-clinic/pull/307) and [`Allows to calculate_volume in mm`](https://github.com/concept-to-clinic/concept-to-clinic/pull/308).
2 years ago
@vessemer opened a new pull request: #308: Allows to calculate_volume in mm
[calculate_volume](https://github.com/concept-to-clinic/concept-to-clinic/pull/304/files#diff-d685e97b90047b1be62100544969f8ac) function now accept centroids in real-world units (mm) if meta was provided, which is logical and also leads to renewal of the [tests](https://github.com/concept-to-clinic/concept-to-clinic/compare/master...vessemer:calculate_volume?expand=1#diff-4dac9e3a237cb8ddd0ca837da32061ce)
The swap in tests is due to the `xyz` way to store volumetric data pushed by the PR #44.
## CLA
- [X] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years ago
@vessemer opened a new pull request: #307: Chose an appropriate DATA_SHAPE
Previous behaviour leads to various `memory error`s. It happened time to time 'cause of unreasonably large choice of the [`DATA_SHAPE`](https://github.com/concept-to-clinic/concept-to-clinic/commit/06117fb72b1bf03ad8b522d2f9e42af1b6711a2c#diff-638bbca55ea01b2d5eeb28b7e52321eeR21). The average maximum height of lungs for a grown man, according to [this research](https://www.researchgate.net/profile/Kevin_Capello/publication/221873867_Linear_dimensions_and_volumes_of_human_lungs_obtained_from_CT_images/links/5a1d79daaca2726120b2c4b5/Linear-dimensions-and-volumes-of-human-lungs-obtained-from-CT-images.pdf) (Table 2), is 282mm with std equals to 22mm and median 274mm (Table 2). Assuming normal distribution, [3 sigmas](https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule) aside the mean will lead to the probability of case `h` belong to the interval `P{|282 - h| <= 66} ~= 0.9974`. Taking into an account the fact that all algorithms which were described (not only implemented ones) works with spacing on z-axis >= 0.9, therefore, having an upper bound value of `(282 + 66) / 0.9 = 386` (voxels) for z-axis should be enough for all cases.
I've made `DATA_SHAPE` to be `(512, 512, 512)` which should fit in memory while having reserved space for the z-axis.
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years ago
@vessemer opened a new pull request: #306: Refactoring
Refactoring of [`evaluate_detection.py`](https://github.com/concept-to-clinic/concept-to-clinic/compare/master...vessemer:refactoring?expand=1#diff-4764e224c7f674ff2dc226aeafcfedb9) and uncommented tests in [`test_classification.py`](https://github.com/concept-to-clinic/concept-to-clinic/compare/master...vessemer:refactoring?expand=1#diff-923cd917610b38983b2ce7907638acc4), as was mentioned in [this](https://github.com/concept-to-clinic/concept-to-clinic/pull/298#pullrequestreview-92612801) comment.
## CLA
- [ x ] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years ago
@vessemer commented on PR #304: Chose an appropriate DATA_SHAPE
> Does this mean it wasn't previously being caught by the code linter that travis runs?
Not all of them, 'cause code linter indicate `E999 SyntaxError: invalid syntax` and not proceed after that.
> This just sounds like a refactoring, so the current test set should (hopefully) be able to confirm nothing broke
That's true.
> This one looks like an enhancement. I noticed that the x and z values were swapped. Is that something specific to using mm units? Or was it a latent bug that you caught?
The swap is done due to the `xyz` way to store volumetric data pushed by the PR #44.
2 years ago
@vessemer commented on PR #304: Chose an appropriate DATA_SHAPE
Thanks, @reubano! Let me describe in a bit more details: commit [All tests acitve](https://github.com/concept-to-clinic/concept-to-clinic/pull/304/commits/32e002d854c9633bdc4538a89933ab60ecbe9708) is just to uncomment the commented tests, then [Codestyle of evaluate_detection.py](https://github.com/concept-to-clinic/concept-to-clinic/pull/304/commits/e4ba764af6c6cb570fdb86efc2632eb59cbf7008) commit is merely about that some thing didn't look right, though in the last commit I've included (additional to the DATA_SHAPE things):
- Remove redundant [`self.scale_factor`](https://github.com/concept-to-clinic/concept-to-clinic/pull/304/files#diff-7ebde87c389dab64f0c9e8a81e6b6391L40) which is neither dependend on DATA_SHAPE nor on [`self.input_shape`](https://github.com/concept-to-clinic/concept-to-clinic/pull/304/files#diff-7ebde87c389dab64f0c9e8a81e6b6391L39) by [this](https://github.com/vessemer/concept-to-clinic/blame/e4ba764af6c6cb570fdb86efc2632eb59cbf7008/prediction/src/algorithms/segment/src/models/simple_3d_model.py#L64-L73) and [this](https://github.com/vessemer/concept-to-clinic/blame/e4ba764af6c6cb570fdb86efc2632eb59cbf7008/prediction/src/algorithms/segment/src/models/simple_3d_model.py#L48-L55).
- [calculate_volume](https://github.com/concept-to-clinic/concept-to-clinic/pull/304/files#diff-d685e97b90047b1be62100544969f8ac) function now accept centroids in real world units (mm) if meta was provided, which is logical and also leads to renewal of the [tests](https://github.com/concept-to-clinic/concept-to-clinic/pull/304/files#diff-4dac9e3a237cb8ddd0ca837da32061ce)
2 years ago
@vessemer commented on PR #304: Chose an appropriate DATA_SHAPE
@lamby, initially I've created this PR to uncomment (activate) all tests, which I forget to uncomment in my previous PR #298, but then I've decided to [figure out an appropriate `DATA_SHAPE`](https://github.com/concept-to-clinic/concept-to-clinic/pull/304#issuecomment-361691349), along with other fixes, thus I've renamed this PR.
2 years ago
@vessemer commented on PR #304: All tests acitve
@reubano, done!
2 years ago
@vessemer commented on PR #304: All tests acitve
I've made `DATA_SHAPE` to be `(512, 512, 512)` which should fit in memory while having reserved space for the z-axis.
2 years ago
@vessemer commented on PR #304: All tests acitve
@reubano, It happens time to time 'cause of unreasonably large choice of the [`DATA_SHAPE`](https://github.com/concept-to-clinic/concept-to-clinic/commit/06117fb72b1bf03ad8b522d2f9e42af1b6711a2c#diff-638bbca55ea01b2d5eeb28b7e52321eeR21). The average maximum height of lungs for a grown man, according to [this research](https://www.researchgate.net/profile/Kevin_Capello/publication/221873867_Linear_dimensions_and_volumes_of_human_lungs_obtained_from_CT_images/links/5a1d79daaca2726120b2c4b5/Linear-dimensions-and-volumes-of-human-lungs-obtained-from-CT-images.pdf) (Table 2), is 282mm with std equals to 22mm and median 274mm (Table 2). Assuming normal distribution, [3 sigmas](https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule) aside the mean will lead to the probability of case `h` belong to the interval `P{|282 - h| <= 66} ~= 0.9974`. Taking into an account the fact that all algorithms which were described (not only implemented ones) works with spacing on z-axis >= 0.9, therefore, having an upper bound value of `(282 + 66) / 0.9 = 386` for z-axis should be enough for all cases.
2 years ago
@vessemer opened a new pull request: #304: All tests acitve
Uncommented tests, as was mentioned in [this](https://github.com/concept-to-clinic/concept-to-clinic/pull/298#pullrequestreview-92612801) comment.
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years ago
@vessemer commented on PR #298: Classification model pipeline
@lamby, good point, done :)
2 years ago
@vessemer commented on PR #298: Classification model pipeline
Okh, [this](https://github.com/concept-to-clinic/concept-to-clinic/blame/413bc74311579bc3eaa9ba9a147eb8bdc937b093/prediction/src/preprocess/preprocess_ct.py#L167) line along with removal [this](https://github.com/concept-to-clinic/concept-to-clinic/blame/fc9137c2c110f035cdd068bf4b2a5ec479357fc2/prediction/src/preprocess/preprocess_ct.py#L111) one makes me suffer for a while and force to code and tests updates. Unfortunately, at the moment of PR #272 I did not have the opportunity to provide a review, now to be clear:
First: `ndarray` coordinates of the real world point should be computed in this way: `(point - origin) / spacing`and not [`(point - origin) * spacing`](https://github.com/concept-to-clinic/concept-to-clinic/blame/fc9137c2c110f035cdd068bf4b2a5ec479357fc2/prediction/src/preprocess/preprocess_ct.py#L111), where `spacing` is the shape of one voxel in real-world units.
Second: `meta.spacing` stores information of aforementioned voxel's shape, and if we apply affine transformation such as zoom, then `meta.spacing` should changes too, which is not the previous behaviour. The way that `preprocess_ct.PreprocessCT` handle `spacing` parameter is to zoom an ndarray exactly to this new `spacing` by setting `zoom_fctr` to be `old_spacing / new_spacing` this means that we need no more to store the `old_spacing` in `meta` and at the same time it's enough to store only current spacing for real-world - ndarray coordinates translations. That's why [this](https://github.com/concept-to-clinic/concept-to-clinic/blame/fc9137c2c110f035cdd068bf4b2a5ec479357fc2/prediction/src/preprocess/preprocess_ct.py#L111) line is important.
This mistakes were propagated to the tests in a way of coordinates picking.
Quick check:
2 years ago
@vessemer commented on issue #294: Nodules augmentation
@reubano, I've meant, that knowing which of three types of nodules we are dealing with leads to better nodule inserting :)
2 years ago
@vessemer commented on issue #294: Nodules augmentation
@WGierke, that was my concern, though we may approach it by extracting probability maps of possible locations or just try as is.
2 years ago
@vessemer commented on issue #294: Nodules augmentation
@reubano, suppose, we have two patches, with and without nodule:
If we can segment out the nodule carefully, then it can be cropped and inserted in a free spot, like that:
2 years ago
@vessemer commented on PR #298: Classification model pipeline
Currently, I'm working on the training part adjustment of `grt123` algorithm.
2 years ago
@vessemer opened a new pull request: #298: Classification model pipeline
Pipeline to train and predict by the model was provided along with an interface for `classification_model`.
## Reference to official issue
#131
## Metrics:
Model' CPM score was described in PR #292:
[CPM](https://doi.org/10.1109/TMI.2010.2072789) over 10-Fold cross validation:
| 0.125 | 0.25 | 0.5 | 1 | 2 | 4 | 8 | Score ([CPM](https://doi.org/10.1109/TMI.2010.2072789)) |
|:-----:| ----- | ----- | ----- | ----- | ----- | ----- |:-----:|
| 0.595 | 0.670 | 0.731 | 0.793 | 0.835 | 0.868 | 0.887 | 0.76 |
| Item | Config |
| :---: | :----: |
| GPU | NVIDIA TITAN X |
| GPU memory usage | 12GiB for batch size 32 |
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years ago
@vessemer created a new issue: #295: First page issues
1) It's not obvious at least for me :( how to proceed to the next pages:
2) The slicing range button appears far beneath the view element:
## Checklist before submitting
- [x] I have confirmed this using the officially supported Docker Compose setup using the `local.yml` configuration and ensured that I built the containers again and they reflect the most recent version of the project at the `HEAD` commit on the `master` branch
- [x] I have searched through the other currently open issues and am confident this is not a duplicate of an existing bug
- [x] I provided a **minimal code snippet** or list of steps that reproduces the bug.
- [x] I provided **screenshots** where appropriate
- [x] I filled out all the relevant sections of this template
2 years ago
@vessemer commented on issue #294: Nodules augmentation
I'm going to work on this issue in a while,
but first, I'm looking for public opinion and cooperation :)
2 years ago
@vessemer created a new issue: #294: Nodules augmentation
This issue dedicated to one possible improvement in the current pipeline of `grt123` algorithm:
As it was said, they used under and oversampling w.r.t. nodules' diameters combined with [hard negative mining](https://www.reddit.com/r/computervision/comments/2ggc5l/what_is_hard_negative_mining_and_how_is_it/ckiuu9i/) to train over imbalanced data. The data have been augmented on each iteration, to preserve generalisation capability.
> For both networks data augmentation is used to artificially increase the amount of data on which they can be trained.
The augmentation described in their pipeline is trivial affine transformations which have been described and implemented in PR #132. Though it's good enough to achieve eminent results, I think there is an area of investigation and the potential gap to be filled. My proposal is to classify nodules by their type of appearance and proximity to other structures: Juxta Plural, on the coastal line of lungs, Juxta Vascular, which appears on the blood vessels and typically grow most rapidly [1](https://www.ncbi.nlm.nih.gov/pubmed/24733031), and Isolated which placed in the lung. These three types are depicted below, accordingly.
Then, since we can well segment nodules, crop them out and based on vessels & lungs segmentation (described in #138) find appropriate spots to place them in the aim to artificially enlarge dataset and therefore improve generalisation capability of the `grt123` model.
Any thoughts will be highly appreciated!
## Acceptance creteria
- [ ] at least 3-folds cross-validation should be performed, demonstraiting [logloss](https://github.com/concept-to-clinic/concept-to-clinic/pull/290) and [CPM](https://github.com/concept-to-clinic/concept-to-clinic/pull/292).
2 years ago
@vessemer commented on issue #230: Ask a Clinician! (add a question, get points)
> Most CAD systems in clinical use today have their internal threshold set to operate somewhere between 1 to 4 false positives per scan on average. Some systems allow the user to vary the threshold. To make the task more challenging, we included low false positive rates in our evaluation. This determines if a system can also identify a significant percentage of nodules with very few false alarms, as might be needed for CAD algorithms that operate more or less autonomously.
- Based on the quote from the LUNA16 [evaluation page](https://luna16.grand-challenge.org/evaluation/), are there some changes in preference, i.e. which amount of false positives per scan (FPPS) should treat as appropriate? Also, I want point to the fact that sensitivities which correspond to lower FPPS are less stable. It can be observed by adding trivial test time augmentation (TTA), e.g. flip along some axis. This trick allows raising a tail and also reducing the variance of FROC without any other manipulations with an algorithm itself. Following plots, without TTA and with TTA accordingly:
- It might be helpful (for algorithms training as well) to acquire types of nodules along with diameter and centroids' coordinates. Here I mean following types: Juxta Plural, Juxta Vascular, Isolated.
- Whilst we inevitably face false positive cases not all of them are obviously non-pathological errors, at least for me :) Would it be worth to try to cluster them out?
Random examples of false positives:
2 years ago
@vessemer commented on PR #292: #271 LUNA16 evaluation
Btw, for 10 fold cross-validation was used split from the [data page](https://luna16.grand-challenge.org/data/) suggested by the LUNA16 authors.
2 years ago
@vessemer commented on PR #292: #271 LUNA16 evaluation
Three months ago I've worked on algorithms unification and training parts, but then master branch diverges from mine and I abandoned that idea for a while. The code I've written lies [here](https://github.com/vessemer/concept-to-clinic/tree/2_classify/prediction/src/algorithms/classify/src). I hope to return to this soon.
2 years ago
@vessemer commented on PR #292: #271 LUNA16 evaluation
I've used my old [pipeline](https://github.com/vessemer/LungCancerDetection/blob/master/IPython/3D_NET.ipynb) since it fits my currently available one TITAN X GPU, also the code looks not that good :)
2 years ago
@vessemer opened a new pull request: #292: #271 LUNA16 evaluation
This allows to compute [FROC](https://doi.org/10.1093/jicru/ndx009) and [CPM](https://doi.org/10.1109/TMI.2010.2072789) metrics with bootstrapping for the classification and detection tasks.
## Description
Free-Response Receiver Operating Characteristic ([FROC](https://doi.org/10.1093/jicru/ndx009)) and competition performance metric ([CPM](https://doi.org/10.1109/TMI.2010.2072789)). It computes an average of the seven sensitivities measured at several false positives per scan (FPPS) thresholds, more concretely, at each FPPS ∈ {0.125, 0.25, 0.5, 1, 2, 4, 8} true positive rate was computed. Mean of which forms the CPM as discussed in #271.
## Reference to official issue
This reference to the #271.
## How Has This Been Tested?
[CPM](https://doi.org/10.1109/TMI.2010.2072789) over 10-Fold cross validation:
| 0.125 | 0.25 | 0.5 | 1 | 2 | 4 | 8 | Score ([CPM](https://doi.org/10.1109/TMI.2010.2072789)) |
|:-----:| ----- | ----- | ----- | ----- | ----- | ----- |:-----:|
| 0.595 | 0.670 | 0.731 | 0.793 | 0.835 | 0.868 | 0.887 | 0.76 |
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years ago
@vessemer commented on issue #283: Continuous improvement of nodule segmentation and volume estimates
Yes, sure. I'll add some comments in `trained_model.calculate_volume` with my next commit, since there is some obscurity :)
2 years, 1 month ago
@vessemer commented on issue #283: Continuous improvement of nodule segmentation and volume estimates
@caseyfitz did you carefully read the code of `trained_model.calculate_volume`? [This] code treat centroids as connected components and deals well with them.
2 years, 1 month ago
@vessemer commented on issue #271: Evaluation Pipeline for Models
For the both identification and classification (false positive reduction) tasks was proposed a handy evaluation [framework](https://luna16.grand-challenge.org/evaluation/) by the LUNA16 authors.
They employed Free-Response Receiver Operating Characteristic ([FROC](https://doi.org/10.1093/jicru/ndx009)) and competition performance metric ([CPM](https://doi.org/10.1109/TMI.2010.2072789)). It computes an average of the seven sensitivities measured at several false positives per scan (FPPS) thresholds, more concretely, at each FPPS ∈ {0.125, 0.25, 0.5, 1, 2, 4, 8} true positive rate was computed. Mean of which forms the CPM. From my point of view, it worth to pay attention to the CPM neither the logloss.
I can work on that to adjust their pipeline, if no one mind.
2 years, 1 month ago
@vessemer commented on issue #268: Classification throws RuntimeError for real nodule location
If you store coordinates in mm, then you don't need to keep old `meta.spacing` values. That is correct, that `params.spacing` usage is currently limited to the one case only, nonetheless, I do not see a point of the restriction you proposed for the `params.spacing` variable.
2 years, 1 month ago
@vessemer commented on issue #268: Classification throws RuntimeError for real nodule location
@Serhiy-Shekhovtsov, Noup, the point is that we don't need to store zooming factor, cause it computed through `meta.spacing` and `params.spacing` [here](https://github.com/concept-to-clinic/concept-to-clinic/blob/666fb1bf410cbd80eb7928d0e3c5f2433ad668a5/prediction/src/preprocess/preprocess_ct.py#L141). Then `meta.spacing` updates [here](https://github.com/concept-to-clinic/concept-to-clinic/blob/666fb1bf410cbd80eb7928d0e3c5f2433ad668a5/prediction/src/preprocess/preprocess_ct.py#L145). Thus position of a candidate given in mm should be obtained via `meta.spacing`. Also, one CT scan may be zoomed several times only carry the final spacing not the sequence of zooming factors.
2 years, 1 month ago
@vessemer commented on issue #268: Classification throws RuntimeError for real nodule location
@Serhiy-Shekhovtsov, thanks for mentioning me, if I correct, then the input coordinates should always be in real units — mm, not in voxels, and for this purpose, we always store current spacing in [meta](https://github.com/concept-to-clinic/concept-to-clinic/blob/666fb1bf410cbd80eb7928d0e3c5f2433ad668a5/prediction/src/preprocess/load_ct.py#L135-L150) which updates with [zoom](https://github.com/concept-to-clinic/concept-to-clinic/blob/666fb1bf410cbd80eb7928d0e3c5f2433ad668a5/prediction/src/preprocess/preprocess_ct.py#L145).
2 years, 1 month ago
@vessemer commented on PR #236: Implemented Imagery selection screen
I've implemented the aforementioned features without `EmitBus` usage, by this [commit](https://github.com/concept-to-clinic/concept-to-clinic/pull/233/commits/4c1420fada44e2665b52aaf56a3fba4a24c76745).
Double the [demonstration](https://github.com/concept-to-clinic/concept-to-clinic/pull/233#issuecomment-346517671) here:
2 years, 2 months ago
@vessemer commented on PR #233: #145 import image series
2 years, 2 months ago
@vessemer commented on PR #242: Allow to select slice
2 years, 3 months ago
@vessemer commented on PR #236: Implemented Imagery selection screen
@Serhiy-Shekhovtsov, in your PR I'm mainly concern with the `Case` model usage, as it was mentioned in the [review](https://github.com/concept-to-clinic/concept-to-clinic/pull/236#pullrequestreview-77660241). I guess it will be better if we allow to user switch between `Case.objects` without losing them. Also, there should be an opportunity to create more than one `Case.object` for a single `ImageSeries.object` (in case of several observators). My PR #233 aimed to deal with such case while introducing minimum changes into existing interface.
I also guess that it is not a good idea to introduce an `EmitBus` listener in such a general component as `TreeViews`.
2 years, 3 months ago
@vessemer commented on issue #150: Vue: implement 'Segmentation' UI component(s)
I'm currently working on that, but only over `cornerstone-tools` segmentation part, thus will be glad to collaborate on this :)
2 years, 3 months ago
@vessemer opened a new pull request: #242: Allow to slect slice
At this moment only slide preview was available, this PR allows selecting the desired slice from a list by clicking on its name.
## Reference to official issue
It also fixes issue #241
## Screenshots (if appropriate):
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 3 months ago
@vessemer commented on issue #145: 'Import' UI element should be able to create backend `ImageSeries` model instance
Good time of a day, @Serhiy-Shekhovtsov
Do not worry, I didn't ignore your comments at all. I've been also concerned about the right way of collaboration, once I got into this project. I've noted your [question](https://github.com/concept-to-clinic/concept-to-clinic/issues/4#issuecomment-327718051) in the issue #4, left unanswered. что-то типа, I'm used to the situations when someone takes the issue and then does nothing. These situations prevent the normal course of work. That is why I've decided to implement it after waiting for a sufficient amount of time.
You wrote [request](https://github.com/concept-to-clinic/concept-to-clinic/issues/145#issuecomment-344731667) about 4 days ago while this issue can easily be implemented in one evening. During this time you've made another PRs #235 and #231. I also checked your fork and found nothing related to this issue. So I decided that you're not interesting in working on this issue at the moment.
I also do not think, that our PRs are waisting of time, even though only one will be accepted, 'cause it usually lead to quality improvement.
From my point of view this discussion itself is a wasting of time.
2 years, 3 months ago
@vessemer commented on PR #233: #145 import image series
2 years, 3 months ago
@vessemer opened a new pull request: #233: #145 import image series
Import image series implementation.
## Reference to official issue
This addresses #145
## Screenshots (if appropriate):
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 3 months ago
@vessemer commented on PR #197: #148 Manipulations with DICOM vue
Well, I guess that for now, it's better to take `cornerstone-tools` as is and merely restrict `jquery` usage by this case only. It'll be beneficial to end up with some minimal version of `jquery` required by `cornerstone-tools` or even get rid of it at all, but, perhaps it'll be an issue for the packaging milestone, isn't it?
@louisgv do you mind creating an issue for this?
2 years, 3 months ago
@vessemer commented on PR #197: #148 Manipulations with DICOM vue
Hey, @lamby, was unavailable for a while, there already exists [PR](https://github.com/chafey/cornerstoneTools/pull/276) for it
2 years, 3 months ago
@vessemer commented on PR #184: #180 CSS adjust margin
@lamby, only now I noted that this issue was `FIRST PR ONLY`, due to the lack of sleep treat it as `POINT BOUNTY`, my excuse.
2 years, 3 months ago
@vessemer opened a new pull request: #197: #148 Manipulations with DICOM vue
Manipulations over DICOM such as `scroll`, `window width` / `window centre`, etc. series were requested by both #148 and #149. Were implemented by [`cornerstoneTools`](https://github.com/chafey/cornerstoneTools). It took a while to distinguish sustainable versions, compatible with this project:
```javascript
"cornerstone-core": " 0.13.0",
"cornerstone-tools": " 0.10.0",
"jquery": "^3.2.1"
```
I'd like to separate them from drawing features and made them part of [`OpenDICOM`](https://github.com/concept-to-clinic/concept-to-clinic/compare/master...vessemer:148_detect_and_select?expand=1#diff-70d179422138a3042c2135a282065626), still any suggestions are highly appreciated!
## Description
In this PR `scroll`, `window width` / `window centre` DICOM manipulators were implemented.
## Reference to official issue
This partially addresses #148
## Screanshots
### Scrolling:
### WWWC:
P.S. there are errors in `cornerstone-tools`' [README](https://github.com/chafey/cornerstoneTools/blame/master/README.md#L51-L55) should be `cornerstoneTools.external` not in plural.
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 3 months ago
@vessemer commented on PR #188: #179 DICOM image viewer
Hey, @antonow, thanks, it looks much better now
I spent last sleepless nights working on this PR, the main difficulties were related with the [`cornerstoneTool`](https://github.com/chafey/cornerstoneTools), besides for it has highly unstable behaviour (coverage status is 4% only) it also crashes with each update of the dependent [`cornerstone-core`](https://github.com/chafey/cornerstone). The latest versions sustainable for our setup are:
```javascript
"cornerstone-core": " 0.13.0",
"cornerstone-tools": "0.9.1"
```
Relating events implementation: It seems that every guide about `$on` and `$emit` on the internet is wrong (at least for versions described above) so that the decision was made to use the bus to handle events. Here is the current view of DICOM slice selection:
2 years, 3 months ago
@vessemer opened a new pull request: #189: RadReport mockup
Implemented mockup under [`ReportAndExport`](https://github.com/concept-to-clinic/concept-to-clinic/compare/master...vessemer:182_RadReport_mockup?expand=1#diff-e715cd99d4fc9baaa867066276d01308) screen over VueJS
## Reference to official issue
This addresses #182
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 3 months ago
@vessemer opened a new pull request: #188: DICOM image viewer
The ability to show DICOM images is one of the most important parts for this project. There are existing libraries that provide a way to show DICOM images that can help with this task. It were two main candidates with proper library to choose from: [ivmartel](https://github.com/ivmartel) and [chafey](https://github.com/chafey). The decision was made to use the [cornerstone](https://github.com/chafey/cornerstone) library for the reason that it flexible and can provide the project with ability to show images in an interactive way. It is also was much easier to use with VueJS than its competitor.
Two main difficulties were met during the work on this issue. The first one was the problem with pixelData serialization. It took some time before a good solution was found.
The second problem was to make library work correctly with VueJS. Choosing the most appropriate llibrary also refers to this problem. A lot of time was spent on testing and studying different libraries and browsing repositories before the appropriate solution was found.
## Description
This is the implementation of DICOM image viewer that was made with the help of [cornerstone](https://github.com/chafey/cornerstone) and [cornerstoneTools](https://github.com/chafey/cornerstoneTools). The image is being taken by the part of the url that adresses it.
## Reference to official issue
This address #179
## Motivation and Context
There is a strong need in the way of showing DICOM imagery. It is impossible to make that project at least somehow usable by common people without DICOMs actually being shown. Furthermore, issues [#148](https://github.com/concept-to-clinic/concept-to-clinic/issues/148) and [#150](https://github.com/concept-to-clinic/concept-to-clinic/issues/150) depend on it.
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 3 months ago
@vessemer opened a new pull request: #185: #181 RadReport's template fields
The models were adopted to suit the [RSNA Radiology Reporting Templates](http://www.radreport.org/template/0000268) for CT Lung Cancer Screening.
## Description
All feilds were assigned to the [`Case`](https://github.com/concept-to-clinic/concept-to-clinic/compare/master...vessemer:181_RadReport_template_fields?expand=1#diff-421ccf31e89cb76aa475b257a8904f4eR55) model, except for those listed below which were assigned to the [`Nodule`](https://github.com/concept-to-clinic/concept-to-clinic/compare/master...vessemer:181_RadReport_template_fields?expand=1#diff-421ccf31e89cb76aa475b257a8904f4eL41) model:
- `diameter`
- `appearance_feature`
- `density_feature`
- `image_no`
Due to the symmetry of right and left pleural spaces, a new model [`PleuralSpace`](https://github.com/concept-to-clinic/concept-to-clinic/compare/master...vessemer:181_RadReport_template_fields?expand=1#diff-421ccf31e89cb76aa475b257a8904f4eR19) was created and it consists of:
- `effusion`
- `calcification`
- `thickening`
- `pneumothorax`
The [`Case`](https://github.com/concept-to-clinic/concept-to-clinic/compare/master...vessemer:181_RadReport_template_fields?expand=1#diff-421ccf31e89cb76aa475b257a8904f4eR55) model then receives two foreign keys for left and right pleural spaces.
## Reference to official issue
This address #181
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 3 months ago
@vessemer opened a new pull request: #184: CSS adjust margin
> We want some more room to work with
## Reference to official issue
This address #180
## Screenshots:
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 3 months ago
@vessemer opened a new pull request: #183: Fixes ModuleNotFoundErrors
This PR is aimed to fix the `ModuleNotFoundErrors`.
## Description
The `ModuleNotFoundError` occurred 'cause of:
- `prediction`: simply needed `PROJECT_DIR` to be included into the `sys.path`.
- `backend.static`: "remove backend static now that frontend has taken over html", from the [commit](https://github.com/concept-to-clinic/concept-to-clinic/commit/50a47acd1b5946b3bdd222395e86640720549344#diff-ff005b5a072d24689bdfda402ecbb693).
- `src.preprocess.load_dicom`: became redundant and been deleted by this [commit](https://github.com/concept-to-clinic/concept-to-clinic/commit/10c9d3a5ab21b8f63e7ecea460ff6ca8a09262d9#diff-0befd03218e1624daba9fe93b2f54d62).
## Reference to official issue
This addresses #164
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 3 months ago
@vessemer opened a new pull request: #176: Documentation of qfpxfd algorithm
However qfpxfd algorithm has taken the 4th place in Data Science bowl, its repo's [link](http://www.cis.pku.edu.cn/faculty/vision/wangliwei/software.html) in issue #22 is outdated and http://www.cis.pku.edu.cn is not sustainable either thus it is generally hard to find any information about it. So an investigation took place in order to find any details about the algorithm. The description of it were taken from the [Luna16](https://luna16.grand-challenge.org/results/). The authors were emailed in order to get an actual info. The description will be updated if authors reply.
## Description
This is the description of qfpxfd algorithm. This team gained the 4th place on the Data Science Bowl 2017 private leaderboard.
## Reference to official issue
This address #22
## Motivation and Context
>At some point, we need to implement an algorithm that can segment the nodules and detect whether they are malicious or not. It makes sense to first evaluate the top 10 solutions for the Data Science Bowl 2017 regarding how performant they are and how easy it would be to use them in our application.
## CLA
- [ ] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 3 months ago
@vessemer commented on issue #157: Give interface backend a home route
Is there something else left that prevents this issue from getting closed?
2 years, 4 months ago
@vessemer commented on issue #25: Document the DL Munich algorithm
Is there something else left that prevents this issue from getting closed?
2 years, 4 months ago
@vessemer commented on issue #22: Document the qfpxfd algorithm
Seems that [repo-link](www.cis.pku.edu.cn/faculty/vision/wangliwei/software.html) became outdated and quick search doesn't provide the answer for me.
2 years, 4 months ago
@vessemer opened a new pull request: #175: Hide hidden files
Filtering out the hidden files from the Vue frontend.
## Description
The filtering is done on the backend side. Since there is nothing like `os.ishidden` I've decided to not over complicate and wrote the static function `ImageAvailableApiView.is_hidden`(https://github.com/concept-to-clinic/concept-to-clinic/compare/master...vessemer:158_hide_hidden_files?expand=1#diff-500c04290e5206134d43cf7c48c3976bR80).
## Reference to official issue
This addresses the issue #158.
## Screenshots:
<!--- Provide a general summary of your changes in the Title above -->
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 4 months ago
@vessemer commented on PR #174: #157 Listed available routes
@isms, done: all squashed and rebased.
2 years, 4 months ago
@vessemer commented on PR #174: #157 Listed available routes
@isms, I have to agree with you, it seems that it fits the requirements of the issue #157.
@reubano can you take a look, please?
2 years, 4 months ago
@vessemer commented on PR #174: #157 Listed available routes
@isms, I guess, that the point of the issue #157 is more about to get rid of an error 404, while navigating to `http://0.0.0.0:8000/`. List of available routes in this case may serve as a fancy hint for the user.
2 years, 4 months ago
@vessemer opened a new pull request: #174: #157 Listed available routes
This `http://0.0.0.0:8000/` route to a page listing all the available routes.
## Description
`http://0.0.0.0:8000/` redirect to `/api/routes` which in turn will return the dictionary of available links.
## Reference to official issue
This addresses #157.
## Motivation and Context
Currently, navigating to `http://0.0.0.0:8000/` returns a 404 error.
## Screenshots:
The API and JSON response:
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 4 months ago
@vessemer opened a new pull request: #172: #166 Refactored predict cubes
The `predict_cubes` function has been refactored to reduce its complexity.
## Description
The `predict_cubes` is now splitted on the support functions:
-[`prepare_data`](https://github.com/concept-to-clinic/concept-to-clinic/compare/master...vessemer:166_predict_cubes_refactoring?expand=1#diff-632542ba71c59c64b64a4bc113a8c4b6R148), by a given patient ID it returns three np.ndarray.
-[`annotate`](https://github.com/concept-to-clinic/concept-to-clinic/compare/master...vessemer:166_predict_cubes_refactoring?expand=1#diff-632542ba71c59c64b64a4bc113a8c4b6R229), by a given model and a volumetric data, it returns a DataFrame.
-[`stats_from_batch`](https://github.com/concept-to-clinic/concept-to-clinic/compare/master...vessemer:166_predict_cubes_refactoring?expand=1#diff-632542ba71c59c64b64a4bc113a8c4b6R289), for each nodule in a batch it returns a list of DataFrame
## Reference to official issue
This addresses #166
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 4 months ago
@vessemer opened a new pull request: #170: 169 MetaData and PreprocessCT refactored 4 grt123
<!--- Provide a general summary of your changes in the Title above -->
## Description
<!--- Describe your changes in detail -->
## Reference to official issue
<!--- If fixing a bug, there should be an existing issue describing it with steps to reproduce -->
<!--- Please link to the issue here: -->
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If adding a new feature or making improvements not already reflected in an official issue, please reference the relevant sections of the design doc -->
## How Has This Been Tested?
<!--- Please describe in detail how you tested your changes. -->
<!--- Include details of your testing environment, and the tests you ran to -->
<!--- see how your change affects other areas of the code, etc. -->
## Screenshots (if appropriate):
## CLA
- [ ] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 4 months ago
@vessemer opened a new pull request: #146: #143 Segmentation Evaluation
The followed measures and distances have been provided: sensitivity, specificity, [Dice coefficient](https://en.wikipedia.org/wiki/S%C3%B8rensen%E2%80%93Dice_coefficient), the [analogy to Dice coefficient](https://github.com/jocicmarko/ultrasound-nerve-segmentation/blob/master/train.py) and [Hausdorff distance](https://en.wikipedia.org/wiki/Hausdorff_distance). This metrics may be employed for segmentation evaluation of both: lungs and nodules.
## Reference to official issue
This address #143
## How Has This Been Tested?
Unit tests were created for each metric and ran over block-typed input.
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 4 months ago
@vessemer commented on PR #142: Implemented simple nodule segmentation algorithm
@WGierke, I guess, the Hausdorff distance will suits it well. If you are mind to open an issue aimed at segmentation evaluation, I'll be glad to implement this feature.
2 years, 4 months ago
@vessemer opened a new pull request: #141: #21 Documented Aidence algorithm
An initial version of the documentation for the Aidence algorithm. The Aidence team gained the 3rd place on the Data Science Bowl 2017 private leaderboard.
## Reference to official issue
This address #21
## Motivation and Context
>At some point we need to implement an algorithm that can segment the nodules and detect whether they are malicious or not. It makes sense to first evaluate the top 10 solutions for the Data Science Bowl 2017 regarding how performant they are and how easy it would be to use them in our application.
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 4 months ago
@vessemer opened a new pull request: #140: Documented the DL Munich algorithm
An initial version of the documentation for the DL Munich algorithm. The DL Munich team gained the 7th place on the Data Science Bowl 2017 private leaderboard.
## Reference to official issue
This address #25
## Motivation and Context
he method of nodules segmentation is well generalised and may be integrated with the Alex | Andre | Gilberto | Shize segmentation algorithm.
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 4 months ago
@vessemer opened a new pull request: #139: Alex | Andre | Gilberto | Shize algorithm
This is an initial documentation of the Alex | Andre | Gilberto | Shize algorithm. The team takes the 8th place on the DSB17 private LB. The approach outstands by both: the mosaic data augmentation approach and the notable noodles detection sliding 3D architectures.
## Reference to official issue
This pull request addresses #26
## Motivation and Context
This approach allows both nodules detection and segmentation at the same stages, which are into of the current area of interest.
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 4 months ago
@vessemer commented on issue #120: Lung Segmentation
Cool, I've opened a new issue [#133](https://github.com/concept-to-clinic/concept-to-clinic/issues/138) dedicated to anatomical structures segmentation for the later milestones.
2 years, 4 months ago
@vessemer created a new issue: #138: Lungs Segmentation | long term
The initial lungs segmentation method has been provided by the PR [#133](https://github.com/concept-to-clinic/concept-to-clinic/pull/133) due to the issue [#120](https://github.com/concept-to-clinic/concept-to-clinic/issues/120). The provided approach, however, can be further improved e.g., by reducing its trend to false positives and made the algorithm more stable. Furthermore, the ability to correctly separate the tissues such as bones, lungs, bronchial, etc. will be beneficial for a further work with data augmentation.
## Possible Solutions
For example, the work of [van Rikxoort et al.](https://www.ncbi.nlm.nih.gov/pubmed/19673192) describes the automatic error detection method via the convex hull complement to a coastline of lungs:
Furthermore, the method provided by [S. Hu et al.](http://ieeexplore.ieee.org/abstract/document/929615/) is aimed at junction line enhancement followed by lungs separation which I've found to be unreasonable resource consumptive though.
The ability of the bronchial / lungs separation described in the paper of [T. Kitasaka et al.](http://campar.in.tum.de/pub/kitasaka2010pulmo/kitasaka2010pulmo.pdf) I guess will also be valuable as an additional instrument of data augmentation.
> Volume rendered results of the ground truth (left), the proposed method (middle) and the [adaptive branch tracing method](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.164.7783&rep=rep1&type=pdf) (right). Yellow region indicates TP while blue region shows FP Though the bronchial lumen is interrupted by a tumor indicated by the arrow, the proposed method can extract the lumen regions beyond the tumor.
P.S. Of course, contributions like a radiologist's handcrafted annotations of anatomical structures inside CT volume will be appreciated.
## Current Behavior
The lungs segmentation lied in the `preprocess/lung_segmentation.py` consists of the followed steps:
Step 1: Convert into a binary image.
Step 2: Remove the blobs connected to the border of the image.
Step 3: Label the image.
Step 4: Keep the labels with 2 largest areas.
Step 5: Erosion operation with a disk of radius 2. To separate lung nodules attached to blood vessels.
Step 6: Closure operation with a disk of radius 10. To keep nodules attached to a lung wall.
Step 7: Fill in the small holes inside the binary mask of lungs.
## Acceptance criteria
The good measure of a lungs segmentation algorithm should be [Hausdorff distance](https://en.wikipedia.org/wiki/Hausdorff_distance) which has efficient implementation by [`scipy.spatial.distance.directed_hausdorff`](https://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.distance.directed_hausdorff.html#scipy.spatial.distance.directed_hausdorff), thus the acceptance criteria will rely on the Hausdorff tests. Nonetheless, this is the long-term issue, thus different type of contribution may be accepted by this issue.
2 years, 4 months ago
@vessemer commented on PR #122: Converted Team 'grt123' identification and classification algorithms
@dchansen, thank you for your input, but I've taken into account the difference of the default values, I've used the [nn-transfer](https://github.com/gzuidhof/nn-transfer) for the model standardization.
2 years, 4 months ago
@vessemer commented on PR #122: Converted Team 'grt123' identification and classification algorithms
@dchansen I'm right now working on that, also I've tried to convert the grt123 classification model into Keras (TF), but I've faced troubles with the `BatchNorm` layer, despite it transfer weights correctly it feeds me with different results (if I just throw away each `BatchNorm` from the graph then the output becomes just identical, thus I need to dig deeper).
What is related to the way of standardization without transferring the model I've almost done.
2 years, 4 months ago
@vessemer commented on issue #120: Lung Segmentation
My excuses for a delay, was in a bad condition last week.
Thanks for the PR [#133](https://github.com/concept-to-clinic/concept-to-clinic/pull/133), @WGierke!
For sure you're right about the PR [#132](https://github.com/concept-to-clinic/concept-to-clinic/pull/132), I've designed it in order to deal with resource consumptive 3D manipulations. This code doesn't include lungs segmentation in any kind :)
Related to this issue, I appreciate the approach of Julian de Wit. However, for the long term, it will be beneficial to deal with the cons of the convenient Hounsfield scale-based lungs segmentation used by de Wit such as instability or trend to false positives. For example, the work of [van Rikxoort et al.](https://www.ncbi.nlm.nih.gov/pubmed/19673192) describes the automatic error detection method via the convex hull complement to a coastline of lungs:
Furthermore, the method provided by [S. Hu et al.](http://ieeexplore.ieee.org/abstract/document/929615/) is aimed at junction line enhancement followed by lungs separation which I've found to be unreasonable resource consumptive though. The ability of the bronchial / lungs separation described in the paper of [T. Kitasaka et al.](http://campar.in.tum.de/pub/kitasaka2010pulmo/kitasaka2010pulmo.pdf) I guess will also be valuable as an additional instrument of data augmentation.
All of the aforementioned thoughts are obviously out of scope for this issue. Thus, again, thanks for the PR!
2 years, 4 months ago
@vessemer commented on PR #132: Data Generator
@lamby, all rebased and stashed.
2 years, 5 months ago
2 years, 5 months ago
@vessemer opened a new pull request: #132: classification algorithm
The data augmentation process is a very common stage for a plenty of approaches in machine learning, especially in computer vision. The majority of the solutions of the #1, #2, #3 has the similar augmentation pipeline (e.g. [grt123](https://github.com/WGierke/concept-to-clinic/blob/457b7a7528fda28ed556d234135d48595eaa13ce/docs/algorithm_grt123.md), [Therapixel](https://github.com/WGierke/concept-to-clinic/blob/d9a20abfb817642ba6d98e403ed6659446125853/docs/algorithm_therapixel.md), [Daniel Hammack](https://github.com/WGierke/concept-to-clinic/blob/a47d8f28435f612d93a8eee694ac5c82500e710f/docs/algorithm_hammack.md), etc.). The aforementioned pipeline mainly consists of Hounsfield scaling, lungs segmentation (#120), CT re-orientation and a batch of trivial, yet resource consumptive spacial operations: zoom, rotation, shear, shift, flip and combinations of them. The convenient solution for the latter process is to build a generator which will yield a processed patches via affine transformations. The worth notable example of such a generator is [Keras DataGenerator](https://keras.io/preprocessing/image/) complemented by the [thorough description](https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html). The main advantage of the Keras approach is that it for each patch build new transformation matrix from scratch but apply it only ones in comparison with step-by-step application of `scipy.ndimage` perturbations e.g. [code of Daniel Hammack](https://github.com/dhammack/DSB2017/blob/67f6474768fa47f9be95c31aa41609e0a986d8a2/training_code/aws/data_generator_fn3.py#L126-L128) where only for a rotation around the origin two calls of affine transformations were performed. Unfortunately, the main disadvantage for us is that Keras DataGenerator doesn't work with 3D data. Thus what have I done is the extension of the DataGenerator in order to deal with volumetric data such as CT scans. I've also achieved compatibility with `scipy.ndimage` to check integrity of the methods by the trusted functions (i.e. `scipy.ndimage.zoom`, `.rotate`, etc.) which I further have used in the unit tests. Resulted speedup is about 2.5 in comparison with sequantial calls of `scipy.ndimage` functions.
## The examples of the application:
### Random Rotation
### Random Zoom
### Random Shear
### Random Shift
### Agregated
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 5 months ago
@vessemer commented on issue #126: SimpleITK: "Unable to determine ImageIO reader for ..." during local pytest
This bug has been occured because SimpleITK can't read the file (obviously), and my initial thought was that the problem is related to the broken `.mhd` file, but since it runs well on both my computers with the different environments and on Travis as well, then I've become confused. May it be because you didn't rerun dockerd and it tries to preserve its previous paths?
2 years, 5 months ago
@vessemer commented on issue #120: Lung Segmentation
@reubano, I guess you're right, it'll be convenient to place the lung segmentation method inside the `segment` directory. For now, the project has been structured in such a way that all `algorithm`'s subdirectories contain `trained_model.py`. Perhaps, it's worth to reform the `segment` part, so that latter will contain `lungs`, `nodules` or something similar. Since all stages performed prior to the classification may be roughly counted as preprocessing.
2 years, 5 months ago
@vessemer commented on PR #122: Converted Team 'grt123' identification algorithm
It was my concern for the further work, I guess it's worth to define at the moment of mvp which ML backend will be used. As @reubano sed in [#2 (comment)](https://github.com/concept-to-clinic/concept-to-clinic/issues/2#issuecomment-325328384):
> [...] of course, the use of TensorFlow/Theano/etc. will be up to the individual implementation.
But since the grt123 algorithm can be ported on the TenserFlow backend will this approach be desired?
2 years, 5 months ago
@vessemer commented on PR #119: Add function that loads MetaImage files
Hi @tdraebing, thanks for your detailed review and suggestions. I've refactored the code, all rebased and stashed.
> From your tests I read that you added data from the LUNA16-competition. Did you check, whether using them in other projects is fine with their license?
As it was claimed on the official page of [LUNA16/data](https://luna16.grand-challenge.org/data/) the dataset was made from LIDC/IDRI database via filtering out CT scans which slice thickness greater than 2.5 mm and may be other processing. The LIDC/IDRI database, in turn, consist of DICOM files and have [Creative Commons Attribution 3.0 Unported License](https://creativecommons.org/licenses/by/3.0/). Thus it's fine to use the LUNA16 data in a project.
> Fully switching to SimpleITK would reduce the number of dependencies and make the code easier to maintain and follow. There would be also some changes in crop_dicom and preprocess_dicom needed. This might also be handled in a new issue.
I guess it will be a good idea to open a new issue dedicated to that task since it will lead to a quite cumbersome refactoring. @tdraebing, would you like to open it?
2 years, 5 months ago
@vessemer created a new issue: #120: Lung Segmentation
Prior to all of [#1](https://github.com/concept-to-clinic/concept-to-clinic/issues/1), [#2](https://github.com/concept-to-clinic/concept-to-clinic/issues/2), [#3](https://github.com/concept-to-clinic/concept-to-clinic/issues/3) stages, preprocessing must be performed over a CT scan. Among a plenty of trivial preprocessing stages such as clipping by value, rescaling and magnitude normalisation outstands lungs segmentation task. Since the majority of the best solutions contains in there pipelines such stage, it's become important to deal with ambiguity. There exists various approaches dedicated to lungs segmentation e.g. via convenient segmentation by [value in Hounsfield scale](https://www.kaggle.com/gzuidhof/full-preprocessing-tutorial), affine/rigid/non-rigid [image registration](http://simpleelastix.readthedocs.io/GettingStarted.html), segmentation using [watershed algorithm](https://www.kaggle.com/ankasor/improved-lung-segmentation-using-watershed) etc. Approaches listed above have their pros and cons and a tradeoff as usual somewhere in between of resources consuming and a quality of segmentation.
## Expected Behavior
Adopted or created lung segmentation algorithm should lie in the `src/preprocess/lungs_segmentation.py`. It also should be integrated into classes `src.preprocess.preprocess_ct.PreprocessCT` and `src.preprocess.preprocess_ct.Params`.
## Current Behavior
For now `PreprocessDicom` doesn't contain any lung segmentation algorithm.
## Acceptance criteria
- [ ] method aimed at lungs segmentation
- [ ] brief justification of a selected approach
- [ ] tests for the methods
2 years, 5 months ago
@vessemer commented on issue #98: Add function that loads MetaImage files
As far as I saw no changes for more than the week, I've created a pull request.
2 years, 5 months ago
@vessemer opened a new pull request: #119: Add function that loads MetaImage files
The `load_metaimage` function that load MetaImage files have been proposed along with `load_ct` ensembling method which aimed to organize loading process. Refactoring has been performed, taking into account various format of meta-information. In order to handle it standardisation via `class MetaData` has been employed, for now, it contains only one parameter (`spacing`) which is used already. Also decoupling of reading and preprocessing was included.
## Motivation and Context
This addresses [#98](https://github.com/concept-to-clinic/concept-to-clinic/issues/98). The MetaImage file format is actively used by plenty of medical studies. The LUNA16 challenge is among them. Since LUNA16 was involved in kaggle DSB 2017 competition, therefore, it's into of our area of interest.
## How Has This Been Tested?
The unit tests were created for each of introduced functions.
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 5 months ago
@vessemer commented on PR #78: Calculate tumor volume
> Nice observation! Would you like to submit an issue for this use case?
I've created [one](https://github.com/concept-to-clinic/concept-to-clinic/issues/98), but I also guess that it'll be necessary to implement the class aimed at metadata standardisation, maybe along with all reading orchestration.
2 years, 5 months ago
@vessemer created a new issue: #98: Add function that loads MetaImage files
In the field of low-dose CT, not the only DICOM data format is wildly used, but MetaImage (`.mhd` / `.raw`) is also taken place. The MetaImage is the text-based tagged file format for medical images. As well as the DICOM format, the MetaImage has descriptors along with the raw data.
## Expected Behavior
The `.mhd` / `.raw` files may be read via `SimpleITK`. So `load_itk` should be provided which takes the `itk_path` as an input and return the `np.ndarray` (right now the `load_dicom` returns the np.ndarray which may be different in the future since MetaImage and DICOM have different metadata and it will be the good decision to build a class which will standardise the metadata by parameters). Furthermore, the orchestrate function `load_CT` aimed at handling different medical image formats, so the `load_CT` will take `CT_path` as an input and will decide whether to return the output of `load_dicom` or `load_itk`.
## Current Behavior
For now, exists only the possibility to read DICOM file format through `load_dicom`.
## Technical details
This feature should be implemented in the `prediction/src/preprocess` folder
## Acceptance criteria
- [ ] method to load MetaImage (`.mhd` / `.raw`) images into memory
- [ ] method to select apropriate loader function
- [ ] tests for the methods
2 years, 5 months ago
@vessemer commented on issue #2: Feature: Implement classification algorithm
@reubano, I've noted that the major part of algorithms aimed at nodule candidates' classification has similar pipelines for preprocessing steps. In general, these pipelines can be separated into two stages: global- (lungs-) and local- (nodule-) levels' operations. It's important to uncouple these stages, i.e. it is neither unnecessary nor consistently to apply zero-mean normalization at the local stage. Moreover, for the local-stage, it is common to apply augmentation on the fly (even at a prediction time). Thus I've introduced the pre-processing class which operates at the global level, along with the possibility to apply it while reading medical files. Related to the data-generators at the local level, it's also quite common to use trivial spatial augmentation, but then the patch approached to the desired shape. The latter transformations may highly deviate from model to model, hence I guess it will be better if we will wrap the `model` by class with a standardized functionality, just to be able to build 'em once with a desired setup, serialise then and use when ever we want.
2 years, 5 months ago
@vessemer commented on PR #78: Calculate tumor volume
@reubano I've addressed the `to_real_volume` by incorporating it into the `calculate_volume` function. I've also added the function `load_meta` into the `load_dicom.py` file, in order to uncouple ugly `os.path.join(dicom_path, '*.dcm')` from the `calculate_volume`. It's worth noting that there exists not only DICOM files, but also the .mhd/.raw which has been used e.g. in LUNA16. Thus it may be useful to support different formats as well.
2 years, 5 months ago
@vessemer commented on issue #2: Feature: Implement classification algorithm
Hi @reubano, as it was mentioned in the technical details from the [issue #2](https://github.com/concept-to-clinic/concept-to-clinic/issues/2):
> This feature should be implemented in the prediction/classify/trained_model/predict method. Code to train the model should live in the prediction/classify/src/ folder.
> A fully serialized version of the model that can be loaded should live in the prediction/classify/assets/ folder using git-lfs.
The `classify` shouldn't lie in the `prediction/src` directory but in the `prediction` itself. Is there a typo in the issue or is it now outdated?
2 years, 5 months ago
2 years, 5 months ago
@vessemer commented on PR #76: Classification algorithm
Hi, @isms, I've done with small fixes and rebase as well.
2 years, 5 months ago
@vessemer commented on PR #78: Calculate tumor volume
Hi @reubano, as you requested in [#13 (comment)](https://github.com/concept-to-clinic/concept-to-clinic/issues/13#issuecomment-325324091),
> While it would be possible for calculate_volume to take dicom_path as an argument, and then calculate centroids on its own, we prefer to uncouple those steps and have a canonical function dedicated to that task.
I've decoupled the functions so that now the `calculate_volume` returns the volume in voxels, and only `to_real_volume` takes into account the voxels' shape and transform the list of volumes in voxels into the list of volumes in mm.
2 years, 5 months ago
@vessemer commented on issue #13: Calculate tumor volume from pixel masks
@reubano, thanks for making it clear. I guess I've considered all your proposition, by [this commit](https://github.com/concept-to-clinic/concept-to-clinic/pull/78/commits/0cd146b29e8f0a45059745bc5a8a946b008ed3c0).
Is there something else I need take into account?
2 years, 5 months ago
@vessemer commented on PR #78: Calculate tumor volume
Thanks, @WGierke, I've followed your suggestion.
2 years, 5 months ago
@vessemer commented on PR #76: Classification algorithm
Thanks for your feedback @WGierke, I think I changed everything accordingly.
2 years, 5 months ago
@vessemer commented on PR #72: Compute tumor volume from binary mask and list of centroids
I've refreshed the pull request [#78](https://github.com/concept-to-clinic/concept-to-clinic/pull/78) and made it clearer.
2 years, 5 months ago
@vessemer opened a new pull request: #78: Calculate tumor volume
## Description
This method computes the volume of connected components placed at the specified centroids, via connected component analysis using `scipy.ndimage.label`. The `voxel_shape` parameter has added to the function in order to derive units of measure.
## Reference to official issue
Calculate tumor volume from pixel masks [#13](https://github.com/concept-to-clinic/concept-to-clinic/issues/13)
## How Has This Been Tested?
The unit test for `calculate_volume` has been added to test for the volume calculation over the different mask instances. During the testing, another issue has acquired at `test_segment`, which was related to the meaningless `segment_path`. Hence the changes were also added to the `segment.predict` function.
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 5 months ago
@vessemer opened a new pull request: #76: Classification algorithm
## Description
The classification algorithm is a deep residual 3D convolutional neural network architecture aimed at feature extraction and dimensionality reduction. It was my solution for the LUNA16 challenge. The method achieved competition performance metric (CPM) of 0.735 and sensitivities of 78.8% and 83.9% at 1 and 4 false positives per scan, respectively. The main benefits are that this algorithm consists of a single architecture in comparison with cumbersome ensemble methods.
## Reference to official issue
Feature: Implement classification algorithm [#2](https://github.com/concept-to-clinic/concept-to-clinic/issues/2)
## Motivation and Context
Usually, it is required by the classification algorithms to preprocess a raw data, (e.g. to fit the voxels' magnitude into unit interval or to resize the CT to the unit dimension). Thus the additional changes related to the `load_dicom` along with `preprocess_dicom` has been included and tested as well.
## How Has This Been Tested?
Unit tests for `trained_model` include:
- Model loading
- Output consistency
Unit tests for `preprocess_dicom`:
- Parameters integrity
- Preprocessing results for various cases
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 5 months ago
@vessemer commented on PR #72: Compute tumor volume from binary mask and list of centroids
The Travis CI build failed due to LFS issues [#69](https://github.com/concept-to-clinic/concept-to-clinic/issues/69).
2 years, 5 months ago
@vessemer opened a new pull request: #72: Compute tumor volume from binary mask and list of centroids
## Description
This method computes the volume of connected components placed at the specified centroids, via connected component analysis using `scipy.ndimage.label`.
## Reference to official issue
Calculate tumor volume from pixel masks [#13](https://github.com/concept-to-clinic/concept-to-clinic/issues/13)
## How Has This Been Tested?
The unit test for `predict` has been added to test for the volume calculation over the different mask cases.
## CLA
- [x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
2 years, 5 months ago
@vessemer commented on issue #2: Feature: Implement classification algorithm
@isms thanks for the quick response. I mean that prior to any model evaluation some preprocessing should take place over the row DICOM data. I've agreed with your concern regarding cumbersome architecture at the starting point.
> Given a model trained to perform this task
Right now for me it is not clear what is meant by the 'model': whether it is a model object from some ML backend such as TensorFlow or Theano, or it's some essence of higher level with its own preprocessing method.
2 years, 5 months ago
@vessemer commented on issue #2: Feature: Implement classification algorithm
As it was already mentioned: different architectures require different preprocessing methods. Whether it will be a good decision if the input of the feature will include the information about preprocessing function either? I hope it'll be a better to open a new issue dedicated to the preprocessing step. In this way, model may be fed along with a parameters' dictionary. The latter will be used in order to create a preprocessing instance.
2 years, 5 months ago
@vessemer commented on issue #13: Calculate tumor volume from pixel masks
I'm a bit confused: how we are going to calculate the real volume of an area while the function does not receive the CT meta data. Also, can you please clarify how the output from the predict method will be serialised? And last: is it necessary to provide the centroids, since all centroids may be re-estimated via the connected components analysis?