Remove cells that don't have any pixel clusters expressed prior to Pixie #1051
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
srivarra
pushed a commit
that referenced
this pull request
Sep 13, 2023
…xie (#1051) * Drop cells without any SOM cluster expressions * Add testing for dropping non-expressed cells
ngreenwald
pushed a commit
that referenced
this pull request
Sep 17, 2023
* added plotting notebook * updated readmed * git merge issues * git merge fixes v2 - docs/landing.md * type * Remove cells that don't have any pixel clusters expressed prior to Pixie (#1051) * Drop cells without any SOM cluster expressions * Add testing for dropping non-expressed cells * Include `img_sub_folder` as parameter in Mantis calls (#1050) * Ensure mantis can read the image sub folder specified * Test on example dataset * Default img_sub_folder to None in create_mantis_dir (standard used elsewhere) * Doc fix and updated example dataset * Update segmentation labels again * Clarify img_sub_folder (mostly for refreshing tests) * Force .github/get_example_dataset.py to use updated revision * Update the file names to look for * fixed clipped plots in the notebooks. * Update HuggingFace version (#1053) * v0.1.10 (#1057) * Added continuous variable segmentation plots * Pin to scikit-image to v0.19.3 (#1060) * pin to 0.19.3 * requirement less than 0.19.3 * Make sure overwrite functionality for pixel clustering propagates into helper functions (#1058) * Propagate overwrite functionality for pixel clustering * Ensure re-normalization prevented on overwrite for pixel SOM re-assignment * normalize_data should be the opposite of overwrite * Single mantis directory (#1061) * single mantis dir, no recopying * post clustering mask suffix * generic clustering adjustments * mask prefix * remove mask prefix * strange merge issue * removed branch filters on CI, so if you branch off of a branch and run GHA, the CI workflow should work * changed ci.yml, filter push on main branch only * python 3.9 doesn't support | for Union type overloading * added fovs parameter for continuous variable plots * updated notebook to include plots * updated cell clustering notebooks to add erosion (default to True), cleaned up some notebook outputs * fixed style imports * re-added test_generate_summary_stats. Was accidently deleted in a refactoring step. * newline at eof fiber_segmentation_test.py * seaborn-paper -> seaborn-v0_8-paper * added cell to view all plotting styles i mask gen nb * pycodestyle --------- Co-authored-by: alex-l-kong <31424707+alex-l-kong@users.noreply.github.com> Co-authored-by: camisowers <38049893+camisowers@users.noreply.github.com>
ngreenwald
pushed a commit
that referenced
this pull request
Sep 18, 2023
* plot_cluster Next Release v0.6.4 (#1012) * 0.6.4 * updated environment.yml * Update README.md * updated windows setup docs * added repo verison in git clone * file fix Deepcell upload loop cleaning (#1023) * loop check for successful deepcell upload * update test zip paths * helper functions pls * zip input, upload, and extract per batch * cleaning * docstring indent * fov -> fovs file name * remove overwrite warning * new dir for each test call * fix bad logic warning testing * remove arg from docstring * add helper function test * batch_num start at 1 * add previously processed warning * set timeouts and scrap retry * correct timeout errors and update tests * timeout 5 mins * add to unprocessed list after loop closes * continue loop after extraction * break instead of continue * skipped processing print fixes * adjust print statements * 3 seconds before second zip call update * updated readthedocs python version * updated plot_utils docstring * docstrings v2 * use GS [1,1] if cbar_visible is False, otherwise use GS [1, 2] for plot + colorbar * added cbar_visible T/F for tests * pycodestyle * added numpy list to cmap / norm function * Remove cells that don't have any pixel clusters expressed prior to Pixie (#1051) * Drop cells without any SOM cluster expressions * Add testing for dropping non-expressed cells * Include `img_sub_folder` as parameter in Mantis calls (#1050) * Ensure mantis can read the image sub folder specified * Test on example dataset * Default img_sub_folder to None in create_mantis_dir (standard used elsewhere) * Doc fix and updated example dataset * Update segmentation labels again * Clarify img_sub_folder (mostly for refreshing tests) * Force .github/get_example_dataset.py to use updated revision * Update the file names to look for * Update HuggingFace version (#1053) * v0.1.10 (#1057) * Pin to scikit-image to v0.19.3 (#1060) * pin to 0.19.3 * requirement less than 0.19.3 * Make sure overwrite functionality for pixel clustering propagates into helper functions (#1058) * Propagate overwrite functionality for pixel clustering * Ensure re-normalization prevented on overwrite for pixel SOM re-assignment * normalize_data should be the opposite of overwrite * Single mantis directory (#1061) * single mantis dir, no recopying * post clustering mask suffix * generic clustering adjustments * mask prefix * remove mask prefix * Mass plotting notebook (#1052) * added plotting notebook * updated readmed * git merge issues * git merge fixes v2 - docs/landing.md * type * Remove cells that don't have any pixel clusters expressed prior to Pixie (#1051) * Drop cells without any SOM cluster expressions * Add testing for dropping non-expressed cells * Include `img_sub_folder` as parameter in Mantis calls (#1050) * Ensure mantis can read the image sub folder specified * Test on example dataset * Default img_sub_folder to None in create_mantis_dir (standard used elsewhere) * Doc fix and updated example dataset * Update segmentation labels again * Clarify img_sub_folder (mostly for refreshing tests) * Force .github/get_example_dataset.py to use updated revision * Update the file names to look for * fixed clipped plots in the notebooks. * Update HuggingFace version (#1053) * v0.1.10 (#1057) * Added continuous variable segmentation plots * Pin to scikit-image to v0.19.3 (#1060) * pin to 0.19.3 * requirement less than 0.19.3 * Make sure overwrite functionality for pixel clustering propagates into helper functions (#1058) * Propagate overwrite functionality for pixel clustering * Ensure re-normalization prevented on overwrite for pixel SOM re-assignment * normalize_data should be the opposite of overwrite * Single mantis directory (#1061) * single mantis dir, no recopying * post clustering mask suffix * generic clustering adjustments * mask prefix * remove mask prefix * strange merge issue * removed branch filters on CI, so if you branch off of a branch and run GHA, the CI workflow should work * changed ci.yml, filter push on main branch only * python 3.9 doesn't support | for Union type overloading * added fovs parameter for continuous variable plots * updated notebook to include plots * updated cell clustering notebooks to add erosion (default to True), cleaned up some notebook outputs * fixed style imports * re-added test_generate_summary_stats. Was accidently deleted in a refactoring step. * newline at eof fiber_segmentation_test.py * seaborn-paper -> seaborn-v0_8-paper * added cell to view all plotting styles i mask gen nb * pycodestyle --------- Co-authored-by: alex-l-kong <31424707+alex-l-kong@users.noreply.github.com> Co-authored-by: camisowers <38049893+camisowers@users.noreply.github.com> * default ero * added erode=True for generate_cluster_mask --------- Co-authored-by: alex-l-kong <31424707+alex-l-kong@users.noreply.github.com> Co-authored-by: camisowers <38049893+camisowers@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is the purpose of this PR?
Closes #1046. In certain cases, cells don't contain any pixel clusters. These should not contribute to the cell SOM training.
How did you implement your changes
In
create_c2pc_data, aftercluster_countsis fully computed for all FOVs, add a step that drops rows that sum to zero across the pixel cluster expression cols. This propagates over tocluster_counts_size_norm(the data that trains the SOM and receives the cluster assignments), since it depends oncluster_counts.