simCLR #1278
simCLR #1278
Conversation
|
Hello @gr33n-made! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:
Comment last updated at 2021-09-09 05:26:47 UTC |
> > Co-authored-by: Nimrais nikola.lukashuk@gmail.com
| def __init__(self, dataset: Dataset, transforms): | ||
| super().__init__() | ||
| self.transforms = transforms | ||
| self._dataset = dataset |
There was a problem hiding this comment.
Do you mean the simCLR wrapper at all?
There was a problem hiding this comment.
no, I mean, that you have 2 references for 1 object: dataset and _dataset
| pos_loss = self.cosineSim(features1, features2).sum(dim=0) / self.tau | ||
| list_neg_loss = [ | ||
| torch.exp(self.cosineSim(features1, torch.roll(features2, i, 1)) / self.tau) | ||
| for i in range(0, bs) |
There was a problem hiding this comment.
If you would follow the simCLR paper you could see that the denominator of the loss contains a positive part. During the training, our aim is to maximize log PosPairSim/(All other similarities). All other similarities include PosSim too.
catalyst/runners/config.py
Outdated
| input_key: str = "features", | ||
| target_key: str = "target", | ||
| loss_key: str = "loss", | ||
| augemention_prefix: str = "aug", |
Co-authored-by: Sergey Kolesnikov <scitator@gmail.com>
Co-authored-by: Sergey Kolesnikov <scitator@gmail.com>
Before submitting (checklist)
catalyst-make-codestyle && catalyst-check-codestyle(pip install -U catalyst-codestyle).make check-docs?pytest .?latestandminimalrequirements?Description
Related Issue
Type of Change
PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
FAQ
Please review the FAQ before submitting an issue: