Add fast path for torch.{mm, addmm, bmm, baddbmm} when the inputs are conjugate transpose #59380

anjali411 · 2021-06-03T15:44:00Z

Stack from ghstack:

Add fast path for torch.{mm, addmm, bmm, baddbmm} when the inputs are conjugate transpose #59380 Add fast path for torch.{mm, addmm, bmm, baddbmm} when the inputs are conjugate transpose

Disabled fastpath for bmm and baddbmm, due to this bug: #64103

30% decrease in peak memory usage when conj_physical is used in the code sample below as well as the backward formulas:

import torch

x=torch.randn(2, 3, dtype=torch.cdouble, device='cuda', requires_grad=True).conj()
y=torch.randn(3, 2, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj()
z=torch.randn(3, 3, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj()

out=torch.addmm(x, y, z)

out.sum().backward()

print(torch.cuda.memory_summary())

Differential Revision: D28898374

fixes #64176

[ghstack-poisoned]

facebook-github-bot · 2021-06-03T15:44:06Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/59380

💊 CI failures summary and remediations

As of commit aea9e83 (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

ghstack-source-id: e33a6f0 Pull Request resolved: #59380

[ghstack-poisoned]

ghstack-source-id: ade5a04 Pull Request resolved: #59380

[ghstack-poisoned]

ghstack-source-id: 1816c2f Pull Request resolved: #59380

[ghstack-poisoned]

ghstack-source-id: 3d23b7a Pull Request resolved: #59380

anjali411 · 2021-06-04T13:27:34Z

@anjali411 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) [ghstack-poisoned]

ghstack-source-id: 2b88040 Pull Request resolved: #59380

Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) [ghstack-poisoned]

ghstack-source-id: 96e6e68 Pull Request resolved: #59380

Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) [ghstack-poisoned]

ghstack-source-id: 982fb51 Pull Request resolved: #59380

Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) [ghstack-poisoned]

ghstack-source-id: 854b7a3 Pull Request resolved: #59380

peterbell10 · 2021-07-09T00:04:26Z

aten/src/ATen/native/LinearAlgebra.cpp

@@ -1031,13 +1031,22 @@ static void addmm_impl_cpu_(
  const int64_t ldb = b.strides()[(transpose_b == transpose_c) ? 1 : 0];
  const int64_t ldc = c.strides()[transpose_c ? 0 : 1];

+  if (a.is_conj() && !transpose_a) {
+    a.conj_physical_();


This potentially modifies an input tensor's values.

Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) [ghstack-poisoned]

ghstack-source-id: c6f0af7 Pull Request resolved: #59380

Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) [ghstack-poisoned]

… inputs are conjugate transpose" Benchmarks for addmm: #59380 (comment) Disabled fastpath for bmm and baddbmm, due to this bug: #64103 30% decrease in peak memory usage when `conj_physical` is used in the code sample below as well as the backward formulas: ``` import torch x=torch.randn(2, 3, dtype=torch.cdouble, device='cuda', requires_grad=True).conj() y=torch.randn(3, 2, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() z=torch.randn(3, 3, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() out=torch.addmm(x, y, z) out.sum().backward() print(torch.cuda.memory_summary()) ``` Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) [ghstack-poisoned]

…addbmm} when the inputs are conjugate transpose" Benchmarks for addmm: #59380 (comment) Disabled fastpath for bmm and baddbmm, due to this bug: #64103 30% decrease in peak memory usage when `conj_physical` is used in the code sample below as well as the backward formulas: ``` import torch x=torch.randn(2, 3, dtype=torch.cdouble, device='cuda', requires_grad=True).conj() y=torch.randn(3, 2, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() z=torch.randn(3, 3, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() out=torch.addmm(x, y, z) out.sum().backward() print(torch.cuda.memory_summary()) ``` Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) [ghstack-poisoned]

… inputs are conjugate transpose" Benchmarks for addmm: #59380 (comment) Disabled fastpath for bmm and baddbmm, due to this bug: #64103 30% decrease in peak memory usage when `conj_physical` is used in the code sample below as well as the backward formulas: ``` import torch x=torch.randn(2, 3, dtype=torch.cdouble, device='cuda', requires_grad=True).conj() y=torch.randn(3, 2, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() z=torch.randn(3, 3, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() out=torch.addmm(x, y, z) out.sum().backward() print(torch.cuda.memory_summary()) ``` Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) [ghstack-poisoned]

…addbmm} when the inputs are conjugate transpose" Benchmarks for addmm: #59380 (comment) Disabled fastpath for bmm and baddbmm, due to this bug: #64103 30% decrease in peak memory usage when `conj_physical` is used in the code sample below as well as the backward formulas: ``` import torch x=torch.randn(2, 3, dtype=torch.cdouble, device='cuda', requires_grad=True).conj() y=torch.randn(3, 2, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() z=torch.randn(3, 3, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() out=torch.addmm(x, y, z) out.sum().backward() print(torch.cuda.memory_summary()) ``` Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) [ghstack-poisoned]

… inputs are conjugate transpose" Benchmarks for addmm: #59380 (comment) Disabled fastpath for bmm and baddbmm, due to this bug: #64103 30% decrease in peak memory usage when `conj_physical` is used in the code sample below as well as the backward formulas: ``` import torch x=torch.randn(2, 3, dtype=torch.cdouble, device='cuda', requires_grad=True).conj() y=torch.randn(3, 2, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() z=torch.randn(3, 3, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() out=torch.addmm(x, y, z) out.sum().backward() print(torch.cuda.memory_summary()) ``` Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) [ghstack-poisoned]

…addbmm} when the inputs are conjugate transpose" Benchmarks for addmm: #59380 (comment) Disabled fastpath for bmm and baddbmm, due to this bug: #64103 30% decrease in peak memory usage when `conj_physical` is used in the code sample below as well as the backward formulas: ``` import torch x=torch.randn(2, 3, dtype=torch.cdouble, device='cuda', requires_grad=True).conj() y=torch.randn(3, 2, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() z=torch.randn(3, 3, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() out=torch.addmm(x, y, z) out.sum().backward() print(torch.cuda.memory_summary()) ``` Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) [ghstack-poisoned]

ghstack-source-id: 96627d3 Pull Request resolved: #59380

anjali411 · 2021-08-30T19:29:43Z

aten/src/ATen/native/TensorFactories.cpp

@@ -1411,17 +1411,18 @@ Tensor from_file(c10::string_view filename, c10::optional<bool> shared, c10::opt
 Tensor clone(const Tensor& src, c10::optional<c10::MemoryFormat> optional_memory_format) {
  auto memory_format =
      optional_memory_format.value_or(MemoryFormat::Preserve);
+  Tensor self;


fixes #64176

anjali411 · 2021-08-30T19:33:35Z

@anjali411 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

ngimel · 2021-08-30T23:06:25Z

test/test_linalg.py

+        def _slice(tensor, fn):
+            return fn(tensor)[:, ::2]


Suggested change

def _slice(tensor, fn):

return fn(tensor)[:, ::2]

def _slice(tensor, fn):

return fn(tensor)[..., ::2]

this can be used for batched tensor also

ngimel · 2021-08-30T23:08:06Z

test/test_linalg.py

+        Ab = torch.randn(2, 3, 6, dtype=torch.cfloat, device=device)
+        Bb = torch.randn(2, 3, 3, dtype=torch.cfloat, device=device)
+        Bb_ = torch.randn(1, 3, 3, dtype=torch.cfloat, device=device).expand(2, 3, 3)
+        out_b = torch.empty(2, 3, 3, device=device, dtype=torch.complex64).transpose(-1, 1)


nit: transpose(-1, -2), or transpose(1,2), this mixing of negative and positive indices looks confusing (same in t_b function)

… inputs are conjugate transpose" Benchmarks for addmm: #59380 (comment) Disabled fastpath for bmm and baddbmm, due to this bug: #64103 30% decrease in peak memory usage when `conj_physical` is used in the code sample below as well as the backward formulas: ``` import torch x=torch.randn(2, 3, dtype=torch.cdouble, device='cuda', requires_grad=True).conj() y=torch.randn(3, 2, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() z=torch.randn(3, 3, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() out=torch.addmm(x, y, z) out.sum().backward() print(torch.cuda.memory_summary()) ``` Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) fixes #64176 [ghstack-poisoned]

…addbmm} when the inputs are conjugate transpose" Benchmarks for addmm: #59380 (comment) Disabled fastpath for bmm and baddbmm, due to this bug: #64103 30% decrease in peak memory usage when `conj_physical` is used in the code sample below as well as the backward formulas: ``` import torch x=torch.randn(2, 3, dtype=torch.cdouble, device='cuda', requires_grad=True).conj() y=torch.randn(3, 2, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() z=torch.randn(3, 3, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() out=torch.addmm(x, y, z) out.sum().backward() print(torch.cuda.memory_summary()) ``` Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) fixes #64176 [ghstack-poisoned]

… inputs are conjugate transpose" Benchmarks for addmm: #59380 (comment) Disabled fastpath for bmm and baddbmm, due to this bug: #64103 30% decrease in peak memory usage when `conj_physical` is used in the code sample below as well as the backward formulas: ``` import torch x=torch.randn(2, 3, dtype=torch.cdouble, device='cuda', requires_grad=True).conj() y=torch.randn(3, 2, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() z=torch.randn(3, 3, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() out=torch.addmm(x, y, z) out.sum().backward() print(torch.cuda.memory_summary()) ``` Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) fixes #64176 [ghstack-poisoned]

…addbmm} when the inputs are conjugate transpose" Benchmarks for addmm: #59380 (comment) Disabled fastpath for bmm and baddbmm, due to this bug: #64103 30% decrease in peak memory usage when `conj_physical` is used in the code sample below as well as the backward formulas: ``` import torch x=torch.randn(2, 3, dtype=torch.cdouble, device='cuda', requires_grad=True).conj() y=torch.randn(3, 2, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() z=torch.randn(3, 3, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() out=torch.addmm(x, y, z) out.sum().backward() print(torch.cuda.memory_summary()) ``` Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) fixes #64176 [ghstack-poisoned]

… inputs are conjugate transpose" Benchmarks for addmm: #59380 (comment) Disabled fastpath for bmm and baddbmm, due to this bug: #64103 30% decrease in peak memory usage when `conj_physical` is used in the code sample below as well as the backward formulas: ``` import torch x=torch.randn(2, 3, dtype=torch.cdouble, device='cuda', requires_grad=True).conj() y=torch.randn(3, 2, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() z=torch.randn(3, 3, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() out=torch.addmm(x, y, z) out.sum().backward() print(torch.cuda.memory_summary()) ``` Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) fixes #64176 [ghstack-poisoned]

…addbmm} when the inputs are conjugate transpose" Benchmarks for addmm: #59380 (comment) Disabled fastpath for bmm and baddbmm, due to this bug: #64103 30% decrease in peak memory usage when `conj_physical` is used in the code sample below as well as the backward formulas: ``` import torch x=torch.randn(2, 3, dtype=torch.cdouble, device='cuda', requires_grad=True).conj() y=torch.randn(3, 2, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() z=torch.randn(3, 3, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() out=torch.addmm(x, y, z) out.sum().backward() print(torch.cuda.memory_summary()) ``` Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) fixes #64176 [ghstack-poisoned]

ghstack-source-id: 8b83796 Pull Request resolved: #59380

anjali411 · 2021-08-31T18:17:21Z

@anjali411 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

… inputs are conjugate transpose" Benchmarks for addmm: #59380 (comment) Disabled fastpath for bmm and baddbmm, due to this bug: #64103 30% decrease in peak memory usage when `conj_physical` is used in the code sample below as well as the backward formulas: ``` import torch x=torch.randn(2, 3, dtype=torch.cdouble, device='cuda', requires_grad=True).conj() y=torch.randn(3, 2, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() z=torch.randn(3, 3, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() out=torch.addmm(x, y, z) out.sum().backward() print(torch.cuda.memory_summary()) ``` Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) fixes #64176 [ghstack-poisoned]

ghstack-source-id: 39d1c5f Pull Request resolved: #59380

…addbmm} when the inputs are conjugate transpose" Benchmarks for addmm: #59380 (comment) Disabled fastpath for bmm and baddbmm, due to this bug: #64103 30% decrease in peak memory usage when `conj_physical` is used in the code sample below as well as the backward formulas: ``` import torch x=torch.randn(2, 3, dtype=torch.cdouble, device='cuda', requires_grad=True).conj() y=torch.randn(3, 2, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() z=torch.randn(3, 3, dtype=torch.cdouble, device='cuda',requires_grad=True).transpose(0, 1).conj() out=torch.addmm(x, y, z) out.sum().backward() print(torch.cuda.memory_summary()) ``` Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) fixes #64176 [ghstack-poisoned]

anjali411 · 2021-09-01T14:09:09Z

@anjali411 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Add fast path for addmm when the inputs are conjugate

68025ca

[ghstack-poisoned]

This was referenced Jun 3, 2021

Conjugate View #54987

Closed

Add neg bit #56058

Closed

facebook-github-bot added the cla signed label Jun 3, 2021

anjali411 added a commit that referenced this pull request Jun 3, 2021

Add fast path for addmm when the inputs are conjugate

6c03481

ghstack-source-id: e33a6f0 Pull Request resolved: #59380

Update on "Add fast path for addmm when the inputs are conjugate"

2cecee9

[ghstack-poisoned]

anjali411 added a commit that referenced this pull request Jun 3, 2021

Add fast path for addmm when the inputs are conjugate

e981711

ghstack-source-id: ade5a04 Pull Request resolved: #59380

Update on "Add fast path for addmm when the inputs are conjugate"

c8f025b

[ghstack-poisoned]

anjali411 added a commit that referenced this pull request Jun 3, 2021

Add fast path for addmm when the inputs are conjugate

e1835ed

ghstack-source-id: 1816c2f Pull Request resolved: #59380

Update on "Add fast path for addmm when the inputs are conjugate"

a090e27

[ghstack-poisoned]

anjali411 added a commit that referenced this pull request Jun 4, 2021

Add fast path for addmm when the inputs are conjugate

780a3b8

ghstack-source-id: 3d23b7a Pull Request resolved: #59380

Update on "Add fast path for addmm when the inputs are conjugate"

4baaa01

Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) [ghstack-poisoned]

anjali411 added a commit that referenced this pull request Jun 4, 2021

Add fast path for addmm when the inputs are conjugate

0bab014

ghstack-source-id: 2b88040 Pull Request resolved: #59380

Update on "Add fast path for addmm when the inputs are conjugate"

fc306fa

Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) [ghstack-poisoned]

anjali411 added a commit that referenced this pull request Jun 7, 2021

Add fast path for addmm when the inputs are conjugate

b373fd2

ghstack-source-id: 96e6e68 Pull Request resolved: #59380

Update on "Add fast path for addmm when the inputs are conjugate"

98e5bfb

Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) [ghstack-poisoned]

anjali411 added a commit that referenced this pull request Jun 23, 2021

Add fast path for addmm when the inputs are conjugate

c79fe6f

ghstack-source-id: 982fb51 Pull Request resolved: #59380

Update on "Add fast path for addmm when the inputs are conjugate"

66a8ef2

Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) [ghstack-poisoned]

anjali411 added a commit that referenced this pull request Jul 8, 2021

Add fast path for addmm when the inputs are conjugate

1620e0f

ghstack-source-id: 854b7a3 Pull Request resolved: #59380

peterbell10 reviewed Jul 9, 2021

View reviewed changes

Update on "Add fast path for addmm when the inputs are conjugate"

e472327

Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) [ghstack-poisoned]

Update on "Add fast path for addmm when the inputs are conjugate"

2d2b635

Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) [ghstack-poisoned]

anjali411 added a commit that referenced this pull request Jul 9, 2021

Add fast path for addmm when the inputs are conjugate

6f9d882

ghstack-source-id: c6f0af7 Pull Request resolved: #59380

Update on "Add fast path for addmm when the inputs are conjugate"

1fc6482

Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) [ghstack-poisoned]

Update on "Add fast path for addmm when the inputs are conjugate"

ff6a8f5

Differential Revision: [D28898374](https://our.internmc.facebook.com/intern/diff/D28898374) [ghstack-poisoned]

anjali411 added a commit that referenced this pull request Aug 30, 2021

Add fast path for addmm when the inputs are conjugate

3eb2476

ghstack-source-id: 96627d3 Pull Request resolved: #59380

anjali411 commented Aug 30, 2021

View reviewed changes

ngimel reviewed Aug 30, 2021

View reviewed changes

anjali411 added a commit that referenced this pull request Aug 31, 2021

Add fast path for addmm when the inputs are conjugate

bc7c69e

ghstack-source-id: 8b83796 Pull Request resolved: #59380

anjali411 added a commit that referenced this pull request Sep 1, 2021

Add fast path for addmm when the inputs are conjugate

d6e5332

ghstack-source-id: 39d1c5f Pull Request resolved: #59380

facebook-github-bot closed this in 5d80a48 Sep 1, 2021

facebook-github-bot deleted the gh/anjali411/127/head branch October 2, 2021 14:27

mberr mentioned this pull request Oct 23, 2021

Use torch.isin when torch 1.10 is released pykeen/pykeen#487

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add fast path for torch.{mm, addmm, bmm, baddbmm} when the inputs are conjugate transpose #59380

Add fast path for torch.{mm, addmm, bmm, baddbmm} when the inputs are conjugate transpose #59380

Uh oh!

anjali411 commented Jun 3, 2021 •

edited

Loading

Uh oh!

facebook-github-bot commented Jun 3, 2021 •

edited

Loading

Uh oh!

anjali411 commented Jun 4, 2021

Uh oh!

peterbell10 Jul 9, 2021

Uh oh!

anjali411 Aug 30, 2021

Uh oh!

anjali411 commented Aug 30, 2021

Uh oh!

ngimel Aug 30, 2021

Uh oh!

ngimel Aug 30, 2021

Uh oh!

anjali411 commented Aug 31, 2021

Uh oh!

anjali411 commented Sep 1, 2021

Uh oh!

Uh oh!

Add fast path for torch.{mm, addmm, bmm, baddbmm} when the inputs are conjugate transpose #59380

Add fast path for torch.{mm, addmm, bmm, baddbmm} when the inputs are conjugate transpose #59380

Uh oh!

Conversation

anjali411 commented Jun 3, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Jun 3, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful links

💊 CI failures summary and remediations

Uh oh!

anjali411 commented Jun 4, 2021

Uh oh!

peterbell10 Jul 9, 2021

Choose a reason for hiding this comment

Uh oh!

anjali411 Aug 30, 2021

Choose a reason for hiding this comment

Uh oh!

anjali411 commented Aug 30, 2021

Uh oh!

ngimel Aug 30, 2021

Choose a reason for hiding this comment

Uh oh!

ngimel Aug 30, 2021

Choose a reason for hiding this comment

Uh oh!

anjali411 commented Aug 31, 2021

Uh oh!

anjali411 commented Sep 1, 2021

Uh oh!

Uh oh!

anjali411 commented Jun 3, 2021 •

edited

Loading

facebook-github-bot commented Jun 3, 2021 •

edited

Loading