Skip to content

Upgrade to PyTorch 1.7 #2757

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Feb 1, 2021
Merged

Upgrade to PyTorch 1.7 #2757

merged 4 commits into from
Feb 1, 2021

Conversation

fritzo
Copy link
Member

@fritzo fritzo commented Jan 29, 2021 •
edited
Loading

Addresses #2754

  • Require torch>=1.7.0
  • Use torch==1.7.0 on travis and docs
  • Remove workarounds for torch==1.6
  • Mark one baseball jit test xfail

@fritzo fritzo added the WIP label Jan 29, 2021
@fritzo fritzo mentioned this pull request Jan 29, 2021
Closed
5 tasks
@fritzo
Copy link
Member Author

fritzo commented Jan 29, 2021 •
edited
Loading

@neerajprad @fehiepsi do you have any ideas how to fix or work around the newly failing baseball example? I can't reproduce locally (either mac or linux), but the error on travis is "RuntimeError: Cannot insert a Tensor that requires grad as a constant."

tests/test_examples.py::test_jit[baseball.py --num-samples=200 --warmup-steps=100 --jit] I 	 Running:
python examples/baseball.py --num-samples=200 --warmup-steps=100 --jit
Original Dataset:
   FirstName    LastName  At-Bats  ...  SeasonAt-Bats  SeasonHits  SeasonAverage
0    Roberto    Clemente       45  ...            412         145          0.352
1      Frank    Robinson       45  ...            471         144          0.306
2      Frank      Howard       45  ...            566         160          0.283
3        Jay   Johnstone       45  ...            320          76          0.238
4        Ken       Berry       45  ...            463         128          0.276
5        Jim     Spencer       45  ...            511         140          0.274
6        Don   Kessinger       45  ...            631         168          0.266
7       Luis    Alvarado       45  ...            183          41          0.224
8        Ron       Santo       45  ...            555         148          0.267
9        Ron     Swaboda       45  ...            245          57          0.233
10      Rico  Petrocelli       45  ...            583         152          0.261
11     Ellie   Rodriguez       45  ...            231          52          0.225
12    George       Scott       45  ...            480         142          0.296
13       Del       Unser       45  ...            322          83          0.258
14     Billy    Williams       45  ...            636         205          0.251
15      Bert  Campaneris       45  ...            603         168          0.279
16   Thurman      Munson       45  ...            453         137          0.302
17       Max       Alvis       45  ...            115          21          0.183

[18 rows x 10 columns]
/home/travis/build/pyro-ppl/pyro/pyro/infer/mcmc/api.py:335: UserWarning: num_chains=4 is more than available_cpu=1. Chains will be drawn sequentially.
  .format(num_chains, available_cpu))

Model: Fully Pooled
===================

phi:
   mean   std  median  25.0%  75.0%   n_eff  r_hat
0  0.27  0.02    0.27   0.26   0.28  312.47    1.0

Number of divergent transitions: 0


Posterior Predictive:
Hit Rate - Initial 45 At Bats
-----------------------------
                   mean   std  median  25.0%  75.0%  ActualHits
Roberto Clemente  12.08  2.98    12.0   11.0   14.0          18
Frank Robinson    11.98  3.01    12.0   10.0   13.0          17
Frank Howard      11.88  3.08    12.0   10.0   13.0          16
Jay Johnstone     11.83  3.03    12.0    8.0   12.0          15
Ken Berry         12.06  3.06    12.0    8.0   12.0          14
Jim Spencer       12.07  3.02    12.0   10.0   13.0          14
Don Kessinger     11.97  3.12    12.0    8.0   12.0          13
Luis Alvarado     11.95  3.01    12.0   10.0   13.0          12
Ron Santo         11.85  2.87    12.0   10.0   13.0          11
Ron Swaboda       11.81  3.06    12.0    8.0   12.0          11
Rico Petrocelli   11.98  3.09    12.0    9.0   13.0          10
Ellie Rodriguez   11.85  3.18    12.0    9.0   13.0          10
George Scott      12.01  2.94    12.0   10.0   13.0          10
Del Unser         11.95  3.09    12.0    8.0   12.0          10
Billy Williams    12.03  3.06    12.0   10.0   13.0          10
Bert Campaneris   11.84  3.16    12.0    9.0   13.0           9
Thurman Munson    11.90  3.13    12.0    9.0   13.0           8
Max Alvis         12.06  3.09    12.0   10.0   13.0           7

Hit Rate - Season Predictions
-----------------------------
                    mean    std  median  25.0%  75.0%  ActualHits
Roberto Clemente  109.92  10.92   110.0  101.0  114.0         145
Frank Robinson    125.13  12.35   125.0  114.0  130.0         144
Frank Howard      150.77  14.12   150.0  137.0  156.0         160
Jay Johnstone      85.08   9.30    85.0   79.0   91.0          76
Ken Berry         123.17  12.44   123.0  116.0  132.0         128
Jim Spencer       135.20  12.94   135.0  125.0  142.0         140
Don Kessinger     167.76  15.26   167.0  154.0  174.0         168
Luis Alvarado      48.70   6.44    48.0   44.0   52.0          41
Ron Santo         147.30  13.80   147.0  139.0  156.0         148
Ron Swaboda        65.09   7.66    65.0   58.0   68.0          57
Rico Petrocelli   155.53  14.22   155.0  141.0  159.0         152
Ellie Rodriguez    61.17   7.80    61.0   55.0   65.0          52
George Scott      127.01  11.99   127.0  117.0  133.0         142
Del Unser          85.71   9.39    85.0   78.0   90.0          83
Billy Williams    168.49  14.38   168.0  158.0  177.0         205
Bert Campaneris   160.37  14.15   159.0  149.0  168.0         168
Thurman Munson    120.33  11.82   120.0  111.0  126.0         137
Max Alvis          30.61   5.25    30.0   24.0   31.0          21

Log pointwise predictive density
--------------------------------
-74.9607

/home/travis/build/pyro-ppl/pyro/pyro/infer/mcmc/api.py:335: UserWarning: num_chains=4 is more than available_cpu=1. Chains will be drawn sequentially.
  .format(num_chains, available_cpu))
Traceback (most recent call last):
  File "/home/travis/build/pyro-ppl/pyro/examples/baseball.py", line 359, in <module>
    main(args)
  File "/home/travis/build/pyro-ppl/pyro/examples/baseball.py", line 271, in main
    mcmc.run(at_bats, hits)
  File "/home/travis/build/pyro-ppl/pyro/pyro/poutine/messenger.py", line 12, in _context_wrap
    return fn(*args, **kwargs)
  File "/home/travis/build/pyro-ppl/pyro/pyro/infer/mcmc/api.py", line 382, in run
    for x, chain_id in self.sampler.run(*args, **kwargs):
  File "/home/travis/build/pyro-ppl/pyro/pyro/infer/mcmc/api.py", line 166, in run
    *args, **kwargs):
  File "/home/travis/build/pyro-ppl/pyro/pyro/infer/mcmc/api.py", line 108, in _gen_samples
    kernel.setup(warmup_steps, *args, **kwargs)
  File "/home/travis/build/pyro-ppl/pyro/pyro/infer/mcmc/hmc.py", line 306, in setup
    z_grads, potential_energy = potential_grad(self.potential_fn, z)
  File "/home/travis/build/pyro-ppl/pyro/pyro/ops/integrator.py", line 82, in potential_grad
    raise e
  File "/home/travis/build/pyro-ppl/pyro/pyro/ops/integrator.py", line 75, in potential_grad
    potential_energy = potential_fn(z)
  File "/home/travis/build/pyro-ppl/pyro/pyro/infer/mcmc/util.py", line 289, in _potential_fn_jit
    self._compiled_fn = torch.jit.trace(_pe_jit, vals, **jit_options)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/torch/jit/_trace.py", line 779, in trace
    name, func, example_inputs, var_lookup_fn, strict, _force_outplace
  File "/opt/python/3.6.7/lib/python3.6/contextlib.py", line 52, in inner
    return func(*args, **kwds)
  File "/home/travis/build/pyro-ppl/pyro/pyro/infer/mcmc/util.py", line 285, in _pe_jit
    return self._potential_fn(params)
  File "/home/travis/build/pyro-ppl/pyro/pyro/infer/mcmc/util.py", line 262, in _potential_fn
    log_joint = self.trace_prob_evaluator.log_prob(model_trace)
  File "/home/travis/build/pyro-ppl/pyro/pyro/infer/mcmc/util.py", line 223, in log_prob
    return model_trace.log_prob_sum()
  File "/home/travis/build/pyro-ppl/pyro/pyro/poutine/trace_struct.py", line 191, in log_prob_sum
    log_p = site["fn"].log_prob(site["value"], *site["args"], **site["kwargs"])
  File "/home/travis/build/pyro-ppl/pyro/pyro/distributions/torch.py", line 89, in log_prob
    + log_binomial(n, k, tol=self.approx_log_prob_tol))
RuntimeError: Cannot insert a Tensor that requires grad as a constant. Consider making it a parameter or input, or detaching the gradient
Tensor:
 18
 17
 16
 15
 14
 14
 13
 12
 11
 11
 10
 10
 10
 10
 10
  9
  8
  7
[ torch.FloatTensor{18} ]
FAILED

EDIT I'll just xfail on travis.

@fehiepsi
Copy link
Member

This is related to our patch for Binomial. I'll take a look.

@fritzo fritzo requested a review from neerajprad January 30, 2021 14:46
neerajprad
neerajprad approved these changes Feb 1, 2021
View reviewed changes
Copy link
Member

@neerajprad neerajprad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. We can fix the baseball example in a separate PR.

@neerajprad neerajprad merged commit 2180f6c into dev Feb 1, 2021
@fritzo fritzo mentioned this pull request Feb 2, 2021
Merged
1 task
@fritzo fritzo deleted the torch-1.7 branch September 27, 2021 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@neerajprad neerajprad neerajprad approved these changes

Assignees
No one assigned
Labels
awaiting review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@fritzo @fehiepsi @neerajprad