Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove deprecated lazy tensor #2615

Merged
merged 1 commit into from
Dec 12, 2024

Conversation

saitcakmak
Copy link
Collaborator

The lazy tensor functionality has been deprecated (and split off as linear operator) for over two years. Since then, the old lazy tensor classes have been raising a deprecation warning and pointing to the corresponding linear operator classes via the deprecated_lazy_tensor helper class.

While profiling candidate generation in Ax / BoTorch, I've noticed 5-15% of the runtime being spent in deprecated_lazy_tensor.__init__. This class is being instantiated hundreds of thousands of times during each candidate generation, adding a significant overhead without a clear benefit. I think it's time to remove it and claw back that overhead.

Here's a sample of an example cProfile output:

         68404451 function calls (65915300 primitive calls) in 150.665 seconds
   Ordered by: internal time
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     1876   42.210    0.023   42.210    0.023 {method 'run_backward' of 'torch._C._EngineBase' objects}
   725407    8.305    0.000   15.168    0.000 functional.py:79(broadcast_shapes)
19530046/18762961    4.030    0.000    5.435    0.000 {built-in method builtins.isinstance}
    46998    3.955    0.000    3.955    0.000 {method 'matmul' of 'torch._C.TensorBase' objects}
  3914524    2.738    0.000    3.857    0.000 symbolic_shapes.py:393(guard_size_oblivious)
   338423    2.047    0.000   14.049    0.000 module.py:1932(__setattr__)
   368383    1.844    0.000    9.709    0.000 _linear_operator.py:155(__init__)
    90242    1.811    0.000    4.984    0.000 inspect.py:2268(_signature_from_function)
    73322    1.732    0.000    1.732    0.000 {built-in method torch.cat}
    22560    1.610    0.000    4.713    0.000 exact_prediction_strategies.py:365(exact_predictive_covar)
    20658    1.548    0.000    4.651    0.000 safe_math.py:422(fatmoid)
**693531/298896    1.546    0.000   22.865    0.000 lazy_tensor.py:32(__init__)**

You can see that deprecated_lazy_tensor.__init__ was called 693531 times, resulting in a 22.865 second overhead in a total runtime of 150.665 seconds.

Copy link
Collaborator

@Balandat Balandat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@saitcakmak saitcakmak merged commit c62c324 into cornellius-gp:main Dec 12, 2024
7 checks passed
saitcakmak added a commit to saitcakmak/aepsych that referenced this pull request Dec 12, 2024
Summary:
Commit hash: c62c324aff0732ab0a1b6f37119e27f4612e33a7

Generated using `./pytorch/fb_build/import_gpytorch.sh`

This includes removal of deprecated lazy tensor (cornellius-gp/gpytorch#2615), which eliminates a significant overhead.

Reviewed By: Balandat

Differential Revision: D67141046
saitcakmak added a commit to saitcakmak/aepsych that referenced this pull request Dec 12, 2024
Summary:

Commit hash: c62c324aff0732ab0a1b6f37119e27f4612e33a7

Generated using `./pytorch/fb_build/import_gpytorch.sh`

This includes removal of deprecated lazy tensor (cornellius-gp/gpytorch#2615), which eliminates a significant overhead.

Reviewed By: Balandat

Differential Revision: D67141046
facebook-github-bot pushed a commit to facebookresearch/aepsych that referenced this pull request Dec 13, 2024
Summary:
Commit hash: c62c324aff0732ab0a1b6f37119e27f4612e33a7

Generated using `./pytorch/fb_build/import_gpytorch.sh`

This includes removal of deprecated lazy tensor (cornellius-gp/gpytorch#2615), which eliminates a significant overhead.

Reviewed By: Balandat

Differential Revision: D67141046

fbshipit-source-id: 34ea7a8c694a51d02e93512c2e5c08e532f0805b
JasonKChow pushed a commit to facebookresearch/aepsych that referenced this pull request Dec 18, 2024
Summary:
Commit hash: c62c324aff0732ab0a1b6f37119e27f4612e33a7

Generated using `./pytorch/fb_build/import_gpytorch.sh`

This includes removal of deprecated lazy tensor (cornellius-gp/gpytorch#2615), which eliminates a significant overhead.

Reviewed By: Balandat

Differential Revision: D67141046

fbshipit-source-id: 34ea7a8c694a51d02e93512c2e5c08e532f0805b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants