Skip to content

Commit

Permalink
update condition slightly
Browse files Browse the repository at this point in the history
  • Loading branch information
robertgshaw2-redhat committed Jul 17, 2024
1 parent 9f9d039 commit 5f2cb45
Showing 1 changed file with 2 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,8 @@ def __init__(self, strategy: str, is_static_input_scheme: bool):
def process_weights_after_loading(self, layer) -> None:
# If per tensor, when we have a fused module (e.g. QKV) with per
# tensor scales (thus N scales being passed to the kernel),
# requantize so we can always run per tensor with torch._scaled_mm
if (self.strategy == QuantizationStrategy.TENSOR
or not self.cutlass_fp8_supported):
# requantize so we can always run per tensor
if self.strategy == QuantizationStrategy.TENSOR:
max_w_scale, weight = requantize_with_max_scale(
weight=layer.weight,
weight_scale=layer.weight_scale,
Expand Down

0 comments on commit 5f2cb45

Please sign in to comment.