Skip to content

FusedBiasAct Native Pytorch Implementation #70

@mandelapatrick

Description

@mandelapatrick

Hi! Great job on this repo - well done! I am trying to crate a native Pytorch implementation of the C++ function fused.fused_bias_act and I was wonder if you can help me. My attempt is below.
Mostly I am not clear what bias, refer, act, grad and scale refer to in the C++ implementation.

def fused_bias_act(inp, bias, refer, act, grad, negative_slope, scale):
noise = bias + refer
fused_act = torch.nn.functional.leaky_relu(inp + noise, negative_slope=negative_slope, inplace=False) * scale
return fused_act

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions