Skip to content

Trying to support Ruyi-Mini-7B #18

@cellzero

Description

@cellzero

Great work! I can't wait to use TeaCache to accelerate diffusion models.

I'm currently trying to integrate TeaCache with Ruyi-Models. However, I think I might have made a mistake, as I'm not getting a good L1 difference visualization according to the paper. Here are the results I obtained.

Ruyi_L1_Visualization

  • Time Embs is the original embeddings of the timestep.
  • Time With Conditions is the timestep embeddings added with condition embeddings (such as text and image).
  • Time Modulated Inputs is the values after transformer.block[0].norm1.
  • Transformer Inputs is the input of transformer.
  • Transformer Outputs is the output just after all transformer blocks, before final layer.
  • Transformer Final Outputs is the final output of transformer.

I've tried several timestep embeddings and model outputs, but it seems that the timestep embeddings don't have a strong correlation with Ruyi's model output. Could you help me identify which timestep embedding and model output should be used to achieve better correlation? Thank you!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions