System Info

In transfomers library, the default value of attn_temperature_tuning in Llama4TextConfig, is 4.

However, I think this value should be boolean because it is used as a condition flag in the forward pass.

Moreover, in the official implementation, that value is boolean, which is defaulted to False. (Offical Config File)

Who can help?

@ArthurZucker

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Any Llama4TextConfig instances.

Expected behavior

The default value of config.attn_temperature_tuning should be True or False depending on input sequence length.