Add support for falcon2

[puneeshkhanna]

No description provided.

[puneeshkhanna]

See https://huggingface.co/tiiuae/falcon-11B/blob/main/config.json regarding new parameter "num_ln_in_parallel_attn" and https://github.com/huggingface/transformers/blob/main/src/transformers/models/falcon/modeling_falcon.py changes with respect to it.

[hchings]

Hi @puneeshkhanna, thanks for your contribution!

models/falcon/convert.py and models/falcon/config.py also need some changes as well for your code changes to work.
We've made the changes so no action needed from your end. We'll close this out once things are fully tested.

[MartinMarciniszyn]

@lfr-0531, could you review this pull request please?

[jaedeok-nvidia]

Hi @MartinMarciniszyn, @hchings is currently working on this PR, and I have reviewed it. We discovered that some adjustments are needed. For example, falcon2 uses rotary_base=50042 which is not addressed in this PR. Additionally, there is an unclear issue that we are actively working on resolving.