Skip to content

Fix inaccurate description of RoPE paper findings #2962

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 22, 2025

Conversation

johnhaofu
Copy link
Contributor

@johnhaofu johnhaofu commented Jul 14, 2025

  • Corrected the description to clarify that the paper shows removing (not rotating) lowest frequencies improves performance
  • The original paper (arXiv:2410.06205) proposes p-RoPE which removes lowest frequencies, not rotates them
  • This addresses the misleading summary that suggested rotation of low frequencies was beneficial

- Corrected the description to clarify that the paper shows removing (not rotating) lowest frequencies improves performance
- The original paper (arXiv:2410.06205) proposes p-RoPE which removes lowest frequencies, not rotates them
- This addresses the misleading summary that suggested rotation of low frequencies was beneficial
@johnhaofu
Copy link
Contributor Author

Note for Reviewers

This is a correction of factual error, not a new blog post. The checklist items in "Preparing the Article" don't apply here as this PR only fixes an inaccurate description of research findings in an existing article.

What was changed:

  • Single line correction in designing-positional-encoding.md
  • Changed misleading description about RoPE paper findings
  • Clarified that the paper proposes removing (not rotating) lowest frequencies

Why this matters:

The original description could mislead readers about the actual research findings from the DeepMind paper (arXiv:2410.06205).

No new assets, thumbnails, or blog metadata changes are needed for this correction.

@merveenoyan
Copy link
Contributor

@FL33TW00D

@FL33TW00D
Copy link
Contributor

Thanks for the tag @merveenoyan.

Thanks for raising this @johnhaofu, you're correct.

@merveenoyan merveenoyan merged commit 4403183 into huggingface:main Jul 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants