Write more unicode tests

[sophiajt]

We had only a couple unicode tests before and it's difficult to see how accurate they were because of how they were testing.

I've since moved the main one to a UI test, which is better, but I think unicode needs to be more thoroughly tested using the new UI mode.

This issue is a todo item to go and add a few more ui tests for various unicode to ensure we've got better coverage.

[Mark-Simulacrum]

@jonathandturner Could you give an update on this? I presume that not much has been done here, but perhaps we could write up what we want to test with regards to unicode and make it an E-easy bug?

[sophiajt]

@Mark-Simulacrum - Quite possibly. Someone who is working in error messages more than I am these days may be able to take this and run with it. The gist is pretty simple, at the time we only had a few unicode tests in the ui directory. I'm not aware of the current state of affairs in the error message testing.

[Mark-Simulacrum]

Okay, so I think what we need here is a concrete list of things that we should test with regards to Unicode, that is, what edge cases we should make sure work. I can't think of anything concrete right now, though.

[behnam]

@jonathandturner, could you please share more about which area of rustc this is issue is about? Is this about the std_unicode crate, compiler itself, or some other thing?

[sophiajt]

@behnam - this was a long time ago, but iirc, the issue was that we only had a handful of unicode tests for the compiler, specifically compile-fail and ui tests.

[czipperz]

What's the progress here? Is this resolved?

[rylev]

Triage: I'm going to close this as it's a bit too general IMO to be helpful. There are several unicode related tests in the compiler for cases where unicode characters end up in places they should not be. A quick search of "unicode" in GitHub reveals many of them. We can reopen this issue if we still feel like that cases where unicode edge cases are not being handled, but we should probably include a list of such cases.

cc @estebank as this largely has to do with providing good diagnostics.