Refine `UriUtils#decode` and `StringUtils#uriDecode` implementation and documentation

[gotson]

I believe i found an issue with UriUtils.decode(String source, Charset charset).

When the source string contains the character ’ (Right Single Quotation Mark - unicode 2019), and other URI characters (to trigger the rewrite of the string), the character is changed to � (End of Medium - unicode 0019).

Here is a sample test in Kotlin that highlights the behaviour:

@Test
  fun test(){
    val c = '\u2019'
    val s = "%20$c"
    val expected = " $c"

    val d = UriUtils.decode(s, Charsets.UTF_8)

    assertThat(d).isEqualTo(expected)
  }

Here is the difference shown from the failing test:

" ’"
" �"

I am using Spring 6.2.0.

[sdeleuze]

I suspect an inconsistency between the provided UTF-16 hexadecimal value and the UTF-8 charset specified, see https://www.fileformat.info/info/unicode/char/2019/index.htm.

Also if you check the Javadoc of the underlying StringUtils#uriDecode method, if am not sure what you try to do is supported ("For all other characters (including those already decoded), the output is undefined").

Any thoughts?

[nosan]

I expect StringUtils.uriDecode to behave similarly to URLDecoder, except that the + sign is treated as a literal + instead of a space (' ').

@Test
void uriDecode() {
    assertThat(URLDecoder.decode("%20\u2019", StandardCharsets.UTF_8)).isEqualTo(" ’"); // success
    assertThat(URLDecoder.decode("\u2019", StandardCharsets.UTF_8)).isEqualTo("’"); // success
    assertThat(StringUtils.uriDecode("\u2019", StandardCharsets.UTF_8)).isEqualTo("’"); // success
    assertThat(StringUtils.uriDecode("%20\u2019", StandardCharsets.UTF_8)).isEqualTo(" ’"); // fail
}

[sdeleuze]

@nosan Despite its name, URLDecoder is per its Javadoc an "utility class for HTML form decoding", it has a different purpose so those differences of behavior are expected.

UriUtils#decode and StringUtils#uriDecode are not meant to deal with multi-byte characters, so that issue is almost a duplicate of #32360. I am turning this issue into a documentation one to do another round of Javadoc refinement to be about more specific on the requirement of the input, and clarify why URLDecoder is linked.