From 9f40cee207254a22022489fb1c889ccca67f2c36 Mon Sep 17 00:00:00 2001 From: Aria Beingessner Date: Fri, 25 Mar 2022 11:14:01 -0400 Subject: [PATCH 1/3] Note that raw string literals are context-sensitive See https://github.com/rust-lang/rust/blob/5187be620c76a313a19b9b596e1bce3a80a345dd/src/grammar/raw-string-literal-ambiguity.md for the original formal proof of this fact. --- src/tokens.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/tokens.md b/src/tokens.md index fb27052da..613f66a52 100644 --- a/src/tokens.md +++ b/src/tokens.md @@ -208,6 +208,14 @@ the characters `U+0022` (double-quote) (except when followed by at least as many `U+0023` (`#`) characters as were used to start the raw string literal) or `U+005C` (`\`) do not have any special meaning. +**Note that this is a _context-sensitive_ grammar, as opposed to _context-free_.** +This is because strings like `r###"I contain only 2 "##s so I'm ok"###` require +a parser to properly count the number of opening #'s and compare that count to two +different values. In practical terms this is very easy for a parser to do, but +a context-free language can't because the only way to express "counting" is +as a destructive operation which forces you to forget the count. This allows +for the "comparison" of two counts (such (as (balanced) (parens))) but not 3+. + Examples for string literals: ```rust From 53e0a8bd6f250cf6f27204ba082f7c540be50581 Mon Sep 17 00:00:00 2001 From: Aria Beingessner Date: Fri, 25 Mar 2022 12:10:03 -0400 Subject: [PATCH 2/3] Link the old context-sensitive proof. --- src/tokens.md | 1 + 1 file changed, 1 insertion(+) diff --git a/src/tokens.md b/src/tokens.md index 613f66a52..5437800a5 100644 --- a/src/tokens.md +++ b/src/tokens.md @@ -216,6 +216,7 @@ a context-free language can't because the only way to express "counting" is as a destructive operation which forces you to forget the count. This allows for the "comparison" of two counts (such (as (balanced) (parens))) but not 3+. +See [the proof for more formal details](https://github.com/rust-lang/rust/blob/5187be620c76a313a19b9b596e1bce3a80a345dd/src/grammar/raw-string-literal-ambiguity.md). Examples for string literals: ```rust From 05fd5e73194eb615d5d6327cbac35ee7f78651fb Mon Sep 17 00:00:00 2001 From: Aria Beingessner Date: Fri, 25 Mar 2022 12:24:35 -0400 Subject: [PATCH 3/3] fixup newline --- src/tokens.md | 1 + 1 file changed, 1 insertion(+) diff --git a/src/tokens.md b/src/tokens.md index 5437800a5..4c8dd0fda 100644 --- a/src/tokens.md +++ b/src/tokens.md @@ -217,6 +217,7 @@ as a destructive operation which forces you to forget the count. This allows for the "comparison" of two counts (such (as (balanced) (parens))) but not 3+. See [the proof for more formal details](https://github.com/rust-lang/rust/blob/5187be620c76a313a19b9b596e1bce3a80a345dd/src/grammar/raw-string-literal-ambiguity.md). + Examples for string literals: ```rust