SwiftSyntax support for module selectors #3091

beccadax · 2025-06-02T19:44:22Z

This PR adds module selectors to SwiftSyntax and SwiftParser (draft of matching compiler feature in swiftlang/swift#34556). This feature was pitched ages ago; a proper proposal is ~~on my todo list~~ now available.

Note: This PR is still a work in progress—in particular, I need to go back and improve the tests, both by expanding their coverage and by adding assertions about the resulting syntax trees—but I'd appreciate feedback on the design while I'm working on that.

Reviewers: If you review this commit-by-commit, I've separated out the big dumb mechanical changes from the ones that actually change parsing logic.

nkcsgexi · 2025-06-02T19:47:56Z

🥳

Sources/SwiftParser/Names.swift

This capability is currently unused, but it’ll matter soon.

And make sure it doesn’t break Objective-C selector lexing.

Treating introducers as keywords is now always conditional on whether `shouldParsePatternBinding(introducer:)` returns `true`. That method has also been modified to correctly handle an edge case with wildcard patterns.

beccadax · 2025-07-10T03:10:31Z

@swift-ci please test

beccadax · 2025-07-11T00:11:22Z

@swift-ci please test

beccadax · 2025-07-11T20:50:43Z

@swift-ci please test

beccadax · 2025-07-12T04:10:37Z

@swift-ci please test

nkcsgexi · 2025-07-14T18:41:10Z

@swift-ci please test macOS

nkcsgexi · 2025-07-14T18:41:24Z

@swift-ci please test Linux

nkcsgexi · 2025-07-14T21:06:15Z

This build failure seems to be real:

12:29:57  /Users/ec2-user/jenkins/workspace/swift-syntax-PR-macOS/branch-main/swift-syntax/Sources/SwiftSyntax/generated/SyntaxTraits.swift:209:7: error: protocol requirement 'moduleSelector' cannot be declared '@_spi' without a default implementation in a protocol extension
12:29:57  207 | 
12:29:57  208 |   @_spi(ExperimentalLanguageFeatures)
12:29:57  209 |   var moduleSelector: ModuleSelectorSyntax? {
12:29:57      |       `- error: protocol requirement 'moduleSelector' cannot be declared '@_spi' without a default implementation in a protocol extension
12:29:57  210 |     get
12:29:57  211 |     set

beccadax · 2025-07-15T00:09:10Z

I can't actually reproduce that failure locally, but I'm pushing a speculative fix.

beccadax · 2025-07-15T00:20:09Z

@swift-ci please test

beccadax · 2025-07-15T01:39:59Z

@swift-ci please test macOS

beccadax · 2025-07-15T01:40:08Z

@swift-ci please test Linux

beccadax · 2025-07-15T01:40:19Z

@swift-ci please test Windows

nkcsgexi · 2025-07-15T17:21:22Z

We are good now with macOS and Linux. Let's try Windows again. @swift-ci please test Windows.

nkcsgexi · 2025-07-15T21:54:33Z

hmm, the Windows CI hit a compiler crasher:

[519/528] Compiling SwiftDiagnostics Convenience.swift
error: compile command failed due to exception 3 (use -v to see invocation)
SIL memory lifetime failure in @$s11SwiftParser13TokenConsumerPAAE27consumeModuleSelectorTokens0C0Qz22moduleNameOrUnexpected_AF010colonColonC0SayAFG5extratSgyF: memory is initialized, but shouldn't be

rintaro

Apologies I didn't review this sooner. I still haven't look into the implementation closely, but here's the first round 🙇

CodeGeneration/Sources/SyntaxSupport/CommonNodes.swift

Sources/SwiftParser/Names.swift

rintaro · 2025-07-16T18:56:59Z

Sources/SwiftParser/Names.swift

+    // Technically the current token *should* be an identifier, but we also want to diagnose other tokens that might be
+    // used by accident (particularly keywords and `_`). However, we don't want to consume tokens which would make the
+    // surrounding structure mis-parse.
+    return self.at(anyIn: StructuralTokens.self) == nil


I don't think it's a good idea to (almost) always consider <token> :: as a module qualifier. E..g.

class ::

This looks to me an incomplete class declaration and just an orphan ::. Even if it's on the same line with no-space, I don't think we need to parse it as a module selector.

beccadax · 2025-07-23T22:08:07Z

Okay, this is a dramatically simpler implementation that doesn't try to do anything special to handle module selectors at invalid locations. I can't say I'm happy with some of the recovery behavior I'm seeing—in particular, parameter lists and tuple patterns seem to just throw up their hands in a pretty ugly way instead of recovering to a TokenKind.comma—but maybe that's a more general problem.

rintaro · 2025-07-23T22:59:47Z

Sources/SwiftParser/Names.swift

+
+extension Parser {
+  /// Parses one or more module selectors, if present.
+  mutating func parseModuleSelector() -> RawModuleSelectorSyntax? {


I know there're several parsing functions that doesn't follow this convention, but:

Suggested change

mutating func parseModuleSelector() -> RawModuleSelectorSyntax? {

mutating func parseModuleSelectorIfPresent() -> RawModuleSelectorSyntax? {

or check self.isAtModuleSelector() before calling this and make this non-optional.

A module selector is never mandatory, so I'm going to rename the method.

rintaro · 2025-07-23T23:09:03Z

Sources/SwiftParser/Names.swift

@@ -68,6 +213,9 @@ extension Parser {
  }

  mutating func parseDeclReferenceExpr(_ flags: DeclNameOptions = []) -> RawDeclReferenceExprSyntax {
+    // Consume a module selector if present.
+    let moduleSelector = self.parseModuleSelector()


I think presence of the module selector should affect flags (DeclNameOptions.keywords).
It would be weird if ModName::default is rejected, but foo.ModName::default is allowed.
related: https://forums.swift.org/t/pitch-module-selectors/80835/82

rintaro · 2025-07-23T23:53:18Z

Sources/SwiftParser/Names.swift

+    }
+
+    // Technically the current token *should* be an identifier, but we also want to diagnose other tokens that might be
+    // used by accident (particularly keywords and `_`). However, we don't want to consume tokens which would make the


Would you show me some examples where consuming keywords as a invalid module identifiers improves the diagnostics?

IMO, keyword as a module name rule should follow existing keyword rules should be like:

Bare keywords as module names are allowed after . (following SE-0071). Otherwise it's not a module selector even if there's :: after that

Some special keywords such as Self, self, _ and Any should be parsed as reserved names (even without .). And they should be rejected/diagnosed accordingly, either in the parser or the type checker.

I'll narrow this to accept .wildcard, .binaryOperator, and the keywords self, Self, super, and Any in addition to identifiers. (These extra tokens are still not valid—they're just assumed to be attempts to write an identifier.) Does that sound like a reasonable list?

Hmm I'm not 100% sure about .binaryOperator. Actually that brought up another question which is whether : should be a "bound" character. (i.e. *::foo is "binary" or "prefix" operator). Either way, for example for foo + ::bar, for me, this look like a missing module identifier before ::. Maybe special casing * with any operator kind would be more reasonable.
(though I personally can't imagine *::ignite() would be a thing because it can't be use after . I.e. for foo.*::ignite(), .* is an operator)

.wildcard, self, Self, super and Any sounds good to me.

Ah foo + ::bar is parsed as a regular binary operator expression either way. Hmm maybe fine then... 🤔

rintaro · 2025-07-24T17:08:13Z

Sources/SwiftSyntax/generated/RenamedChildrenCompatibility.swift

@@ -1473,13 +1473,12 @@ extension ConsumeExprSyntax {
 }

 extension DeclReferenceExprSyntax {
-  @available(*, deprecated, renamed: "unexpectedBeforeBaseName")
-  public var unexpectedBeforeIdentifier: UnexpectedNodesSyntax? {


This accessor is removed now? I think it's okay, but any idea how much effort would it take to preserve this?

Ah, there was a bug in the changes I made to hide experimental children in compatibility layers. Will fix.

rintaro · 2025-07-24T17:59:36Z

Sources/SwiftParser/Declarations.swift

+
+      // If we got `::` when it's not allowed, reject it
+      var unexpectedBeforeTrailingPeriod: RawUnexpectedNodesSyntax?
+      if keepGoing?.tokenKind == .colonColon && !hasImportKind {


This allows:

import struct Foo::Bar::Baz

which I don't think intentional?

Also nit: If you use optional var please initialize it with nil explicitly. Or make it let and make else branch to initialize it.

import struct Foo.Bar.Baz is also semantically invalid but permitted by SwiftParser. ASTGen needs to diagnose both, and the same check would handle both of them.

Clang submodules are valid and accepted in import-path, so ASTGen won't diagnose it.
E.g. import class Foundation.NSData.NSData works. I know this is equivalent with import class Foundation.NSData, but we cannot just reject it.

Huh—that fact dropped out of my memory, but I must have known it when I created ImportPath in the compiler back in 2020, because it's clearly designed to support it. Weird how that happens sometimes.

I'll have to decide if I want to support import class Foundation.NSData::NSData or not. (I'm thinking no—we allow :: there so that we allow the same syntax that's valid at a use site, and in the current proposal a submodule wouldn't be valid at a use site.)

I agree we should NOT support import class Foundation.NSData::NSData.
If isAtModuleSpecifier() after import-kind, I think it's okay to just parse the module specifier and the following identifier, then return it.

Initializers for nodes with experimental node children need to be marked `@_spi`. This PR: • Adds that attribute. • Generates an alternative which *doesn’t* use SPI as part of the compatibility layer. • As a side effect, adds a `Child.Refactoring.introduced` case that can be used to generate compatibility `unexpected` properties. No functional change in this commit, but it will affect the code generation in the next one.

Changes the syntax tree to represent module selectors: • A `ModuleSelectorSyntax` node represents a module selector abstractly. • The following nodes now have an optional `moduleSelector` child: • `DeclReferenceExprSyntax` • `IdentifierTypeSyntax` • `MacroExpansionExprSyntax` • `MemberTypeSyntax` • BasicFormat knows the preferred format for module selectors. Other components, particularly the parser, were also updated to continue building, though without any changes in behavior. Parser implementation will come in a future commit.

Changes it to share code with `parseTypeIdentifier()` and clean up the member type parsing a little. Also tweaks call sites of `parseTypeIdentifier()`.

This commit ports over tests from the compiler’s (future) `test/NameLookup/module_selector.swift` file and makes sure the correct uses parse as expected. It also tests that ill-formed module selectors (ones with a missing or non-identifier module name) are diagnosed correctly. This commit doesn’t handle module selectors in scoped `import` statements; the related test has been XFAILed.

This restricts the code styles developers can use, but makes it less likely to eat syntax that was meant to belong to a different statement after an incomplete line.

beccadax force-pushed the mod-squad branch from a20cd45 to d7511a3 Compare June 2, 2025 19:48

beccadax commented Jun 2, 2025

View reviewed changes

Sources/SwiftParser/Names.swift Outdated Show resolved Hide resolved

beccadax force-pushed the mod-squad branch from d7511a3 to 4c34e4b Compare June 4, 2025 00:57

[NFC] Thread experimental features through lexer

69d50cd

This capability is currently unused, but it’ll matter soon.

beccadax force-pushed the mod-squad branch 3 times, most recently from 1f47aa2 to eaa6956 Compare July 9, 2025 22:04

beccadax added 4 commits July 9, 2025 15:24

Add experimental colonColon token

1360c37

And make sure it doesn’t break Objective-C selector lexing.

[NFC] Refactor parsing of suppressed types

06f2097

[NFC] Tighten checking for PBD introducers

ee93aa6

Treating introducers as keywords is now always conditional on whether `shouldParsePatternBinding(introducer:)` returns `true`. That method has also been modified to correctly handle an edge case with wildcard patterns.

[NFC] Correct @_implements diagnostic typo

d848067

beccadax force-pushed the mod-squad branch 2 times, most recently from da6eff6 to f94b58a Compare July 10, 2025 03:00

beccadax force-pushed the mod-squad branch from f94b58a to 7652336 Compare July 10, 2025 21:17

beccadax marked this pull request as ready for review July 11, 2025 20:50

beccadax requested review from ahoppen, bnbarham and hamishknight as code owners July 11, 2025 20:50

beccadax force-pushed the mod-squad branch from 7652336 to 8adfdb1 Compare July 11, 2025 23:26

beccadax requested a review from rintaro July 15, 2025 18:21

rintaro reviewed Jul 16, 2025

View reviewed changes

beccadax force-pushed the mod-squad branch from 46eb9ee to bb512ff Compare July 23, 2025 22:01

beccadax force-pushed the mod-squad branch from bb512ff to 533ae59 Compare July 23, 2025 23:42

rintaro reviewed Jul 23, 2025

View reviewed changes

rintaro reviewed Jul 24, 2025

View reviewed changes

beccadax added 6 commits July 25, 2025 16:53

[NFC] Refactor parseQualifiedTypeIdentifier()

10a6ea9

Changes it to share code with `parseTypeIdentifier()` and clean up the member type parsing a little. Also tweaks call sites of `parseTypeIdentifier()`.

Support module selectors in scoped imports

2807c92

Forbid newline after ::

5f9ae5d

This restricts the code styles developers can use, but makes it less likely to eat syntax that was meant to belong to a different statement after an incomplete line.

beccadax force-pushed the mod-squad branch from 533ae59 to 5f9ae5d Compare July 31, 2025 00:47

	mutating func parseModuleSelector() -> RawModuleSelectorSyntax? {
	mutating func parseModuleSelectorIfPresent() -> RawModuleSelectorSyntax? {

SwiftSyntax support for module selectors #3091

Are you sure you want to change the base?

SwiftSyntax support for module selectors #3091

Conversation

beccadax commented Jun 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nkcsgexi commented Jun 2, 2025

Uh oh!

Uh oh!

beccadax commented Jul 10, 2025

Uh oh!

beccadax commented Jul 11, 2025

Uh oh!

beccadax commented Jul 11, 2025

Uh oh!

beccadax commented Jul 12, 2025

Uh oh!

nkcsgexi commented Jul 14, 2025

Uh oh!

nkcsgexi commented Jul 14, 2025

Uh oh!

nkcsgexi commented Jul 14, 2025

Uh oh!

beccadax commented Jul 15, 2025

Uh oh!

beccadax commented Jul 15, 2025

Uh oh!

beccadax commented Jul 15, 2025

Uh oh!

beccadax commented Jul 15, 2025

Uh oh!

beccadax commented Jul 15, 2025

Uh oh!

nkcsgexi commented Jul 15, 2025

Uh oh!

nkcsgexi commented Jul 15, 2025

Uh oh!

rintaro left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

rintaro Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

beccadax commented Jul 23, 2025

Uh oh!

rintaro Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rintaro Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

beccadax Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rintaro Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

beccadax commented Jun 2, 2025 •

edited

Loading

rintaro left a comment •

edited

Loading

rintaro Jul 16, 2025 •

edited

Loading

rintaro Jul 23, 2025 •

edited

Loading

rintaro Jul 23, 2025 •

edited

Loading

beccadax Jul 29, 2025 •

edited

Loading

rintaro Jul 29, 2025 •

edited

Loading

rintaro Jul 25, 2025 •

edited

Loading