Revert to macOS-14 image for ARM64 iOS testing

[freakboy3742]

In an attempt to conserve disk space, GitHub Actions has stopped consistently providing iOS simulator images. See
actions/runner-images#12541 and actions/runner-images#12751 for discussion of this transition.

This means the "default" configuration no longer works for iOS simulation. This PR reverts to using the macos-14 image for now, to restore iOS CI to a working state.

(A previous version of this PR tried to fix the problem by explicitly setting the Xcode version; although that approach is what GitHub recommends, at the current time their simulator images are... not working, so macos-14 is the working option option)

[freakboy3742]

Ok - this clearly isn't it by itself; I need to dig deeper to work out why the simulators are either failing to start, or locking up while starting.

[henryiii]

A temporary workaround might be to use macOS-14 (more for projects affected than us). And it looks like 16.3+ is still provided, so I'm not sure why this would be affecting us, if 16.4 is fine?

[henryiii]

Any idea how long it might take to work out a fix? (Trying to see if it's worth delaying a release over, and also trying to judge if I should try the macos-14 workaround in places like pybind11).

[henryiii]

From what I understand, it's the iOS component missing, not the Xcode version?

[freakboy3742]

Any idea how long it might take to work out a fix? (Trying to see if it's worth delaying a release over, and also trying to judge if I should try the macos-14 workaround in places like pybind11).

Working on this will be at the top of my todo list on Monday (my time; Sunday evening US time). I can't absolutely promise I'll have a fix by EOB Monday, but that will be my goal.

If you need something before then, the macos-14 fix might be worth a try; performance of the simulator isn't as good on 14 runners for some reason... but that beats not working at all.

And yes - Xcode is still installed; but the iOS SDK and simulators aren't. The problem is working out what combination of simulators is installed, because the default behavior that the iOS testbed is using doesn't appear to be working. There's also a pretty big performance disparity when using certain simulator/SDK combos, which I think is related to "first time load/run" caching - the default simulators avoid this.

[freakboy3742]

I've spent the day trying to work out what is going on with the 20250811 macOS-15 image... and the short version is it's a mess.

• They've removed all but the "most recent 3" SDK and simulator runtimes from the macOS-15 image.
• They've kept the default Xcode version as 16.0 - but that defaults to iOS 18.0 as a default SDK, so builds fail.
• They've announced an intention to move to Xcode 16.4 on August 25, and suggest that if you're updating now, you might as well preventatively update to 16.4 now.
• If you do switch to Xcode 16.4, it should default to the iOS 18.5 simulator - but the simulators appear to be configured in such a way that test devices won't start.

I've logged the problem with Github, as it seems to indicate a bigger problem with how they've configured iOS simulators.

I think there might be an alternative fix that involves explicitly naming the simulator that you want to use to run the tests - the current CPython iOS testbed does do discovery to find an appropriate simulator, but it only looks for a device name, not an OS version as well. I'll continue to investigate this, but for now, reverting to macOS-14 appears the best option.

[henryiii]

Should we also note this in our docs?

[freakboy3742]

Should we also note this in our docs?

If it persists, it should definitely be noted; however, initial responses on the ticket seem to suggest that GitHub recognises there's something wrong with the image that they've published (or the simulators on the image, anyway), so I've got some hope it might be resolved and we can restore macOS-15 testing.