Remove fewer Storage calls in CopyProp and GVN #142531

ohadravid · 2025-06-15T07:51:38Z

Modify the CopyProp and GVN MIR optimization passes to remove fewer Storage{Live,Dead} calls, allowing for better optimizations by LLVM - see #141649.

Details

The idea is to use a new MaybeUninitializedLocals analysis and remove only the storage calls of locals that are maybe-uninit when accessed in a new location.

rustbot · 2025-06-15T07:51:44Z

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

matthiaskrgr · 2025-06-15T09:37:15Z

@bors try @rust-timer queue

…try> Remove fewer Storage calls in `copy_prop` Modify the `copy_prop` MIR optimization pass to remove fewer `Storage{Live,Dead}` calls, allowing for better optimizations by LLVM - see #141649. ### Details This is my attempt to fix the mentioned issue (this is the first part, I also implemented a similar solution for GVN in [this branch](https://github.com/rust-lang/rust/compare/master...ohadravid:rust:better-storage-calls-gvn-v2?expand=1)). The idea is to use the `MaybeStorageDead` analysis and remove only the storage calls of `head`s that are maybe-storage-dead when the associated `local` is accessed (or, conversely, keep the storage of `head`s that are for-sure alive in _every_ relevant access). When combined with the GVN change, the final example in the issue (#141649 (comment)) is optimized as expected by LLVM. I also measured the effect on a few functions in `rav1d` (where I originally saw the issue) and observed reduced stack usage in several of them. This is my first attempt at working with MIR optimizations, so it's possible this isn't the right approach — but all tests pass, and the resulting diffs appear correct. r? tmiasko since he commented on the issue and pointed to these passes.

bors · 2025-06-15T09:38:28Z

⌛ Trying commit d24d035 with merge ef7d206...

bors · 2025-06-15T12:05:29Z

☀️ Try build successful - checks-actions
Build commit: ef7d206 (ef7d20666974f0dac45b03e051f2e283f9d9f090)

rust-timer · 2025-06-15T13:31:55Z

Finished benchmarking commit (ef7d206): comparison URL.

Overall result: ❌ regressions - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.3%	[0.2%, 0.4%]	8
Regressions ❌ (secondary)	0.3%	[0.2%, 0.4%]	7
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.3%	[0.2%, 0.4%]	8

Max RSS (memory usage)

Results (primary 0.7%, secondary 3.4%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	3.5%	[1.8%, 5.0%]	5
Regressions ❌ (secondary)	3.4%	[3.4%, 3.4%]	1
Improvements ✅ (primary)	-3.9%	[-6.5%, -2.0%]	3
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.7%	[-6.5%, 5.0%]	8

Cycles

Results (primary -0.6%, secondary -0.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	3.8%	[3.8%, 3.8%]	1
Improvements ✅ (primary)	-0.6%	[-0.6%, -0.6%]	1
Improvements ✅ (secondary)	-4.1%	[-4.1%, -4.1%]	1
All ❌✅ (primary)	-0.6%	[-0.6%, -0.6%]	1

Binary size

Results (primary 0.0%, secondary 0.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	0.2%	[0.0%, 0.8%]	10
Regressions ❌ (secondary)	0.1%	[0.0%, 0.1%]	5
Improvements ✅ (primary)	-0.2%	[-0.8%, -0.0%]	8
Improvements ✅ (secondary)	-0.2%	[-0.2%, -0.2%]	1
All ❌✅ (primary)	0.0%	[-0.8%, 0.8%]	18

Bootstrap: 757.399s -> 756.065s (-0.18%)
Artifact size: 372.20 MiB -> 372.12 MiB (-0.02%)

ohadravid · 2025-06-15T14:55:18Z

@matthiaskrgr - I updated the impl to stop re-checking once a head is found to be maybe-dead, which should be a bit better

matthiaskrgr · 2025-06-15T15:06:54Z

@bors try @rust-timer queue

bors · 2025-06-15T15:08:08Z

⌛ Trying commit 905e968 with merge c0a2949...

…try> Remove fewer Storage calls in `copy_prop` Modify the `copy_prop` MIR optimization pass to remove fewer `Storage{Live,Dead}` calls, allowing for better optimizations by LLVM - see #141649. ### Details This is my attempt to fix the mentioned issue (this is the first part, I also implemented a similar solution for GVN in [this branch](https://github.com/rust-lang/rust/compare/master...ohadravid:rust:better-storage-calls-gvn-v2?expand=1)). The idea is to use the `MaybeStorageDead` analysis and remove only the storage calls of `head`s that are maybe-storage-dead when the associated `local` is accessed (or, conversely, keep the storage of `head`s that are for-sure alive in _every_ relevant access). When combined with the GVN change, the final example in the issue (#141649 (comment)) is optimized as expected by LLVM. I also measured the effect on a few functions in `rav1d` (where I originally saw the issue) and observed reduced stack usage in several of them. This is my first attempt at working with MIR optimizations, so it's possible this isn't the right approach — but all tests pass, and the resulting diffs appear correct. r? tmiasko since he commented on the issue and pointed to these passes.

cjgillot · 2025-06-15T15:45:26Z

Should this check happen in Replacer::visit_local, and move the replacement of storage statements to a dedicated cleanup visitor?

bors · 2025-06-15T17:41:36Z

☀️ Try build successful - checks-actions
Build commit: c0a2949 (c0a294957df10fc3880e1677c72c0cf122485509)

ohadravid · 2025-06-15T18:12:43Z

Should this check happen in Replacer::visit_local

I'm not sure how to make this work: using ResultsCursor requires a &body, but it's not possible to have that while running a MutVisitor since it requires a &mut body.

Is there a different way to do this?

rust-timer · 2025-06-15T20:15:45Z

Finished benchmarking commit (c0a2949): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.3%	[0.2%, 0.4%]	9
Regressions ❌ (secondary)	0.3%	[0.2%, 0.4%]	7
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.2%	[-0.2%, -0.2%]	1
All ❌✅ (primary)	0.3%	[0.2%, 0.4%]	9

Max RSS (memory usage)

Results (primary -0.1%, secondary -1.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	4.2%	[3.4%, 5.8%]	4
Regressions ❌ (secondary)	3.1%	[3.1%, 3.1%]	1
Improvements ✅ (primary)	-4.4%	[-6.6%, -1.8%]	4
Improvements ✅ (secondary)	-5.8%	[-5.8%, -5.8%]	1
All ❌✅ (primary)	-0.1%	[-6.6%, 5.8%]	8

Cycles

Results (secondary -1.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.3%	[2.3%, 2.3%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.6%	[-2.6%, -2.5%]	2
All ❌✅ (primary)	-	-	0

Binary size

Results (primary -0.0%, secondary 0.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	0.2%	[0.0%, 0.8%]	10
Regressions ❌ (secondary)	0.1%	[0.0%, 0.1%]	5
Improvements ✅ (primary)	-0.2%	[-0.8%, -0.0%]	8
Improvements ✅ (secondary)	-0.2%	[-0.2%, -0.2%]	1
All ❌✅ (primary)	-0.0%	[-0.8%, 0.8%]	18

Bootstrap: 756.494s -> 757.685s (0.16%)
Artifact size: 372.15 MiB -> 372.11 MiB (-0.01%)

compiler/rustc_mir_transform/src/copy_prop.rs

ohadravid · 2025-06-29T08:52:24Z

@tmiasko - implemented all the changes 😄

I also updated the GVN code since they applied there are well (use a single storage_to_remove bitset, added a test and a fix for unreachable blocks, updated the base issue test to use MIR, added FileCheck annotations) @cjgillot

I can also split this PR if needed, and I'll polish the git history when you think this looks good enough 🧹

PS
I'm getting (again) CI errors on reordered storage statements in some tests (like tests/mir-opt/pre-codegen/derived_ord.rs), but I'm not sure why.

cjgillot · 2025-07-02T09:36:08Z

compiler/rustc_mir_transform/src/gvn.rs

+        } else {
+            // Remove the storage statements of all the reused locals.
+            state.reused_locals.clone()
+        };


This logic is very close to the one in copy_prop. Could you merge them into a single compute_storage_to_remove function in rustc_mir_transform::ssa? It would take a set of relevant locals (copy-prop : copy_classes[local] != local, GVN : reused_locals) and compute storage_to_remove.

I can, but I think such a function will need to also accept a map: Fn(Local) -> Local (|local| copy_class[local] in copy_prop) since in copy_prop we check the head and not the actual visit_local's local param. We could change the storage check to run after the replacement, but then we'll need to add a separate StorageRemover (similar to GVN).

Edit: Also, after implementing #142531 (comment), the storage check logic diverged even more between the two passes.

compiler/rustc_mir_transform/src/ssa.rs

cjgillot · 2025-07-02T09:45:30Z

compiler/rustc_mir_transform/src/gvn.rs

+            PlaceContext::MutatingUse(MutatingUseContext::AsmOutput)
+            | PlaceContext::MutatingUse(MutatingUseContext::Call)
+            | PlaceContext::MutatingUse(MutatingUseContext::Store)
+            | PlaceContext::MutatingUse(MutatingUseContext::Yield)


They do not require the local to be initialized, but they do require it to have storage. Mixing the two notions makes me uneasy.

I can probably add a test that triggers this. Is the right solution to use two separate analysis checks (so for these we'll check the MaybeStorageDead)?

Actually, thinking about this some more: we know that the reused_locals have valid storage before their original assignments.

For any replacements (eg target of the Store) the local cannot be a reused local anyway.

What do you think?

compiler/rustc_mir_transform/src/copy_prop.rs

… to remove fewer storage statements

…r storage statements

…chable blocks

rustbot assigned tmiasko Jun 15, 2025

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jun 15, 2025

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jun 15, 2025

matthiaskrgr mentioned this pull request Jun 15, 2025

ci: git Unknown option: -C #142534

Open

This comment has been minimized.

Sign in to view

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Jun 15, 2025

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jun 15, 2025

This comment has been minimized.

Sign in to view

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jun 15, 2025

tmiasko reviewed Jun 15, 2025

View reviewed changes

compiler/rustc_mir_transform/src/copy_prop.rs Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

ohadravid requested a review from tmiasko June 29, 2025 08:52

This comment has been minimized.

Sign in to view

cjgillot added the A-mir-opt-GVN Area: MIR opt Global Value Numbering (GVN) label Jul 1, 2025

cjgillot reviewed Jul 2, 2025

View reviewed changes

tmiasko reviewed Jul 2, 2025

View reviewed changes

compiler/rustc_mir_transform/src/copy_prop.rs Outdated Show resolved Hide resolved

compiler/rustc_mir_transform/src/copy_prop.rs Outdated Show resolved Hide resolved

ohadravid added 21 commits July 4, 2025 16:03

Added a mir-opt test for generating storage statements for scoped locals

1a27ca9

Implement MaybeUninitializedLocals analysis for copy_prop mir-opt…

7998070

… to remove fewer storage statements

Added gvn test for removed local storage annotations

c5764f7

Use MaybeUninitializedLocals analysis in GVN mir-opt to remove fewe…

c7c12c6

…r storage statements

Use a single bitset to check the storage in copy_prop, only check rea…

831f5f7

…chable blocks

Added test for preserving head storage

c2d50e6

Move MaybeUninitializedLocals to the ssa module

aaad5c7

Added FileCheck to copy prop storage tests

215d574

Simplify GVN storage checker to use a single bitset

3555cd5

Improve GVN storage tests

4db121d

GVN unnit analysis shouldn't check unreachable blocks

d1f625a

Add more FileCheck statements in gvn storage tests

6d099f6

Added a GVN storage test for borrowed value

19a5268

Improved wording in comments and logs

b287c99

Added a storage usage inverse order test

ff6eafd

Move borrowed_locals filtering out of StorageChecker

8a85783

in copy_prop, we should only ignore storage statement

6e1bb6c

Remove MaybeUninitializedLocals's new

bd09be5

Added another small test to gvn storage

b408a84

fix comment in copy_prop_storage_preserve_head.rs

e542fc9

rerun and bless again after rebase (reordered storage statements)

1f45aef

ohadravid force-pushed the better-storage-calls-copy-prop branch from 8963fa2 to 1f45aef Compare July 4, 2025 13:14

ohadravid requested review from cjgillot and tmiasko July 4, 2025 17:31

Remove fewer Storage calls in CopyProp and GVN #142531

Are you sure you want to change the base?

Remove fewer Storage calls in CopyProp and GVN #142531

Conversation

ohadravid commented Jun 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Details

Uh oh!

rustbot commented Jun 15, 2025

Uh oh!

matthiaskrgr commented Jun 15, 2025

Uh oh!

This comment has been minimized.

bors commented Jun 15, 2025

Uh oh!

bors commented Jun 15, 2025

Uh oh!

This comment has been minimized.

rust-timer commented Jun 15, 2025

Overall result: ❌ regressions - please read the text below

Uh oh!

ohadravid commented Jun 15, 2025

Uh oh!

matthiaskrgr commented Jun 15, 2025

Uh oh!

This comment has been minimized.

bors commented Jun 15, 2025

Uh oh!

cjgillot commented Jun 15, 2025

Uh oh!

bors commented Jun 15, 2025

Uh oh!

This comment has been minimized.

ohadravid commented Jun 15, 2025

Uh oh!

rust-timer commented Jun 15, 2025

Overall result: ❌✅ regressions and improvements - please read the text below

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

ohadravid commented Jun 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

Choose a reason for hiding this comment

Uh oh!

ohadravid Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ohadravid commented Jun 15, 2025 •

edited

Loading

ohadravid commented Jun 29, 2025 •

edited

Loading

ohadravid Jul 4, 2025 •

edited

Loading