Skip to content

[v25.1.x] storage: CORE-10056: Remove contiguous allocations in lock_manager #26168

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

vbotbuildovich
Copy link
Collaborator

Backport of PR #26111

The `range` function allocated a std::vector of futures before
acquiring all locks, and then held the locks in a std::vector.
This was observed causing OOM in a large cluster. The std::vector
usage has been replaced with chunked_vector.

(cherry picked from commit a59cdd8)
@vbotbuildovich vbotbuildovich added this to the v25.1.x-next milestone May 15, 2025
@vbotbuildovich vbotbuildovich added the kind/backport PRs targeting a stable branch label May 15, 2025
@vbotbuildovich vbotbuildovich requested a review from wdberkeley May 15, 2025 18:35
@wdberkeley wdberkeley requested a review from andrwng May 15, 2025 18:44
@vbotbuildovich
Copy link
Collaborator Author

vbotbuildovich commented May 15, 2025

Retry command for Build#66074

please wait until all jobs are finished before running the slash command

/ci-repeat 1
tests/rptest/tests/cloud_storage_timing_stress_test.py::CloudStorageTimingStressTest.test_cloud_storage_with_partition_moves@{"cleanup_policy":"compact,delete"}
tests/rptest/tests/compaction_recovery_test.py::CompactionRecoveryUpgradeTest.test_index_recovery_after_upgrade
tests/rptest/tests/random_node_operations_test.py::RandomNodeOperationsTest.test_node_operations@{"cloud_storage_type":1,"enable_failures":false,"mixed_versions":false,"with_chunked_compaction":false,"with_iceberg":true,"with_tiered_storage":true}
tests/rptest/tests/scram_test.py::SaslPlainTest.test_plain_authn@{"client_type":3,"sasl_plain_enabled":true,"scram_type":1}
tests/rptest/tests/random_node_operations_test.py::RandomNodeOperationsTest.test_node_operations@{"cloud_storage_type":1,"enable_failures":true,"mixed_versions":false,"with_chunked_compaction":true,"with_iceberg":false,"with_tiered_storage":true}

@vbotbuildovich
Copy link
Collaborator Author

CI test results

test results on build#66074
test_class test_method test_arguments test_kind job_url test_status passed reason
CloudStorageTimingStressTest test_cloud_storage_with_partition_moves {"cleanup_policy": "compact,delete"} ducktape https://buildkite.com/redpanda/redpanda/builds/66074#0196d57e-ad71-4fd0-b6b0-7eadf86c0505 FLAKY 20/21 upstream reliability is '96.96969696969697'. current run reliability is '95.23809523809523'. drift is 1.7316 and the allowed drift is set to 0. The test should FAIL
CompactionRecoveryUpgradeTest test_index_recovery_after_upgrade ducktape https://buildkite.com/redpanda/redpanda/builds/66074#0196d57e-ad71-4fd0-b6b0-7eadf86c0505 FLAKY 20/21 upstream reliability is '96.96969696969697'. current run reliability is '95.23809523809523'. drift is 1.7316 and the allowed drift is set to 0. The test should FAIL
RandomNodeOperationsTest test_node_operations {"cloud_storage_type": 1, "enable_failures": false, "mixed_versions": false, "with_chunked_compaction": false, "with_iceberg": true, "with_tiered_storage": true} ducktape https://buildkite.com/redpanda/redpanda/builds/66074#0196d57e-ad6f-4db7-a5cd-d4c405ccb7ea FLAKY 20/21 upstream reliability is '100.0'. current run reliability is '95.23809523809523'. drift is 4.7619 and the allowed drift is set to 0. The test should FAIL
RandomNodeOperationsTest test_node_operations {"cloud_storage_type": 1, "enable_failures": true, "mixed_versions": false, "with_chunked_compaction": true, "with_iceberg": false, "with_tiered_storage": true} ducktape https://buildkite.com/redpanda/redpanda/builds/66074#0196d57e-ad72-4ea1-ac9c-0cdc608bd793 FLAKY 20/21 upstream reliability is '100.0'. current run reliability is '95.23809523809523'. drift is 4.7619 and the allowed drift is set to 0. The test should FAIL
SaslPlainTest test_plain_authn {"client_type": 3, "sasl_plain_enabled": true, "scram_type": 1} ducktape https://buildkite.com/redpanda/redpanda/builds/66074#0196d57e-ad6f-4db7-a5cd-d4c405ccb7ea FLAKY 20/21 upstream reliability is '100.0'. current run reliability is '95.23809523809523'. drift is 4.7619 and the allowed drift is set to 0. The test should FAIL

@WillemKauf
Copy link
Contributor

/ci-repeat 1
skip-redpanda-build
skip-units
tests/rptest/tests/cloud_storage_timing_stress_test.py::CloudStorageTimingStressTest.test_cloud_storage_with_partition_moves@{"cleanup_policy":"compact,delete"}
tests/rptest/tests/compaction_recovery_test.py::CompactionRecoveryUpgradeTest.test_index_recovery_after_upgrade
tests/rptest/tests/random_node_operations_test.py::RandomNodeOperationsTest.test_node_operations@{"cloud_storage_type":1,"enable_failures":false,"mixed_versions":false,"with_chunked_compaction":false,"with_iceberg":true,"with_tiered_storage":true}
tests/rptest/tests/scram_test.py::SaslPlainTest.test_plain_authn@{"client_type":3,"sasl_plain_enabled":true,"scram_type":1}
tests/rptest/tests/random_node_operations_test.py::RandomNodeOperationsTest.test_node_operations@{"cloud_storage_type":1,"enable_failures":true,"mixed_versions":false,"with_chunked_compaction":true,"with_iceberg":false,"with_tiered_storage":true}

@wdberkeley wdberkeley merged commit 011385a into redpanda-data:v25.1.x May 16, 2025
17 checks passed
@piyushredpanda piyushredpanda modified the milestones: v25.1.x-next, v25.1.5 Jun 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/build area/redpanda kind/backport PRs targeting a stable branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants