Add HTTPBenchmarkApp with consolidated "real‑world" benchmarks #101
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Motivation: Added a self‑contained SwiftNIO example that demonstrates realistic throughput and latency scenarios—file streaming, high‑concurrency sums, partial I/O patterns, and lock contention. A command‑line tool that can run these benchmarks in‑process and print percentile statistics helps both users and contributors understand and tune performance. Modications: - `HTTPBenchmarkApp` implementing four HTTP endpoints and four in‑process benchmarks. - CLI flags: - `--run-all-benchmarks` to run all scenarios back‑to‑back. - `--samples <N>` to customize iteration count (default 10). - `--use-io-uring` to switch to `NIOTSEventLoopGroup` when available. - Benchmark helpers: - `measure(_:)` and `measureMultiple(iterations:block:)` for timing. - `calculateStatistics(from:)` to compute p0/p25/p50/p75/p90/p99/p100. - `formatBenchmarkTable(metric:stats:)` to render Unicode tables. Results: Users can now build and run a single tool to: 1. Execute workload benchmarks entirely in‑process, with configurable sample counts and percentile output. 2. Optionally serve HTTP endpoints for external latency measurements. This provides an extensible SwiftNIO example app that is both educational and practically useful for performance tuning.
Lukasa
requested changes
Apr 23, 2025
| @Option(help: "Host to bind on") var host: String = "127.0.0.1" | ||
| @Option(help: "Port to bind on") var port: Int = 8080 | ||
| @Option(help: "Number of samples for each consolidated benchmark") var samples: Int = 10 | ||
| @Flag(help: "Enable io_uring backend (requires NIOTransportServices)") var useIOUring: Bool = false |
There was a problem hiding this comment.
This looks wrong: NIOTransportServices doesn't use io_uring, it uses Network.framework.
| // will execute consolidated benchmarks and output a report. | ||
| // Optionally, the --use-io-uring flag enables NIOTSEventLoopGroup (Linux io_uring). | ||
| // | ||
| //===----------------------------------------------------------------------===// |
There was a problem hiding this comment.
This license header isn't valid.
| let server = try bootstrap.bind(host: host, port: port).wait() | ||
| print("HTTPBenchmarkApp running on \(host):\(port)") | ||
| try server.closeFuture.wait() | ||
| try group.syncShutdownGracefully() |
There was a problem hiding this comment.
This seems a bit odd: we either run benchmarks or a HTTP server. Why that choice?
| } | ||
| dg.wait() | ||
| } | ||
| print(formatBenchmarkTable(metric: "Lock Contention (ms)", stats: stats4)) |
There was a problem hiding this comment.
These benchmarks don't seem to be particularly HTTP related.
| } | ||
| } | ||
|
|
||
| extension BenchmarkRequestHandler: @unchecked Sendable {} |
There was a problem hiding this comment.
This type definitely isn't Sendable.
|
Thank you for your reviews @Lukasa! Just so it doesn't seem like I've abandoned this project, I wanted to put this here: I'll continue working on this right after my exams. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivations
Modifications
HTTPBenchmarkAppimplementing four HTTP endpoints and four in‑process benchmarks.--run-all-benchmarksto run all scenarios back‑to‑back.--samples <N>to customize iteration count (default 10).--use-io-uringto switch toNIOTSEventLoopGroupwhen available.measure(_:)andmeasureMultiple(iterations:block:)for timing.calculateStatistics(from:)to compute p0/p25/p50/p75/p90/p99/p100.formatBenchmarkTable(metric:stats:)to render Unicode tables.Results
Users can now build and run a single tool to:
1. Execute workload benchmarks entirely in‑process, with configurable sample counts and percentile output.
2. Optionally serve HTTP endpoints for external latency measurements.
This provides an extensible SwiftNIO example app that is useful for performance tuning.