Perf regression caused by initial support of RFC 3191

[ridwanabdillahi]

Spawned off of #91779 (comment) . The perf run shows a noticeable regression after the initial round of changes to implement RFC 3191.

This issue is meant to track resolving the perf regression, so it doesn't get lost in the noise.

@rustbot claim

[ridwanabdillahi]

I ran a variety of experiments to improve the perf regression, but it seems this was caused by the overall overhead of adding a new feature which not only added new symbols but also added fields to the CrateMetadata object. This causes more data to be serialized and deserialized for every crate even if the items are blank. The performance analysis showed that the majority of scenarios causing the regression were the incr-unchanged scenario which is what I would expect from additional metadata reads.