[SampleFDO][TypeProf]Support vtable type profiling for ext-binary and text format #148002
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
mingmingl-llvm
requested review from
snehasish,
modiking,
WenleiHe,
david-xl,
teresajohnson and
wlei-llvm
|
@llvm/pr-subscribers-pgo Author: Mingming Liu (mingmingl-llvm) ChangesThis change extends SampleFDO ext-binary and text format to record the vtable symbols and their counts for virtual calls inside a function. The vtable profiles will allow the compiler to annotate vtable types on IR instructions and perform vtable-based indirect call promotion. An RFC is in https://discourse.llvm.org/t/rfc-vtable-type-profiling-for-samplefdo/87283 Given a function below, the before vs after of a function's profile is illustrated in text format in the table:
Key points for this change:
We don't plan to add the vtable support to non-extensible format mainly because of the maintenance cost to keep backward compatibility for prior versions of profile data.
Patch is 30.51 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/148002.diff 11 Files Affected:
diff --git a/llvm/include/llvm/ProfileData/SampleProf.h b/llvm/include/llvm/ProfileData/SampleProf.h
index fb2d4d3cc50ed..d15b96db58fef 100644
--- a/llvm/include/llvm/ProfileData/SampleProf.h
+++ b/llvm/include/llvm/ProfileData/SampleProf.h
@@ -62,7 +62,8 @@ enum class sampleprof_error {
uncompress_failed,
zlib_unavailable,
hash_mismatch,
- illegal_line_offset
+ illegal_line_offset,
+ duplicate_vtable_type
};
inline std::error_code make_error_code(sampleprof_error E) {
@@ -91,6 +92,8 @@ struct is_error_code_enum<llvm::sampleprof_error> : std::true_type {};
namespace llvm {
namespace sampleprof {
+constexpr char kVTableProfPrefix[] = "vtables ";
+
enum SampleProfileFormat {
SPF_None = 0,
SPF_Text = 0x1,
@@ -204,6 +207,9 @@ enum class SecProfSummaryFlags : uint32_t {
/// SecFlagIsPreInlined means this profile contains ShouldBeInlined
/// contexts thus this is CS preinliner computed.
SecFlagIsPreInlined = (1 << 4),
+
+ /// SecFlagHasVTableTypeProf means this profile contains vtable type profiles.
+ SecFlagHasVTableTypeProf = (1 << 5),
};
enum class SecFuncMetadataFlags : uint32_t {
@@ -303,7 +309,7 @@ struct LineLocation {
}
uint64_t getHashCode() const {
- return ((uint64_t) Discriminator << 32) | LineOffset;
+ return ((uint64_t)Discriminator << 32) | LineOffset;
}
uint32_t LineOffset;
@@ -318,16 +324,28 @@ struct LineLocationHash {
LLVM_ABI raw_ostream &operator<<(raw_ostream &OS, const LineLocation &Loc);
+/// Key represents the id of a vtable and value represents its count.
+/// TODO: Rename class FunctionId to SymbolId in a separate PR.
+using TypeCountMap = std::map<FunctionId, uint64_t>;
+
+/// Write \p Map to the output stream. Keys are linearized using \p NameTable
+/// and written as ULEB128. Values are written as ULEB128 as well.
+std::error_code
+serializeTypeMap(const TypeCountMap &Map,
+ const MapVector<FunctionId, uint32_t> &NameTable,
+ raw_ostream &OS);
+
/// Representation of a single sample record.
///
/// A sample record is represented by a positive integer value, which
/// indicates how frequently was the associated line location executed.
///
/// Additionally, if the associated location contains a function call,
-/// the record will hold a list of all the possible called targets. For
-/// direct calls, this will be the exact function being invoked. For
-/// indirect calls (function pointers, virtual table dispatch), this
-/// will be a list of one or more functions.
+/// the record will hold a list of all the possible called targets and the types
+/// for virtual table dispatches. For direct calls, this will be the exact
+/// function being invoked. For indirect calls (function pointers, virtual table
+/// dispatch), this will be a list of one or more functions. For virtual table
+/// dispatches, this record will also hold the type of the object.
class SampleRecord {
public:
using CallTarget = std::pair<FunctionId, uint64_t>;
@@ -746,6 +764,7 @@ using BodySampleMap = std::map<LineLocation, SampleRecord>;
// memory, which is *very* significant for large profiles.
using FunctionSamplesMap = std::map<FunctionId, FunctionSamples>;
using CallsiteSampleMap = std::map<LineLocation, FunctionSamplesMap>;
+using CallsiteTypeMap = std::map<LineLocation, TypeCountMap>;
using LocToLocMap =
std::unordered_map<LineLocation, LineLocation, LineLocationHash>;
@@ -928,6 +947,14 @@ class FunctionSamples {
return &Iter->second;
}
+ /// Returns the TypeCountMap for inlined callsites at the given \p Loc.
+ const TypeCountMap *findCallsiteTypeSamplesAt(const LineLocation &Loc) const {
+ auto Iter = VirtualCallsiteTypeCounts.find(mapIRLocToProfileLoc(Loc));
+ if (Iter == VirtualCallsiteTypeCounts.end())
+ return nullptr;
+ return &Iter->second;
+ }
+
/// Returns a pointer to FunctionSamples at the given callsite location
/// \p Loc with callee \p CalleeName. If no callsite can be found, relax
/// the restriction to return the FunctionSamples at callsite location
@@ -989,6 +1016,42 @@ class FunctionSamples {
return CallsiteSamples;
}
+ /// Return all the callsite type samples collected in the body of the
+ /// function.
+ const CallsiteTypeMap &getCallsiteTypeCounts() const {
+ return VirtualCallsiteTypeCounts;
+ }
+
+ /// Returns the type samples for the un-drifted location of \p Loc.
+ TypeCountMap &getTypeSamplesAt(const LineLocation &Loc) {
+ return VirtualCallsiteTypeCounts[mapIRLocToProfileLoc(Loc)];
+ }
+
+ /// Scale \p Other sample counts by \p Weight and add the scaled result to the
+ /// type samples for the undrifted location of \p Loc.
+ template <typename T>
+ sampleprof_error addCallsiteVTableTypeProfAt(const LineLocation &Loc,
+ const T &Other,
+ uint64_t Weight = 1) {
+ static_assert((std::is_same_v<typename T::key_type, StringRef> ||
+ std::is_same_v<typename T::key_type, FunctionId>) &&
+ std::is_same_v<typename T::mapped_type, uint64_t>,
+ "T must be a map with StringRef or FunctionId as key and "
+ "uint64_t as value");
+ TypeCountMap &TypeCounts = getTypeSamplesAt(Loc);
+ bool Overflowed = false;
+
+ for (const auto [Type, Count] : Other) {
+ FunctionId TypeId(Type);
+ bool RowOverflow = false;
+ TypeCounts[TypeId] = SaturatingMultiplyAdd(
+ Count, Weight, TypeCounts[TypeId], &RowOverflow);
+ Overflowed |= RowOverflow;
+ }
+ return Overflowed ? sampleprof_error::counter_overflow
+ : sampleprof_error::success;
+ }
+
/// Return the maximum of sample counts in a function body. When SkipCallSite
/// is false, which is the default, the return count includes samples in the
/// inlined functions. When SkipCallSite is true, the return count only
@@ -1043,6 +1106,10 @@ class FunctionSamples {
mergeSampleProfErrors(Result,
FSMap[Rec.first].merge(Rec.second, Weight));
}
+ for (const auto &[Loc, OtherTypeMap] : Other.getCallsiteTypeCounts())
+ mergeSampleProfErrors(
+ Result, addCallsiteVTableTypeProfAt(Loc, OtherTypeMap, Weight));
+
return Result;
}
@@ -1286,6 +1353,23 @@ class FunctionSamples {
/// collected in the call to baz() at line offset 8.
CallsiteSampleMap CallsiteSamples;
+ /// Map virtual callsites to the vtable from which they are loaded.
+ ///
+ /// Each entry is a mapping from the location to the list of vtables and their
+ /// sampled counts. For example, given:
+ ///
+ /// void foo() {
+ /// ...
+ /// 5 inlined_vcall_bar();
+ /// ...
+ /// 5 inlined_vcall_baz();
+ /// ...
+ /// 200 inlined_vcall_qux();
+ /// }
+ /// This map will contain two entries. One with two types for line offset 5
+ /// and one with one type for line offset 200.
+ CallsiteTypeMap VirtualCallsiteTypeCounts;
+
/// IR to profile location map generated by stale profile matching.
///
/// Each entry is a mapping from the location on current build to the matched
diff --git a/llvm/include/llvm/ProfileData/SampleProfReader.h b/llvm/include/llvm/ProfileData/SampleProfReader.h
index bfe079fbe536f..fe8266247b2db 100644
--- a/llvm/include/llvm/ProfileData/SampleProfReader.h
+++ b/llvm/include/llvm/ProfileData/SampleProfReader.h
@@ -703,6 +703,14 @@ class LLVM_ABI SampleProfileReaderBinary : public SampleProfileReader {
/// otherwise same as readStringFromTable, also return its hash value.
ErrorOr<std::pair<SampleContext, uint64_t>> readSampleContextFromTable();
+ /// Read all virtual functions' vtable access counts for \p FProfile.
+ std::error_code readCallsiteVTableProf(FunctionSamples &FProfile);
+
+ /// Read bytes from the input buffer pointed by `Data` and decode them into
+ /// \p M. `Data` will be advanced to the end of the read bytes when this
+ /// function returns. Returns error if any.
+ std::error_code readVTableTypeCountMap(TypeCountMap &M);
+
/// Points to the current location in the buffer.
const uint8_t *Data = nullptr;
@@ -727,6 +735,10 @@ class LLVM_ABI SampleProfileReaderBinary : public SampleProfileReader {
/// to the start of MD5SampleContextTable.
const uint64_t *MD5SampleContextStart = nullptr;
+ /// If true, the profile has vtable profiles and reader should decode them
+ /// to parse profiles correctly.
+ bool ReadVTableProf = false;
+
private:
std::error_code readSummaryEntry(std::vector<ProfileSummaryEntry> &Entries);
virtual std::error_code verifySPMagic(uint64_t Magic) = 0;
diff --git a/llvm/include/llvm/ProfileData/SampleProfWriter.h b/llvm/include/llvm/ProfileData/SampleProfWriter.h
index e84b2095efd7b..9dbeaf56509b0 100644
--- a/llvm/include/llvm/ProfileData/SampleProfWriter.h
+++ b/llvm/include/llvm/ProfileData/SampleProfWriter.h
@@ -217,13 +217,20 @@ class LLVM_ABI SampleProfileWriterBinary : public SampleProfileWriter {
std::error_code writeBody(const FunctionSamples &S);
inline void stablizeNameTable(MapVector<FunctionId, uint32_t> &NameTable,
std::set<FunctionId> &V);
-
+
MapVector<FunctionId, uint32_t> NameTable;
-
+
void addName(FunctionId FName);
virtual void addContext(const SampleContext &Context);
void addNames(const FunctionSamples &S);
+ /// Write \p CallsiteTypeMap to the output stream \p OS.
+ std::error_code
+ writeCallsiteVTableProf(const CallsiteTypeMap &CallsiteTypeMap,
+ raw_ostream &OS);
+
+ bool WriteVTableProf = false;
+
private:
LLVM_ABI friend ErrorOr<std::unique_ptr<SampleProfileWriter>>
SampleProfileWriter::create(std::unique_ptr<raw_ostream> &OS,
@@ -412,8 +419,7 @@ class LLVM_ABI SampleProfileWriterExtBinaryBase
class LLVM_ABI SampleProfileWriterExtBinary
: public SampleProfileWriterExtBinaryBase {
public:
- SampleProfileWriterExtBinary(std::unique_ptr<raw_ostream> &OS)
- : SampleProfileWriterExtBinaryBase(OS) {}
+ SampleProfileWriterExtBinary(std::unique_ptr<raw_ostream> &OS);
private:
std::error_code writeDefaultLayout(const SampleProfileMap &ProfileMap);
diff --git a/llvm/lib/ProfileData/SampleProf.cpp b/llvm/lib/ProfileData/SampleProf.cpp
index 60c1393616713..f8706186454af 100644
--- a/llvm/lib/ProfileData/SampleProf.cpp
+++ b/llvm/lib/ProfileData/SampleProf.cpp
@@ -47,6 +47,24 @@ bool FunctionSamples::ProfileIsPreInlined = false;
bool FunctionSamples::UseMD5 = false;
bool FunctionSamples::HasUniqSuffix = true;
bool FunctionSamples::ProfileIsFS = false;
+
+std::error_code
+serializeTypeMap(const TypeCountMap &Map,
+ const MapVector<FunctionId, uint32_t> &NameTable,
+ raw_ostream &OS) {
+ encodeULEB128(Map.size(), OS);
+ for (const auto &[TypeName, SampleCount] : Map) {
+ if (auto NameIndexIter = NameTable.find(TypeName);
+ NameIndexIter != NameTable.end()) {
+ encodeULEB128(NameIndexIter->second, OS);
+ } else {
+ // If the type is not in the name table, we cannot serialize it.
+ return sampleprof_error::truncated_name_table;
+ }
+ encodeULEB128(SampleCount, OS);
+ }
+ return sampleprof_error::success;
+}
} // namespace sampleprof
} // namespace llvm
@@ -93,6 +111,8 @@ class SampleProfErrorCategoryType : public std::error_category {
return "Function hash mismatch";
case sampleprof_error::illegal_line_offset:
return "Illegal line offset in sample profile data";
+ case sampleprof_error::duplicate_vtable_type:
+ return "Duplicate vtable type in one map";
}
llvm_unreachable("A value of sampleprof_error has no message.");
}
@@ -126,6 +146,7 @@ sampleprof_error SampleRecord::merge(const SampleRecord &Other,
for (const auto &I : Other.getCallTargets()) {
mergeSampleProfErrors(Result, addCalledTarget(I.first, I.second, Weight));
}
+
return Result;
}
@@ -178,6 +199,17 @@ raw_ostream &llvm::sampleprof::operator<<(raw_ostream &OS,
return OS;
}
+static void printTypeCountMap(raw_ostream &OS, LineLocation Loc,
+ const TypeCountMap &TypeCountMap) {
+ if (TypeCountMap.empty()) {
+ return;
+ }
+ OS << Loc << ": vtables: ";
+ for (const auto &[Type, Count] : TypeCountMap)
+ OS << Type << ":" << Count << " ";
+ OS << "\n";
+}
+
/// Print the samples collected for a function on stream \p OS.
void FunctionSamples::print(raw_ostream &OS, unsigned Indent) const {
if (getFunctionHash())
@@ -192,7 +224,13 @@ void FunctionSamples::print(raw_ostream &OS, unsigned Indent) const {
SampleSorter<LineLocation, SampleRecord> SortedBodySamples(BodySamples);
for (const auto &SI : SortedBodySamples.get()) {
OS.indent(Indent + 2);
+ const auto &Loc = SI->first;
OS << SI->first << ": " << SI->second;
+ if (const TypeCountMap *TypeCountMap =
+ this->findCallsiteTypeSamplesAt(Loc)) {
+ OS.indent(Indent + 2);
+ printTypeCountMap(OS, Loc, *TypeCountMap);
+ }
}
OS.indent(Indent);
OS << "}\n";
@@ -214,6 +252,11 @@ void FunctionSamples::print(raw_ostream &OS, unsigned Indent) const {
OS << Loc << ": inlined callee: " << FuncSample.getFunction() << ": ";
FuncSample.print(OS, Indent + 4);
}
+ auto TypeSamplesIter = VirtualCallsiteTypeCounts.find(Loc);
+ if (TypeSamplesIter != VirtualCallsiteTypeCounts.end()) {
+ OS.indent(Indent + 2);
+ printTypeCountMap(OS, Loc, TypeSamplesIter->second);
+ }
}
OS.indent(Indent);
OS << "}\n";
diff --git a/llvm/lib/ProfileData/SampleProfReader.cpp b/llvm/lib/ProfileData/SampleProfReader.cpp
index cf7874041f3bf..b8b55f72935b5 100644
--- a/llvm/lib/ProfileData/SampleProfReader.cpp
+++ b/llvm/lib/ProfileData/SampleProfReader.cpp
@@ -197,8 +197,30 @@ enum class LineType {
CallSiteProfile,
BodyProfile,
Metadata,
+ VirtualCallTypeProfile,
};
+static bool parseTypeCountMap(StringRef Input,
+ DenseMap<StringRef, uint64_t> &TypeCountMap) {
+ for (size_t Index = Input.find_first_not_of(' '); Index != StringRef::npos;) {
+ size_t n1 = Input.find(':', Index);
+ if (n1 == StringRef::npos)
+ return false; // No colon found, invalid format.
+ StringRef TypeName = Input.substr(Index, n1 - Index);
+ // n2 is the start index of count.
+ size_t n2 = n1 + 1;
+ // n3 is the start index after the 'target:count' pair.
+ size_t n3 = Input.find_first_of(' ', n2);
+ uint64_t Count;
+ if (Input.substr(n2, n3 - n2).getAsInteger(10, Count))
+ return false; // Invalid count.
+ TypeCountMap[TypeName] = Count;
+ Index = (n3 == StringRef::npos) ? StringRef::npos
+ : Input.find_first_not_of(' ', n3);
+ }
+ return true;
+}
+
/// Parse \p Input as line sample.
///
/// \param Input input line.
@@ -215,6 +237,7 @@ static bool ParseLine(const StringRef &Input, LineType &LineTy, uint32_t &Depth,
uint64_t &NumSamples, uint32_t &LineOffset,
uint32_t &Discriminator, StringRef &CalleeName,
DenseMap<StringRef, uint64_t> &TargetCountMap,
+ DenseMap<StringRef, uint64_t> &TypeCountMap,
uint64_t &FunctionHash, uint32_t &Attributes,
bool &IsFlat) {
for (Depth = 0; Input[Depth] == ' '; Depth++)
@@ -289,6 +312,7 @@ static bool ParseLine(const StringRef &Input, LineType &LineTy, uint32_t &Depth,
n4 = AfterColon.find_first_of(' ');
n4 = (n4 != StringRef::npos) ? n3 + n4 + 1 : Rest.size();
StringRef WordAfterColon = Rest.substr(n3 + 1, n4 - n3 - 1);
+ // Break the loop if parsing integer succeeded.
if (!WordAfterColon.getAsInteger(10, count))
break;
@@ -306,6 +330,10 @@ static bool ParseLine(const StringRef &Input, LineType &LineTy, uint32_t &Depth,
// Change n3 to the next blank space after colon + integer pair.
n3 = n4;
}
+ } else if (Rest.starts_with(kVTableProfPrefix)) {
+ LineTy = LineType::VirtualCallTypeProfile;
+ return parseTypeCountMap(Rest.substr(strlen(kVTableProfPrefix)),
+ TypeCountMap);
} else {
LineTy = LineType::CallSiteProfile;
size_t n3 = Rest.find_last_of(':');
@@ -374,14 +402,15 @@ std::error_code SampleProfileReaderText::readImpl() {
uint64_t NumSamples;
StringRef FName;
DenseMap<StringRef, uint64_t> TargetCountMap;
+ DenseMap<StringRef, uint64_t> TypeCountMap;
uint32_t Depth, LineOffset, Discriminator;
LineType LineTy;
uint64_t FunctionHash = 0;
uint32_t Attributes = 0;
bool IsFlat = false;
if (!ParseLine(*LineIt, LineTy, Depth, NumSamples, LineOffset,
- Discriminator, FName, TargetCountMap, FunctionHash,
- Attributes, IsFlat)) {
+ Discriminator, FName, TargetCountMap, TypeCountMap,
+ FunctionHash, Attributes, IsFlat)) {
reportError(LineIt.line_number(),
"Expected 'NUM[.NUM]: NUM[ mangled_name:NUM]*', found " +
*LineIt);
@@ -410,6 +439,14 @@ std::error_code SampleProfileReaderText::readImpl() {
DepthMetadata = 0;
break;
}
+
+ case LineType::VirtualCallTypeProfile: {
+ mergeSampleProfErrors(
+ Result, InlineStack.back()->addCallsiteVTableTypeProfAt(
+ LineLocation(LineOffset, Discriminator), TypeCountMap));
+ break;
+ }
+
case LineType::BodyProfile: {
FunctionSamples &FProfile = *InlineStack.back();
for (const auto &name_count : TargetCountMap) {
@@ -591,6 +628,59 @@ SampleProfileReaderBinary::readSampleContextFromTable() {
return std::make_pair(Context, Hash);
}
+std::error_code
+SampleProfileReaderBinary::readVTableTypeCountMap(TypeCountMap &M) {
+ auto NumVTableTypes = readNumber<uint32_t>();
+ if (std::error_code EC = NumVTableTypes.getError())
+ return EC;
+
+ for (uint32_t I = 0; I < *NumVTableTypes; ++I) {
+ auto VTableType(readStringFromTable());
+ if (std::error_code EC = VTableType.getError())
+ return EC;
+
+ auto VTableSamples = readNumber<uint64_t>();
+ if (std::error_code EC = VTableSamples.getError())
+ return EC;
+
+ if (!M.insert(std::make_pair(*VTableType, *VTableSamples)).second)
+ return sampleprof_error::duplicate_vtable_type;
+ }
+ return sampleprof_error::success;
+}
+
+std::error_code
+SampleProfileReaderBinary::readCallsiteVTableProf(FunctionSamples &FProfile) {
+ if (!ReadVTableProf)
+ return sampleprof_error::success;
+
+ // Read the vtable type profile for the callsite.
+ auto NumCallsites = readNumber<uint32_t>();
+ if (std::error_code EC = NumCallsites.getError())
+ return EC;
+
+ for (uint32_t I = 0; I < *NumCallsites; ++I) {
+ auto LineOffset = readNumber<uint64_t>();
+ if (std::error_code EC = LineOffset.getError())
+ return EC;
+
+ if (!isOffsetLegal(*LineOffset))
+ return sampleprof_error::illegal_line_offset;
+
+ auto Discriminator = readNumber<uint64_t>();
+ if (std::error_code EC = Discriminator.getError())
+ return EC;
+
+ // Here we handle FS discriminators:
+ const uint32_t DiscriminatorVal = (*Discriminator) & getDiscriminatorMask();
+
+ if (std::error_code EC = readVTableTypeCountMap(FProfile.getTypeSamplesAt(
+ LineLocation(*LineOffset, DiscriminatorVal))))
+ return EC;
+ }
+ return sampleprof_error::success;
+}
+
std::error_code
SampleProfileReaderBinary::readProfile(FunctionSamples &FProfile) {
auto NumSamples = readNumber<uint64_t>();
@@ -671,7 +761,7 @@ SampleProfileReaderBinary::readProfile(FunctionSamples &FProfile) {
return EC;
}
- return sampleprof_error::success;
+ return readCallsiteVTableProf(FProfile);
}
std::error_code
@@ -733,6 +823,8 @@ std::error_code SampleProfileReaderExtBinaryBase::readOneSection(
FunctionSamples::ProfileIsPreInlined = ProfileIsPreInlined = true;
if (hasSecFlag(Entry, SecProfSummaryFlags::SecFlagFSDiscriminator))
FunctionSamples::ProfileIsFS = ProfileIsFS = true;
+ if (hasSecFlag(Entry, SecProfSummaryFlags::SecFlagHasVTableTypeProf)...
[truncated]
|
david-xl
reviewed
Jul 11, 2025
| @@ -318,16 +324,28 @@ struct LineLocationHash { | |||
|
|
|||
| LLVM_ABI raw_ostream &operator<<(raw_ostream &OS, const LineLocation &Loc); | |||
|
|
|||
| /// Key represents the id of a vtable and value represents its count. | |||
| /// TODO: Rename class FunctionId to SymbolId in a separate PR. | |||
| using TypeCountMap = std::map<FunctionId, uint64_t>; | |||
There was a problem hiding this comment.
Add document on FunctionId/SymbolId which refers to vtable symbol. 'Type' refers to C++ polymorphic class types.
| @@ -989,6 +1016,42 @@ class FunctionSamples { | |||
| return CallsiteSamples; | |||
| } | |||
|
|
|||
| /// Return all the callsite type samples collected in the body of the | |||
There was a problem hiding this comment.
nit : return vtable access samples for C++ types collected ..
| return VirtualCallsiteTypeCounts; | ||
| } | ||
|
|
||
| /// Returns the type samples for the un-drifted location of \p Loc. |
| @@ -1286,6 +1353,23 @@ class FunctionSamples { | |||
| /// collected in the call to baz() at line offset 8. | |||
| CallsiteSampleMap CallsiteSamples; | |||
|
|
|||
| /// Map virtual callsites to the vtable from which they are loaded. | |||
There was a problem hiding this comment.
Nit: map a vcall site to the list of accessed vtables by the site. The vcallsite is referenced by its source location, and ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change extends SampleFDO ext-binary and text format to record the vtable symbols and their counts for virtual calls inside a function. The vtable profiles will allow the compiler to annotate vtable types on IR instructions and perform vtable-based indirect call promotion. An RFC is in https://discourse.llvm.org/t/rfc-vtable-type-profiling-for-samplefdo/87283
Given a function below, the before vs after of a function's profile is illustrated in text format in the table:
0: 636241
1: 681458, calls: _Z10createTypei:681458
3: 543499, calls: _ZN12_GLOBAL__N_18Derived24funcEii:410621 _ZN8Derived14funcEii:132878
5.1: 602201, calls: _ZN12_GLOBAL__N_18Derived2D0Ev:454635 _ZN8Derived1D0Ev:147566
7: 511057
}
0: 636241
1: 681458, calls: _Z10createTypei:681458
3: 543499, calls: _ZN12_GLOBAL__N_18Derived24funcEii:410621 _ZN8Derived14funcEii:132878
3: vtables: _ZTV8Derived1:1377 _ZTVN12_GLOBAL__N_18Derived2E:4250
5.1: 602201, calls: _ZN12_GLOBAL__N_18Derived2D0Ev:454635 _ZN8Derived1D0Ev:147566
5.1: vtables: _ZTV8Derived1:227 _ZTVN12_GLOBAL__N_18Derived2E:765
7: 511057
}
Key points for this change:
map<LineLocation, map<FunctionId, uint64_t>>is introduced in a function's in-memory representation FunctionSamples.SecProfSummaryFlags. The ext-binary profile reader parses the integer type flag and reads this bit. If it's set, the profile reader will parse vtable profiles.-extbinary-write-vtable-type-prof). The ext-binary profile writer reads the boolean option and decide whether to set the section flag bit and serialize the in-memory class members corresponding to vtables.llvm-profdata overlap --samplefor the vtable profiles. A subsequent change will do it to keep this one focused on the profile format change.We don't plan to add the vtable support to non-extensible format mainly because of the maintenance cost to keep backward compatibility for prior versions of profile data.