Skip to content

git2 feature: Custom Odb backends #1180

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 23 commits into
base: master
Choose a base branch
from

Conversation

tecc
Copy link
Contributor

@tecc tecc commented Aug 17, 2025

This PR is largely an attempt at making a "safe" API for adding custom ODB backends.
Additions made for this purpose have been concentrated in src/odb_backend.rs, with minor modifications to src/odb.rs and src/lib.rs to make it accessible.

A disclaimer, however: custom ODB backends lack documentation in libgit2, so most of the code and documentation (of which there is plenty) is based upon what I think the code does; if someone else knows better, feel free to tell me and I'll fix it.

I also haven't verified correctness nor safety, although I suspect the code does not violate all too many of Rust's memory rules.

The code has however been documented as thoroughly as I could be bothered to. It's not guaranteed to be correct, because libgit2 lacks documentation, but it should be largely correct as I cross-referenced with both libgit2 and libgit2sharp's source code and documentation to what extent I could.

Usage

This is largely untested (as of writing), so I may have made mistakes here and there.

  1. Create a type that contains all the information you want from your backend (let's call it YourBackend).
  2. Implement git2::odb_backend::OdbBackend trait for YourBackend. The only required method is fn supported_operations(&self) -> SupportedOperations - we'll come back to it.
  3. Implement the methods you want your backend to support. All methods come with a great deal of implementation notes and recommendations.
  4. In the supported_operations method, add a flag for each method your backend supports. The appropriate flags are described in each method's documentation.
  5. To add YourBackend as a backend to an odb: Odb, use odb.add_custom_backend(your_backend, priority).
  6. If you want to access YourBackend, you can do so with the CustomOdbBackend handle that was returned in the previous call.

TODO

There may also be some TODOs littered throughout the code.

  • Implement the readstream and writestream methods
  • Implement the foreach method
  • Add more implementation notes; the documentation is currently very lacking.
  • Create an example backend using this system. Perhaps try to reimplement one of libgit2's builtin backends?
  • Combine duplicated bindings (most notably odb_backend::Indexer to indexer::Indexer and odb_backend::IndexerProgress to indexer::Progress)
  • Reconsider the naming of some of the types and methods before merging

@tecc tecc changed the title git2 feat: Custom Odb backends git2 feature: Custom Odb backends Aug 17, 2025
tecc added 23 commits August 17, 2025 19:12
Added type definition for struct `libgit2-sys::git_odb_stream`.
Previously incorrectly defined as an opaque enum.
See git2/odb_backend.h line 196.

Added type definition for enum `git_odb_stream_t`.
I believe it corresponds to the `mode` field of `git_odb_stream`, but
that's just a guess.
See git2/odb_backend.h line 182.
Added struct `git_config_backend_entry`.
See git2/sys/config.h line 27.

Added struct `git_config_iterator`.
Previously incorrectly defined as an empty enum.
See git2/sys/config.h line 49.

Added struct `git_config_backend`.
See git2/sys/config.h line 69.

Added constant `GIT_CONFIG_BACKEND_VERSION`.
See git2/sys/config.h line 103.

Added struct `git_config_backend_memory_options`.
See git2/sys/config.h line 148.

Added constant `GIT_CONFIG_BACKEND_MEMORY_OPTIONS_VERSION`.
See git2/sys/config.h line 165.

Added function `git_config_add_backend`.
See git2/sys/config.h line 140.

Added function `git_config_backend_from_string`.
See git2/sys/config.h line 181.

Added function `git_config_backend_from_values`.
See git2/sys/config.h line 197.

Added function `git_config_init_backend`.
See git2/sys/config.h line 116.
Added struct `git_reference_iterator`.
See git2/sys/refdb_backend.h line 35.
`git_commit_nth_gen_ancestor`'s first argument has been changed to
`ancestor` from the previous `commit` to bring it closer to the actual
definition (git2/commit.h line 282) and because the previous name is
also used for the second argument.
`CustomOdbBackend` does not drop the inner `Backend` value, leaving it
to libgit2 calling the backend's `free` function.
This function cannot be called if the `git_odb_add_backend` call fails.

Now, `Odb::add_custom_backend` first creates the inner `Backend` value
and only when the value has successfully been passed to libgit2 do we
stop managing the memory.
Some methods don't necessarily return just `git_error_code`.
Added more documentation to clarify how to do error handling properly.

Clarified the default behaviour of `OdbBackend::exists`.
Added `OdbBackend::write_multipack_index`, corresponding to the
`writemidx` function of `git_odb_backend`.

Fix reference to `Odb` in `OdbBackend` documentation.
This is mostly to bring it inline with the other modules' code style.
Added `OdbBackend::open_writepack`, for opening streams that write
packfiles.

Added `OdbWritepack` trait for custom Writepack implementations.

Added an associated type `OdbBackend::Writepack: OdbWritepack` that
tells git2 what data to include for writepack implementations.

Implemented `OdbWritepack` for `Infallible` so that people can opt out
of implementing the Writepack trait.
The implementation panics when any method is called, though that would
be undefined behaviour since it would require `Infallible` to have been
instantiated.

A lot of types were duplicated in the process of this commit. I want
other people's thoughts on how to merge them before I make any changes
to their code, so I'll open a draft PR soon.
@tecc tecc force-pushed the feat/custom-odb-backend branch from 5fa4485 to 5e4fb91 Compare August 18, 2025 12:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant