Async FilesystemStore #3931
Async FilesystemStore #3931
Conversation
Also provide a wrapper to allow a sync kvstore to be used.
|
👋 Hi! I see this is a draft PR. |
joostjager
force-pushed
the
async-fsstore
branch
4 times, most recently
from
29b8bcf to
81ad668
Compare
| let this = Arc::clone(&self.inner); | ||
|
|
||
| Box::pin(async move { | ||
| tokio::task::spawn_blocking(move || { |
There was a problem hiding this comment.
Mhh, so I'm not sure if spawning blocking tasks for every IO call is the way to go (see for example https://docs.rs/tokio/latest/tokio/fs/index.html#tuning-your-file-io: "To get good performance with file IO on Tokio, it is recommended to batch your operations into as few spawn_blocking calls as possible."). Maybe there are other designs that we should at least consider before moving forward with this approach. For example, we could create a dedicated pool of longer-lived worker task(s) that process a queue?
If we use spawn_blocking, can we give the user control over which runtime this exactly will be spawned on? Also, rather than just doing wrapping, should we be using tokio::fs?
There was a problem hiding this comment.
Mhh, so I'm not sure if spawning blocking tasks for every IO call is the way to go (see for example https://docs.rs/tokio/latest/tokio/fs/index.html#tuning-your-file-io: "To get good performance with file IO on Tokio, it is recommended to batch your operations into as few spawn_blocking calls as possible.").
If we should batch operations, I think the current approach is better than using tokio::fs? Because it already batches the various operations inside kvstoresync::write.
Further batching probably needs to happen at a higher level in LDK, and might be a bigger change. Not sure if that is worth it just for FIlesystemStore, especially when that store is not the preferred store for real world usage?
For example, we could create a dedicated pool of longer-lived worker task(s) that process a queue?
Isn't Tokio doing that already when a task is spawned?
If we use spawn_blocking, can we give the user control over which runtime this exactly will be spawned on? Also, rather than just doing wrapping, should we be using tokio::fs?
With tokio::fs, the current runtime is used. I'd think that that is then also sufficient if we spawn ourselves, without a need to specifiy which runtime exactly?
More generally, I think the main purpose of this PR is to show how an async kvstore could be implemented, and to have something for testing potentially. Additionally if there are users that really want to use this type of store in production, they could. But I don't think it is something to spend too much time on. A remote database is probably the more important target to design for.
| } | ||
|
|
||
| /// Provides additional interface methods that are required for [`KVStore`]-to-[`KVStore`] | ||
| /// data migration. | ||
| pub trait MigratableKVStore: KVStore { | ||
| pub trait MigratableKVStore: KVStoreSync { |
There was a problem hiding this comment.
How will we solve this for an KVStore?
There was a problem hiding this comment.
I think this comment belongs in #3905?
We might not need to solve it now, as long as we still require a sync implementation alongside an async one? If we support async-only kvstores, then we can create an async version of this trait?
lightning-persister/src/fs_store.rs
Outdated
|
|
||
| // Per path lock that ensures that we don't have concurrent writes to the same file. The lock also encapsulates the | ||
| // latest written version per key. | ||
| locks: Mutex<HashMap<PathBuf, Arc<RwLock<HashMap<String, u64>>>>>, |
There was a problem hiding this comment.
What is the HashMap-in-the-HashMap for? dest_file_path is already the 'full path' containing the key, so no reason to add yet another key-indexed versioning structure in there?
There was a problem hiding this comment.
Oh missed that push of the key to the path. Yes, then map-in-map is unnecessary. Fixed.
joostjager
force-pushed
the
async-fsstore
branch
from
81ad668 to
e462bce
Compare
Need to keep the last written version
|
Removed garbage collector, because we need to keep the last written version. |
Try out for an async store with eventually consistent writes. It is just using tokio's
spawn_blocking, because that is whattokio::fswould otherwise do as well. Usingtokio::fswould make it complicated to reuse the sync code.Builds on #3905