bug 1827748: feature: opm (index|registry) prune command #243

djzager · 2020-03-31T07:06:12Z

Description of the change:

This makes it possible for a user to take an index image or a database
referencing a large collection of operator bundles to filter down the
collection.

Motivation for the change:

Make it possible to scale down operator collections.

Reviewer Checklist

Implementation matches the proposed design, or proposal is updated to match implementation
Sufficient unit test coverage
Sufficient end-to-end test coverage
Docs updated or added to /docs
Commit messages sensible and descriptive

openshift-ci-robot · 2020-03-31T07:06:21Z

Hi @djzager. Thanks for your PR.

I'm waiting for a operator-framework member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

mhrivnak

Great start. I do think that in order to be useful, it needs the ability to specify the channel with a package. The input could be a file of such pairs as CSV (comma-separated-values in this case), or this command could take positional args on the command line where the package and channel have some designated separator (maybe still a comma).

mhrivnak · 2020-03-31T11:38:18Z

cmd/opm/index/filter.go

+	if err := indexCmd.MarkFlagRequired("from-index"); err != nil {
+		logrus.Panic("Failed to set required `from-index` flag for `index filter`")
+	}
+	indexCmd.Flags().StringSliceP("operators", "o", nil, "comma separated list of operators to filter")


Maybe "packages" would work well here. That would especially prevent confusion around -o being so heavily associated with output in k8s and other contexts.

Though probably the best option is to replace this with a way to specify a list of package:channel pairs.

I agree that "packages" may be the least confusing. The problem for me is that I took these flags off of cmd/opm/index/delete.go and so I'm hesitant to 1) mess with what already exists 2) change flags just for the filter command.

ecordell · 2020-03-31T15:51:24Z

/ok-to-test

cmd/opm/registry/rm.go

djzager · 2020-04-16T15:55:00Z

/retest

djzager · 2020-04-16T17:56:09Z

/retest

djzager · 2020-04-16T19:14:43Z

/retest

rwsu

lgtm. There is a more SQL way to query which packages to delete from sqlite that might result in cleaner code. But not a biggie.

rwsu · 2020-04-16T22:07:12Z

pkg/lib/registry/registry.go

+
+	// get all the packages
+	lister := sqlite.NewSQLLiteQuerierFromDb(db)
+	packages, err := lister.ListPackages(context.TODO())


The other way to do this would be to write a query to select all packages with a NOT IN in place of selecting all packages. Then convert this list of package names into a comma separated string that can used in a single call tosqlite.NewSQLRemoverForPackages(dbLoader, pkg).

djzager · 2020-04-17T01:05:49Z

/assign @ecordell @njhale

djzager · 2020-04-21T12:22:24Z

/hold

I broke something. Will fix and add appropriate testcase.

This makes it possible for a user to prune an index image or an operator database referencing a large collection of operator bundles.

djzager · 2020-04-21T19:04:39Z

/hold cancel

djzager · 2020-04-21T19:34:39Z

/retest

ecordell · 2020-04-24T02:38:33Z

/lgtm

Thanks for the wait @djzager!

This looks good modulo some refactoring that we need to do across the entire indexer.

Just so you know, we have to either file a BZ for this or wait for master to open (because we currently build upstream and downstream from this repo).

openshift-ci-robot · 2020-04-24T17:30:04Z

@djzager: This pull request references Bugzilla bug 1827748, which is invalid:

expected the bug to target the "4.5.0" release, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

bug 1827748: feature: opm (index|registry) prune command

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

djzager · 2020-04-24T17:30:15Z

/bugzilla refresh

openshift-ci-robot · 2020-04-24T17:30:17Z

@djzager: This pull request references Bugzilla bug 1827748, which is invalid:

expected the bug to target the "4.5.0" release, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

jmrodri · 2020-04-24T17:40:37Z

/bugzilla refresh

openshift-ci-robot · 2020-04-24T17:40:43Z

@jmrodri: This pull request references Bugzilla bug 1827748, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.5.0) matches configured target release for branch (4.5.0)
bug is in the state NEW, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

ecordell · 2020-04-25T11:14:54Z

/approve

openshift-ci-robot · 2020-04-25T11:15:54Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: djzager, ecordell, mhrivnak

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [ecordell]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot · 2020-04-25T12:42:42Z

@djzager: All pull requests linked via external trackers have merged: operator-framework/operator-registry#243. Bugzilla bug 1827748 has been moved to the MODIFIED state.

In response to this:

bug 1827748: feature: opm (index|registry) prune command

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot requested review from anik120 and awgreene March 31, 2020 07:06

openshift-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Mar 31, 2020

mhrivnak reviewed Mar 31, 2020

View reviewed changes

openshift-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Mar 31, 2020

djzager force-pushed the filtercmd branch from 51dca19 to 02fc177 Compare April 3, 2020 19:28

djzager commented Apr 3, 2020

View reviewed changes

cmd/opm/registry/rm.go Outdated Show resolved Hide resolved

djzager mentioned this pull request Apr 3, 2020

"packages" flag to opm registry rm should be required #250

Closed

djzager force-pushed the filtercmd branch from 02fc177 to 9086eb2 Compare April 3, 2020 20:06

djzager mentioned this pull request Apr 6, 2020

feat: opm package pruning operator-framework/enhancements#15

Merged

djzager force-pushed the filtercmd branch from 9086eb2 to 6ce438a Compare April 13, 2020 21:01

djzager changed the title ~~feature: opm (index|registry) filter command~~ feature: opm (index|registry) prune command Apr 16, 2020

djzager force-pushed the filtercmd branch 2 times, most recently from a28056d to f193cfc Compare April 16, 2020 13:14

mhrivnak approved these changes Apr 16, 2020

View reviewed changes

djzager force-pushed the filtercmd branch from f193cfc to 14b53aa Compare April 16, 2020 16:46

djzager force-pushed the filtercmd branch from 14b53aa to 5233ba5 Compare April 16, 2020 21:21

rwsu reviewed Apr 16, 2020

View reviewed changes

djzager force-pushed the filtercmd branch from 5233ba5 to 6194b66 Compare April 17, 2020 01:04

openshift-ci-robot assigned ecordell and njhale Apr 17, 2020

djzager force-pushed the filtercmd branch 2 times, most recently from 5ec19da to 5876c1d Compare April 17, 2020 01:50

openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 21, 2020

djzager force-pushed the filtercmd branch from 7fbd549 to 209c402 Compare April 21, 2020 15:15

openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 21, 2020

djzager force-pushed the filtercmd branch from 209c402 to b2043ab Compare April 21, 2020 19:02

openshift-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 21, 2020

feature: opm (index|registry) prune command

ea6c6a4

This makes it possible for a user to prune an index image or an operator database referencing a large collection of operator bundles.

djzager force-pushed the filtercmd branch from b2043ab to ea6c6a4 Compare April 21, 2020 19:04

openshift-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 21, 2020

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Apr 24, 2020

djzager changed the title ~~feature: opm (index|registry) prune command~~ bug 1827748: feature: opm (index|registry) prune command Apr 24, 2020

openshift-ci-robot added bugzilla/low bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels Apr 24, 2020

openshift-ci-robot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Apr 24, 2020

openshift-ci-robot removed the bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. label Apr 24, 2020

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 25, 2020

openshift-merge-robot merged commit 6e6a14d into operator-framework:master Apr 25, 2020

dmesser mentioned this pull request May 18, 2020

RFE: mirror an index subset based on Subscriptions #203

Closed

cdjohnson mentioned this pull request Jan 21, 2021

Enhancement for opm content management features operator-framework/enhancements#26

Open

bug 1827748: feature: opm (index|registry) prune command #243

bug 1827748: feature: opm (index|registry) prune command #243

Uh oh!

Conversation

djzager commented Mar 31, 2020

Uh oh!

openshift-ci-robot commented Mar 31, 2020

Uh oh!

mhrivnak left a comment

Choose a reason for hiding this comment

Uh oh!

mhrivnak Mar 31, 2020

Choose a reason for hiding this comment

Uh oh!

djzager Apr 3, 2020

Choose a reason for hiding this comment

Uh oh!

ecordell commented Mar 31, 2020

Uh oh!

Uh oh!

djzager commented Apr 16, 2020

Uh oh!

djzager commented Apr 16, 2020

Uh oh!

djzager commented Apr 16, 2020

Uh oh!

rwsu left a comment

Choose a reason for hiding this comment

Uh oh!

rwsu Apr 16, 2020

Choose a reason for hiding this comment

Uh oh!

djzager commented Apr 17, 2020

Uh oh!

djzager commented Apr 21, 2020

Uh oh!

djzager commented Apr 21, 2020

Uh oh!

djzager commented Apr 21, 2020

Uh oh!

ecordell commented Apr 24, 2020

Uh oh!

openshift-ci-robot commented Apr 24, 2020

Uh oh!

djzager commented Apr 24, 2020

Uh oh!

openshift-ci-robot commented Apr 24, 2020

Uh oh!

jmrodri commented Apr 24, 2020

Uh oh!

openshift-ci-robot commented Apr 24, 2020

Uh oh!

ecordell commented Apr 25, 2020

Uh oh!

openshift-ci-robot commented Apr 25, 2020

Uh oh!

openshift-ci-robot commented Apr 25, 2020

Uh oh!

Uh oh!