Govern OpenCollective using SourceCred

[Shauna]

Title: Govern OpenCollective using SourceCred

Use Case Overview

OpenFoobar is an open source org that’s using open collective to raise money to support the development of their software. Currently the founder of the project, Anne, has total control over which invoices get approved, aka who money contributed to the project goes to. She wants to let the community decide, but only community members who are genuinely invested in the project, not random people or total newcomers.

She decides to use SourceCred to determine who should be given decision-making power in the community, and picks a number of metrics, like PRs contributed and props received, that seem reasonable to her. Now she needs to connect SourceCred to OpenCollective.

Example User Flows

Flow 1

Trigger: a user requests to join the group of people who decide on approving invoices

Preconditions:

• the OpenFoobar group has already been set up on OpenCollective, SourceCred and Metagov Prototype, and the accounts connected

Steps:

1. a user, Bose, requests on PolicyKit to be added to the community (not sure how roles work in PolicyKit, but if there’s the concept of ‘people with role X can vote in a given situation’, then say they request to be given role X)
2. the OpenFoobar community on PolicyKit has a policy saying that anyone with SourceCred above N can have that role/join that community
3. the PolicyKit community queries the Metagov Prototype asking “what is the cred of Bose in our corresponding SourceCred community?” MetaGov queries the appropriate community and returns the answer to PolicyKit
4. given that information, PolicyKit is able to reject or accept Bose’s request

Flow 2

Trigger: a user submits an invoice to OpenCollective

Preconditions

• the OpenFoobar group has already been set up on OpenCollective, SourceCred and Metagov Prototype, and the accounts connected
• at least one person has gotten enough “cred” in SourceCred to qualify
• there is existing money in OpenCollective to be distributed

Steps

1. a user submits an invoice to OpenCollective
2. OpenColletive notifies the Metagov Prototype
3. The Metagov Prototpe notifies PolicyKit
4. PolicyKit hosts a vote on whether to approve the invoice
5. PolicyKit notifies MetaGov of the outcome of the vote
6. Metagov Prototype messages OpenCollective to approve or reject the invoice

Thinking through the above steps as an engineer because I can’t help myself

1. a user, Clarisa, submits an invoice to OpenCollective
2. OpenCollective knows somehow to check the Metagov Prototype instead of just asking Anne (so this will definitely require some changes by OpenCollective). It sends a notification to the Metagov Prototype server that an invoice has been submitted. (Alternatively, it might use a Metagov Protocol to store locally the information about which governance platform to ask for approval, but for now let’s assume a server.)
3. Either the Metagov server matches all actions from the OpenCollective OpenFoobar group to their corresponding PolicyKit community, or perhaps it matches the action ‘approve invoices’ specifically, but in either case it sends a message to a PolicyKit server saying essentially, “An invoice has been submitted, is it approved?”
4. The PolicyKit community has a policy which says anyone with role X in the community can vote on the invoice, so it starts a vote of those people. When the vote is complete, it sends a message either to OpenCollective directly or via the Metagov prototype saying Clarisa’s request is approved or not approved.

[miriam]

Thanks @Shauna, this is great!

@Shauna

PolicyKit hosts a vote on whether to approve the invoice

My guess is that this vote would occur on the OpenCollective discussion forum. Is that what you had in mind? One of the pre-req’s is that PolicyKit has an integration for Open Collective.

[miriam]

@Shauna

OpenCollective knows somehow to check the Metagov Prototype instead of just asking Anne (so this will definitely require some changes by OpenCollective).

I can see this use-case going two ways:

1. The system enforces the policy, meaning Anne is blocked from approving the invoice. The invoice is approved or rejected programmatically when the vote is closed.
2. The system does not enforce the policy. Anne can approve/reject the invoice at any time. Step 6, (“Metagov Prototype messages OpenCollective to approve or reject the invoice”), takes the form of a post on the forum or a direct message to Anne. It’s up to Anne to make the final approval/rejection.

I think this use case as posted is describing (1), but as you point out it may require big changes to OpenCollective, & API access to do things as serious as approve/reject invoices. Option (2) may be accomplished with less access to OpenCollective (we “only” need webhooks for invoices, and API access to post on a forum and see reactions –– this would still require changes to OpenCollective, but is more realistic).

I think this is a good example of an issue that Metagov will probably face across many platforms and libraries, due to lack of API access. My feeling is that supporting a use-case like (2) would still be useful and worthwhile. What do you think?

[Shauna]

My guess is that this vote would occur on the OpenCollective discussion forum. Is that what you had in mind? One of the pre-req’s is that PolicyKit has an integration for Open Collective.

Can you say a bit more about how you see that happening? How would PolicyKit “translate” the discussion on the forum into a vote? Would it need to parse the syntax of posts in specific ways (ie “if user X writes ‘%%final vote: Y%%’ count that as a yes vote on Y”)?

[Shauna]

I think this is a good example of an issue that Metagov will probably face across many platforms and libraries, due to lack of API access. My feeling is that supporting a use-case like (2) would still be useful and worthwhile. What do you think?

I strongly agree we should support both use case types in the long term, although I guess it’s an open question which type we should focus on for the prototype.

(It might be good to adopt terminology for this. Enforcing vs Non-Enforcing seems pretty descriptive.)

[miriam]

The way that PolicyKit currently conducts votes is by creating a post as the PolicyKit bot-user, and monitoring the responses. For the Slack integration, it counts and reactions to the post. For the Reddit integration, it looks for replies containing “-1” or “+1”. OpenCollective forums support emoji reactions, so I see it happening in a way similar to Slack, but it could support string matching or something else.

[Shauna]

Got it - I didn’t realize that’s how PolicyKit integrated with Slack and Reddit, but it makes perfect sense.

To what extent can we abstract a pattern here? Can we expect all governance “drivers” to:

• be able to create an account on the platform they’re integrating with (or perhaps this should happen through the metagov prototype somehow?)
• conduct decision-making processes by starting some sort of thread or making a post with a specific format (this might only work with discussion-based platforms)
• parse user responses in a predictable but potentially customizable way (ie by default thumbs up is yes and thumbs down is no, but if you want to say surprise face emoji means wait, you can configure that; or alternatively saying 'by default we look for +1 to mean X and -1 to mean Y but you can configure what means X and Y)

[miriam]

I like the idea of abstracting a pattern for governance “drivers.” As I understand it, the thing that differentiates a “driver” from any other module is that it is capable of enforcing some governance policy. I would make it more broad, and say that drivers can:

1. “Listen” for actions happening on- or off-platform*.
2. Initiate some decision-making process to determine what to do next (for example, initiating a vote)
3. Initiate an on- or off-platform action based on the result of (2).

*“platform” being the main underlying platform being governed. In this use case it’s OpenCollective. For the first prototype, it’s Discourse

Steps 1 and 3 probably require authenticated API access, and ability to message users on-platform.

In my mind, the “driver” doesn’t need to be capable of conducting the decision-making process itself. For step 2 it could, for example, offload the decision-making to some “voting module” (backed by Loomio, for example). I feel like this is the core thing that will make the whole system expressive/modular. Without it, we would need a separate driver for every kind of decision-making engine. The driver seems like quite a heavy-weight system (listening to all platform actions!), so my feeling is that there should only be 1 driver per metagov instance. We may use PolicyKit as the “driver” for this prototype, but I don’t think the “driver” abstraction (if we go with that) should be so tightly coupled to the behavior the PolicyKit happens to support.

I can apply PolicyKit to this description of a “driver,” and it fits but with some heavy restrictions:

1. The trigger action must happen on the platform.
2. The “decision” is limited to a yes/no decision (to move on to “success” or “fail” step in PK). The decision-making happens by making a post and monitoring user reactions/responses.
3. PolicyKit seems to be designed with the intention that the resulting action is always the same as the trigger action (“if passed, execute the PlatformAction that was the initial trigger which was previously reverted”). However, because you can input arbitrary code in success and fail blocks, it’s not necessarily restricting other actions from taking place.

Finally… the PolicyKit paper outlines 4 requirements for integrating platforms with PolicyKit, which I think are relevant to your original question:

1. Authentication flow that allows PolicyKit to perform and govern a broad set of actions
2. A way to send messages to users on the platform.
3. PolicyKit can execute and revert actions that take place on the platform.
4. The integration must incorporate a listener to listen for user actions on the platform as well as a listener for votes on a notification message. For instance, votes could be recorded via an emoji reaction or a reply to a notification message.

[Shauna]

I would make it more broad, and say that drivers can:

1. “Listen” for actions happening on- or off-platform*.
2. Initiate some decision-making process to determine what to do next (for example, initiating a vote)
3. Initiate an on- or off-platform action based on the result of (2).

This sounds broadly right, although we shouldn’t assume the decision-making process has to be manual or actively conducted by human beings in response to an action; that is, communities should be able to set automated rules (though that is almost definitely out of scope for the prototype).

In my mind, the “driver” doesn’t need to be capable of conducting the decision-making process itself. For step 2 it could, for example, offload the decision-making to some “voting module” (backed by Loomio, for example). I feel like this is the core thing that will make the whole system expressive/modular. Without it, we would need a separate driver for every kind of decision-making engine. The driver seems like quite a heavy-weight system (listening to all platform actions!), so my feeling is that there should only be 1 driver per metagov instance.

Could we say that the core responsibilities of the driver are “listening for triggers” and “returning responses”? They could generate the response themselves, or they could offload the decision to a voting module, but they’ve always got to be able to answer the question of “Can action X happen in Y context?” or, more ambitiously, “Someone wants X to happen in Y context - what should happen?” (where the answer could be X or it could be something else).

Relatedly… could the driver be the metagov prototype itself? If it’s just listening and returning responses?

[miriam]

Agreed, the decision-making process could be automated rules.

@Shauna

Could we say that the core responsibilities of the driver are “listening for triggers” and “returning responses”? They could generate the response themselves, or they could offload the decision to a voting module, but they’ve always got to be able to answer the question of “Can action X happen in Y context?” or, more ambitiously, “Someone wants X to happen in Y context - what should happen?” (where the answer could be X or it could be something else).

Yes, I think those are the core responsibilities. I hope this isn’t being overly nitpicky, I think the distinction is important:
I would say the question the driver should be answering is not “Someone wants X to happen in Y context - what should happen?” but instead “X happened in Y context - what should happen?” if it’s going to be expressive enough to support many of these use cases. I agree though, it’s definitely more ambitious. The driver is being notified about something that happened on the platform, not of someones intent to do something (right?). If the system then chooses to revert that action, as PolicyKit sometimes does, that’s fine.

PolicyKit seems very much built this way (to answer “Can action X happen?”), and that’s why it has this lifecycle of the trigger action being immediately reverted and then re-executed only if the condition passes. But it seems like governance is more than deciding whether or not things can happen.

Even your use case here doesn’t seem to fit this paradigm of checking whether something “can” happen. I’d say it’s not “Can user submit invoice?” or even “User wants to submit invoice, what should happen?” – it’s more like: "User submitted invoice - what should happen?" (In other words, the system is not governing the submission action at all. The submission just acts as a trigger for some other process which happens to result in the expense being approved or rejected). Would you say that’s a fair characterization?

To give another example, what if we wanted to support something like “If a user is reported for the 3rd time, initiate a vote on whether they should be suspended, and if it passes, suspend them.” The question is “User X was reported, what should happen?” NOT Can User X be reported? etc. The trigger action is the user being reported, and the resulting action is something else.

Does that make sense? And if so, is it way too ambitious or for the prototype?

Relatedly… could the driver be the metagov prototype itself? If it’s just listening and returning responses?

I think so. That is exactly where I arrived at the end of last week, and did a bad job explaining it to the group. @thelastjosh and I talked about this.

I think the answer to this depends on the answer to my last question above. If we want this first prototype to be expressive – more expressive than what PolicyKit allows – then I’d say there needs to be some “metagov core” that is the driver, and handles listening for triggers and returning responses.

OTOH, maybe we could achieve something expressive enough by pushing the boundaries of PolicyKit. I’m still no PolicyKit expert so this is a question I’m trying to answer right now. The PK authoring tool does let you write code into each step, so we could maybe make it behave this way, even though it’s not the intended purpose of the tool. (That doesn’t feel great to me, though).

I’ll leave it at that… Thank you @Shauna for your thoughtful responses and questions.

[Shauna]

@miriam

I would say the question the driver should be answering is not “Someone wants X to happen in Y context - what should happen?” but instead “X happened in Y context - what should happen?” if it’s going to be expressive enough to support many of these use cases. […] The submission just acts as a trigger for some other process which happens to result in the expense being approved or rejected). Would you say that’s a fair characterization?

I like the distinction you’ve made here and largely agree.

How would we frame an interaction which is actually asking for permission, though? Where the platform being governed is integrated enough that we don’t need to rollback an action if it’s disallowed - the platform is actually querying before implementing anything?

Platform: “User wants to submit an invoice, what should happen?”
Metagov (via driver): “You should refuse them.”

Does that fit conceptually? (Will hold off on evaluating how ambitious this is until we’ve hammered out what “it” is.)

@miriam

If we want this first prototype to be expressive – more expressive than what PolicyKit allows – then I’d say there needs to be some “metagov core” that is the driver, and handles listening for triggers and returning responses.

OTOH, maybe we could achieve something expressive enough by pushing the boundaries of PolicyKit.

It seems like it might actually be easier for you to build the driver separate from PolicyKit than to figure out a way of wrestling an unfamiliar codebase into doing what you want. Plus I think long term we’d want a core driver to be implementation-nonspecific. So I’d lean towards that approach, but am open to rethinking.

And you’re welcome - I’m getting a lot out of this conversation too.

[miriam]

@Shauna

How would we frame an interaction which is actually asking for permission, though? Where the platform being governed is integrated enough that we don’t need to rollback an action if it’s disallowed - the platform is actually querying before implementing anything?

Hmm, I hadn’t thought about that. I was thinking that metagov would just be reacting to things happening on the platform, not that the platform would invoke metagov directly.

If the platform is invoking metagov in real time to make permissions decisions, I’d think we want to have a synchronous request-response interaction (unlike the event-driven Driver model we’ve been discussing so far, where the Driver subscribes to events). It feels to me like it would be a different kind of system.

[Shauna]

@miriam

Hmm, I hadn’t thought about that. I was thinking that metagov would just be reacting to things happening on the platform, not that the platform would invoke metagov directly.

If the platform is invoking metagov in real time to make permissions decisions, I’d think we want to have a synchronous request-response interaction (unlike the event-driven Driver model we’ve been discussing so far, where the Driver subscribes to events). It feels to me like it would be a different kind of system.

In trying to write a couple more fleshed-out use cases to argue my point, I ended up coming most of the way around to your point of view.

Imagine you have a Forum Platform that does not currently restrict who can post, and a community wants to use Metagov to add a layer of collective moderation so that newcomers to the community have their posts reviewed before they’re allowed through. The Forum Platform would query metagov, asking, “A person wants to make a post. Should we let them do it?” Metagov would reply, “Wait” and then, eventually, “Post it” or “Reject It”.

However, upon reflection, if a platform is already adding a pause into their system, so that they can stop and query Metagov and wait on the response, we can conceptualize them as having broken their “make a post” action down into two steps: submit a post, and publish a post. So it still fits the formulation you suggested:

Platform: A user has taken action ‘submit a post’. What should happen?
Metagov: some time later Take action ‘publish post’.
Platform: Okay, I’ll publish the post.

This allows Metagov to remain asynchronous, although I think achieving a synchronous version of this cycle in specific cases would be a good goal way down the line.

It does put a heavier burden on the integrating platform, because they’ll need to implement the distinction (in this case, perhaps by creating a submitted/published flag on posts) but if they don’t want to make that change, they can stick to governing whatever action it is reactively rather than proactively.

We might want to create some standard terminology for this, as it may be a common pattern for platforms that want to integrate with Metagov to need to add something like this submit/request/try vs implement distinction.