Content moderation using a machine learning service

Metagov should support API calls to AI/ML services such as Perspective API, a machine learning service that screens for toxic comments. Since these services are built to be accessed through API calls, building a Metagov plugin for them should be relatively straightforward.

The value of accessing these APIs through Metagov are (1) Metagov provides a curation service, (2) we can add additional governance processes for logging, exposing, and governing the usage of these services. To develop (2), I’ve created a separate use-case on AI governance.

A typical Metagov AI use-case would likely follow the contours of PolicyKit’s use-case with Perspective API. From the PolicyKit paper:

PolicyKit can integrate with external web APIs to support governance. In this example, a platform policy calls the Jigsaw Perspective API [33], to return a toxicity score for the text, which the policy uses to filter out toxic comments. By being able to call external APIs, PolicyKit policies can use resources on the internet to augment their capabilities.

Other examples of ML services

There are many, many other possible ML services besides Perspective API’s toxicity screen, e.g. see Algorithmia’s public marketplace for algorithms / models. For example, it may be interesting to integrate a sentiment analysis service. That said, most of these services do not have anything directly to do with governance, and should not be targeted for integration.

I think it would also be interesting to explore the use of AI to build / search for optimal institutional configurations, e.g. see this paper: “Optimal Auctions through Deep Learning”. However, we will likely first have to define the institutional search space. Maybe @aleksandarpetrov and I will make some progress on this in the next month?