AI governance

An AI feature in a security platform has to be answerable: what it is allowed to do, who approved it, what it cost, and what it was asked. Virtik's approach is to route the AI features described here through one gateway that applies the organisation's policy before anything runs, instead of leaving each screen to decide for itself.

This document describes those controls as they are built, including where they stop. It does not describe what an AI model will say, because that is not something the platform can promise.

One gateway, for the features it governs#

Requests from ChatVRTK and from the Command Dock's assistant, and the AI helpers in Prevention and Training, go to the AI gateway. Speech and knowledge embeddings use it too. The gateway checks the organisation's policy, then calls a provider, and records the request and its outcome.

• Nothing runs without a provider. If none is available, the gateway says so ("No AI provider is connected for this organization"). It does not substitute canned text. A provider can be an organisation connection made by an administrator, Virtik's own managed allowance where the plan includes it, or a member's personal key where the organisation's policy allows one.
• ChatVRTK keeps a failed turn. The message is saved before the call, and a failed turn is stored with an error. The AI helpers in Prevention and Training store nothing when a request fails.

Operating modes#

Each request resolves to one of three modes: Fast, Verified or Critical. A request can ask for Auto, which resolves to the organisation's default.

• Critical adds an instruction that the model label what it says as confirmed, inferred or unverified. Verified adds an instruction too. A caller can set its own temperature; where it does not, Critical uses a low one.
• Virtik raises the mode itself when a request mentions dispatch, evacuation, hazards, emergencies or lockdown, and raises it to at least Verified for confidential or restricted data. It also raises it for requests that involve tools of high risk.
• An organisation chooses a default mode, and whether people may lower it. A plan can limit the modes that can be chosen, but it does not limit the system's own raising of the mode for the cases above.

The risk gate for tools#

When an AI action would do something rather than answer, it is classified by risk and checked against the person asking.

An approval applies only to the action it was given for: if the action changes, it no longer matches. An approval expires after 30 minutes and cannot be used twice. The administrator who approves can be the person who asked: there is no segregation of duties on approvals. Approvals are listed in ChatVRTK's Actions tab.

Budgets and limits#

An organisation sets a budget for itself and, separately, per user, a period (daily, monthly or none), and a rate limit a minute. These are set per provider. A budget is soft, which lets requests through and returns a warning that the interface shows only as a coloured bar, or hard, which refuses requests with a "budget reached" message once the limit is reached.

A hard budget also refuses a request when usage cannot be counted, when the model has no price, and when the rate limit cannot be evaluated. Cost figures are estimates from list prices. Only an organisation administrator can change the policy.

Data leaving the platform#

Which classes of data may be sent to a provider is set per provider. The class is declared by the feature that sends the request; it is not detected from the content. ChatVRTK and Training declare "internal", and Prevention declares "confidential". An organisation-wide data policy also exists, but it has no screen. Its default sends confidential data to "require approval" and denies restricted data.

"Require approval" is not implemented: a request under it is not sent, and the gateway says that no approval workflow exists yet. The result is that, by default, Prevention's AI suggestions are refused until the policy is changed.

Where AI is used#

Current availability#