Auditing information in the app

The auditor app provides functionality to audit information stored in the app against what is actually on AnVIL.

Audit results classes

Results from an audit are returned as an object that is a subclass of AnVILAudit. The subclasses have a method ok() that indicates if the audit was successful or if any errors were detected. It also can list the set of model instances in the app that were audited against AnVIL using get_verified_results(); a dictionary of model instances with detected errors and the errors themselves using get_error_results(); and the set of records that exist on AnVIL but are not in the app using get_not_in_app_results(); and any “not in app” records that have been marked as ignored get_ignored_results().

Audits for different models check different things and will report different potential errors.

Model-specific auditing

Billing project auditing

The BillingProjectAudit class can be used to audit all BillingProject model instances in the app. It runs the following checks:

  1. All BillingProject model instances in the app also exist on AnVIL.

It does not check if there are Billing Projects on AnVIL that don’t have a record in the app.

Account auditing

The AccountAudit class can be used to audit all Account model instances in the app. It runs the following checks:

  1. All Account model instances in the app also exist on AnVIL.

It does not check if there are Accounts on AnVIL that don’t have a record in the app, since this is expected to be the case.

Managed Group auditing

The ManagedGroupAudit class can be used to audit all ManagedGroup model instances in the app. It runs the following checks:

  1. All ManagedGroup model instances in the app also exist on AnVIL.

  2. The service account running the app has the same role (admin vs member) in the app as on AnVIL.

  3. The membership of each group in the app matches the membership on AnVIL (by running an ManagedGroupMembershipAudit audit for each ManagedGroup).

  4. No groups that have the app service account as an Admin exist on AnVIL.

Membership auditing for a single group can be done using the ManagedGroupMembershipAudit class. This class performs the following checks:

  1. All account members of this ManagedGroup in the app are also members in AnVIL.

  2. All account admin of this ManagedGroup in the app are also admin in AnVIL.

  3. All group members of this ManagedGroup in the app are also members in AnVIL.

  4. All group admin of this ManagedGroup in the app are also admin in AnVIL.

  5. All admin in AnVIL are also recorded in the app.

  6. All members in AnVIL are also recorded in the app.

If desired, specific membership records can be ignored by creating an IgnoredManagedGroupMembership instance in the app. Ignored records will be included in the audit results, but will not be considered errors.

Workspace auditing

The WorkspaceAudit class can be used to audit all Workspace model instances in the app. It runs the following checks:

  1. All Workspace model instances in the app also exist on AnVIL.

  2. The service account running the app is an owner on AnVIL of all the Workspace model instances.

  3. The Workspace has the same authorization domains in the app as on AnVIL.

  4. The access to each Workspace in the app matches the access on AnVIL (by running an WorkspaceSharingAudit audit for each Workspace).

  5. No workspaces that have the app service account as an owner exist on AnVIL.

  6. The workspace is_locked status matches AnVIL.

  7. The workspace is_requester_pays status matches AnVIL.

Sharing for a workspace can be audited using the WorkspaceSharingAudit class. This class performs the following checks:

  1. All groups that have access in the app also have access in AnVIL.

  2. Each ManagedGroup that has access in the app has the same access in AnVIL.

  3. The can_compute value is the same in the app and on AnVIL.

  4. The can_share value is as expected on AnVIL based on the group’s role.

  5. No groups or accounts on AnVIL have access to the workspace that are not recorded in the app.

Running audits

Auditing views

The app provides a number of views for auditing various models.

Workspaces and ManagedGroups have additional audit views that can audit the sharing and membership, respectively.

Auditing via management command

The app also provides a management command (run_anvil_audit) that can run audits and (optionally) send an email report. This command can be used to run audits on a regular schedule, e.g., weekly audits via a cron job.

Here are some examples of calling this command:

# To audit all models and print a report to the terminal.
python manage.py run_anvil_audit

# To audit all models and send an email report to test@example.com.
python manage.py run_anvil_audit --email test@example.com

# To audit just the BillingProject and Account models.
python manage.py run_anvil_audit --models BillingProject Account

More information can be found in the help for run_anvil_audit.

# To audit all models and print a report to the terminal.
python manage.py run_anvil_audit --help