Terraform Private Vertex AI Workbench Notebooks Module

This module deploys Vertex AI Workbench User-Managed Notebooks and Vertex AI Workbench Managed Notebooks on GCP that are private (internal IP only), while still remaining easily usable and accessible by the user of the notebook.

Introduction

User-Managed and Managed Notebooks on Vertex AI allow users to easily deploy GCE VMs with JupyterLab pre-installed. These VMs use an external IP address by default for access to the internet for the purposes of setting up the instance (running startup scripts), exposing the Jupyter user interface, and accessing 3rd party services (such as PyPI and GitHub).

The problem with this is the security issue that arises with external IP addresses. GCE VM Security best practice states that VMs should not have an external IP assigned to it (unless absolutely necessary), as it opens the instance up to attack from the public internet. Therefore, by default, Vertex AI Managed and User-Managed Notebooks are vulnerable to attack from the public internet too.

This Terraform module is the answer to this problem. The module deploys Vertex Workbench instances using only internal IP addresses while still allowing the user interface to be accessed directly via the GCP Vertex Workbench GUI by clicking the 'OPEN JUPYTERLAB' button.

Architecture

The following Notebook/Network Architecture is deployed in this module:

For more information about each component of this design, please refer to ARCHITECTURE.md

Development

Local setup

• Install pre-commit
• Install the pre-commit hooks - pre-commit install

README

The README file is autogenerated using terraform-docs. This is done when you create a pull request (or push to an existing PR).

You can customise the template (including this text for example) in .github/workflows/pr-checks.yml.

Usage

There is an example of how to use this module in the example folder but simple usage is as follows:

module "private_vertex_nb_instances" {
  source                        = "teamdatatonic/secure-vertex-workbench/google"
  version                       = "2.0.0"
  project                       = var.project
  zone                          = var.zone
  region                        = var.region
  notebooks                     = var.notebooks
  additional_vertex_nb_sa_roles = var.additional_vertex_nb_sa_roles
  vpc_network_name              = var.vpc_network_name
  subnet_ip_cidr_range          = var.subnet_ip_cidr_range
  vpc_sc_enabled                = false
  gcs_bucket_name               = var.gcs_bucket_name
  gcs_labels                    = var.gcs_labels
  additional_fw_rules           = var.additional_fw_rules
}

Then perform the following commands on the root folder:

• terraform init to get the plugins
• terraform plan to see the infrastructure plan
• terraform apply to apply the infrastructure build
• terraform destroy to destroy the built infrastructure

Requirements

Name	Version
terraform	>= 0.14.5

Providers

No providers.

Modules

Name	Source	Version
private_vertex_nb_instances	../	n/a

Resources

No resources.

Inputs

Name	Description	Type	Default	Required
additional_fw_rules	Additional firewall rules that you may want to create to allow other traffic	list(object({ name = string description = string direction = string priority = number ranges = list(string) source_tags = optional(list(string)) source_service_accounts = optional(list(string)) target_tags = optional(list(string)) target_service_accounts = optional(list(string)) allow = list(object({ protocol = string ports = list(string) })) deny = list(object({ protocol = string ports = list(string) })) log_config = optional(object({ metadata = string })) }))	[]	no
additional_vertex_nb_sa_roles	Additional roles that you may want to assign to the Vertex AI NB SA	list(string)	[]	no
gcs_bucket_name	Name of the GCS Bucket that will contain the post startup script	string	n/a	yes
gcs_labels	Labels to attach to the GCS Bucket. Useful for labelling resources for billing purposes	map(string)	null	no
notebooks	A map containing the containing the configuration for the desired Vertex AI Workbench User-Managed Notebooks	map(object({ labels = map(string), instance_owner = string, metadata = map(string), type = string, access_type = optional(string) }))	{}	no
project	Your GCP Project ID	string	n/a	yes
region	The GCP region for the GCS bucket and Artifact Registry	string	n/a	yes
subnet_ip_cidr_range	The name of your VPC Subnetwork	string	"10.0.0.0/21"	no
vpc_network_name	The name of your VPC Network	string	n/a	yes
zone	The GCP Zone for Vertex Notebook User-Managed Instances	string	n/a	yes

Outputs

Name	Description
dns_policy_id	an identifier for the dns policy with format projects/{{project}}/policies/{{name}}
firewall_id	an identifier for the fw rules with format projects/{{project}}/global/firewalls/{{name}}
gcr-response-policy-rule_id	an identifier for the gcr dns policy rule with format projects/{{project}}/responsePolicies/{{response_policy}}/rules/{{rule_name}}
gcs_bucket_self_link	The URI of the created bucket.
gcs_bucket_url	The base URL of the bucket, in the format gs://
google-managed-notebooks-response-policy-rule-id	an identifier for the notebooks dns policy rule with format projects/{{project}}/responsePolicies/{{response_policy}}/rules/{{rule_name}}
google_managed_notebook_id	an identifier for the fw rules with format projects/{{project}}/global/firewalls/{{name}}
googleapis-response-policy-rule_id	an identifier for the googleapis dns policy rule with format projects/{{project}}/responsePolicies/{{response_policy}}/rules/{{rule_name}}
nat_id	an identifier for the NAT with format {{project}}/{{region}}/{{router}}/{{name}}
pkg-response-policy-rule_id	an identifier for the pkg.dev dns policy rule with format projects/{{project}}/responsePolicies/{{response_policy}}/rules/{{rule_name}}
private_ip_alloc_id	an identifier for the private ip allocation with format projects/{{project}}/global/addresses/{{name}}
private_ip_self_link	The URI of the created IP address.
router_id	an identifier for the router with format projects/{{project}}/regions/{{region}}/routers/{{name}}
router_self_link	The URI of the created router.
subnetwork_id	an identifier for the VPC subnetwork with format projects/{{project}}/regions/{{region}}/subnetworks/{{name}}
subnetwork_self_link	The URI of the created subnetwork.
user-managed-notebooks-response-policy-rule-id	an identifier for the notebooks dns policy rule with format projects/{{project}}/responsePolicies/{{response_policy}}/rules/{{rule_name}}
user_managed_notebook_id	an identifier for the fw rules with format projects/{{project}}/global/firewalls/{{name}}
vpc_id	an identifier for the VPC network with format projects/{{project}}/global/networks/{{name}}
vpc_self_link	The URI of the created VPC.