Google Cloud release notes

The WITH RECURSIVE feature has been added to Google Standard SQL for BigQuery and is now in Preview. This feature allows a query in a WITH clause to refer to either itself or to queries defined later in the WITH clause.

Cloud Debugger now has Preview support for VPC Service Controls.

M89 release

• TensorFlow Enterprise 2.8 is now available and includes Long Term Version Support. TensorFlow Enterprise 2.8 is available in both Deep Learning Containers and Deep Learning VM Images.
• Upgraded TensorFlow Enterprise 2.6.2 to 2.6.3.

M89 release

• TensorFlow Enterprise 2.8 is now available and includes Long Term Version Support. TensorFlow Enterprise 2.8 is available in both Deep Learning VM Images and Deep Learning Containers.
• Upgraded TensorFlow Enterprise 2.6.2 to 2.6.3.

Event Threat Detection, a built-in service of Security Command Center, launched the Exfiltration: BigQuery Data to Google Drive rule to Preview. This rule detects events where the protected organization's BigQuery data is saved, through extraction operations, to a Google Drive folder. For more information, see Event Threat Detection rules.

You can now view SLOs on a custom dashboard. For more information, see Display SLOs on a dashboard.

Query Insights lets you configure the query sampling rate. For information, see Using Query Insights to improve query performance.

Added v1 gcloud commands.

Added a guide on how to concatenate multiple input videos into a single output video.

Added a guide on how to include captions and subtitles in an output video.

BigQuery now supports materialized views without aggregation and materialized views with inner join. This feature is generally available (GA).

You can now use Cloud EKM with a Virtual Private Network (preview). This means you can access your external key manager with a private endpoint.

See Using Cloud EKM with VPC to learn more.

The Key Access Justifications (KAJ) feature is now generally available in Cloud SQL. You can use KAJ as part of Cloud External Key Manager (EKM). KAJ enables you to view the reason for each Cloud EKM request. Additionally, based on the justification provided, you can automatically approve or deny a request. For related information, see the Overview.

The Key Access Justifications (KAJ) feature is now generally available in Cloud SQL. You can use KAJ as part of Cloud External Key Manager (EKM). KAJ enables you to view the reason for each Cloud EKM request. Additionally, based on the justification provided, you can automatically approve or deny a request. For related information, see the Overview.

The Key Access Justifications (KAJ) feature is now generally available in Cloud SQL. You can use KAJ as part of Cloud External Key Manager (EKM). KAJ enables you to view the reason for each Cloud EKM request. Additionally, based on the justification provided, you can automatically approve or deny a request. For related information, see the Overview.

In GKE, you can now filter Pub/Sub cluster notifications by notification type. For more information, see Receive cluster notifications.

When creating a maintenance exclusion window, you can restrict the exclusion to specify types of maintenance. For example, during a specific time period you can exclude minor upgrades from occurring on your cluster. For more information, see Maintenance exclusions documentation.

Virtual Machine Threat Detection, a built-in service of Security Command Center Premium, is in Preview. During the Preview, VM Threat Detection detects cryptocurrency mining software, which is among the most common types of software installed in compromised cloud environments.

For more information, see Virtual Machine Threat Detection conceptual overview.

Web Security Scanner, a built-in service of Security Command Center, released the CACHEABLE_PASSWORD_INPUT and SESSION_ID_LEAK finding types.

For more information, see Web Security Scanner findings.

Web Security Scanner, a built-in service of Security Command Center, provides detectors for the OWASP Top 10 2017 and OWASP Top 10 2021. For more information, see Detectors and Compliance.

The Anthos Service Mesh dashboard in the Cloud Console now supports cross-project clusters, Anthos on GKE-on-vSphere (on-prem), and Anthos on Bare Metal. For more information, see Observability overview.

Cloud CDN support for custom named cookies and headers in the cache key are Generally Available. You can use these features for A/B (multivariate) testing, canary testing, and similar scenarios. Allowlisting of query parameters is now also enabled for backend buckets, to allow for cache busting.

For details, see the caching documentation.

The Healthcare Natural Language API now uses Vertex AI to extract medical insights from medical text. A new model is now used for healthcare-specific entity extraction.

Cloud Translation - Advanced (v3) support for a regional EU endpoint is now generally available (GA).

Workflows is now available in the following regions:

• europe-west1 (Belgium)
• europe-west6 (Zurich)
• us-east1 (South Carolina)

Release 1.9.4

Anthos clusters on bare metal 1.9.4 is now available for download. To upgrade, see Upgrade Anthos on bare metal. Anthos clusters on bare metal 1.9.4 runs on Kubernetes 1.21.

Release 1.10.1

Anthos clusters on bare metal 1.10.1 is now available for download. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.10.1 runs on Kubernetes 1.21.

You can now set an expiry time for all newly created service account keys in your project, folder, or organization. This feature is in Preview. To use this feature, request access to the Preview release.

Storage Transfer Service now offers Preview support for moving data between two filesystems and keeping them in sync on a periodic schedule. Transfers can be orchestrated by creating a transfer job through the API or gcloud command-line tool. See Transfer data between POSIX file systems for details.

This launch offers a managed way to migrate from a self-managed filesystem to Filestore. If you have on-premises systems generating massive amounts of data that needs to be processed in Google Cloud, you can now use Storage Transfer Service to accelerate data transfer from an on-prem filesystem to a cloud filesystem.

Storage Transfer Service now offers Preview support for preserving POSIX attributes and symlinks when transferring to, from, and between POSIX filesystems. Attributes include the user ID of the owner, the group ID of the owning group, the mode or permissions, the modification time, and the size of the file. See Metadata preservation for details.

Internal HTTP(S) Load Balancing now supports Shared VPC configurations where the load balancer's frontend and URL map can be created in a host or service project, while the backend services and backends can be distributed across multiple service projects in the Shared VPC environment. This is referred to as cross-project service referencing. Cross-project backend services can be referenced in a single URL map.

Cross-project service referencing gives service developers and admins autonomy over the exposure of their services through the centrally managed load balancer.

For details, see:

• Shared VPC architectures
• Setting up Internal HTTP(S) Load Balancing with Shared VPC

This feature is available in Preview.

You can now collect Apache Tomcat metrics and logs from the Ops Agent, starting with version 2.9.0. For more information, see Monitoring third-party applications: Apache Tomcat.

You can now collect PostgreSQL metrics and logs from the Ops Agent, starting with version 2.9.0. For more information, see Monitoring third-party applications: PostgreSQL.

You can now collect Elasticsearch logs from the Ops Agent, starting with version 2.9.0. For more information, see Monitoring third-party applications: Elasticsearch.

Starting with Ops Agent version 2.9.0, you can use the exclude_logs logging processor to prevent the Ops Agent from ingesting specific logs. For more information, see Logging processors.

You can now collect Apache Tomcat metrics and logs from the Ops Agent, starting with version 2.9.0. For more information, see Monitoring third-party applications: Apache Tomcat.

You can now collect PostgreSQL metrics and logs from the Ops Agent, starting with version 2.9.0. For more information, see Monitoring third-party applications: PostgreSQL.

Generally available: Support for up to 48 vCPUs and 312 GB memory on virtual machine (VM) instances that have a single T4 GPU attached is now generally available.

For more information, see Network bandwidths and GPUs.

Dataproc Metastore Auxiliary versions are available in Preview.

Spanner database type is available in Preview.

Enrichment using the Knowledge Graph is now Generally Available.

For more information, see Enterprise Knowledge Graph field enrichment.

Security Command Center supports CIS Google Cloud Computing Foundations Benchmark v1.2.0 (CIS Google Cloud Foundation 1.2.0).

The following detectors have been added:

• BIGQUERY_TABLE_CMEK_DISABLED
• CONFIDENTIAL_COMPUTING_DISABLED
• DNS_LOGGING_DISABLED
• SQL_EXTERNAL_SCRIPTS_ENABLED
• SQL_LOG_DURATION_DISABLED
• SQL_LOG_ERROR_VERBOSITY
• SQL_LOG_EXECUTOR_STATS_ENABLED
• SQL_LOG_HOSTNAME_ENABLED
• SQL_LOG_MIN_ERROR_STATEMENT_SEVERITY
• SQL_LOG_MIN_MESSAGES
• SQL_LOG_PARSER_STATS_ENABLED
• SQL_LOG_PLANNER_STATS_ENABLED
• SQL_LOG_STATEMENT
• SQL_LOG_STATEMENT_STATS_ENABLED
• SQL_REMOTE_ACCESS_ENABLED
• SQL_SKIP_SHOW_DATABASE_DISABLED
• SQL_TRACE_FLAG_3625
• SQL_USER_CONNECTIONS_CONFIGURED
• SQL_USER_OPTIONS_CONFIGURED

For more information, see Detectors and compliance.

Explainable AI in BigQuery ML is now generally available (GA). This feature helps you understand BigQuery ML prediction or forecasting results at scale. For additional information about explainable AI, see the following:

• Explainable AI documentation
• Blog post: BigQuery Explainable AI helps you interpret your ML models
• Tutorials for regression, classification, and forecasting tasks

The following resource types are now publicly available through the Analyze Policy APIs (AnalyzeIamPolicy and AnalyzeIamPolicyLongrunning):

• Cloud Dataplex
  • dataplex.googleapis.com/Asset
  • dataplex.googleapis.com/Zone
  • dataplex.googleapis.com/Task
  • dataplex.googleapis.com/Lake

Added support for LoggingLogBucket resource.

Added support for CloudFunctionsFunction resource.

Added fields spec.alertStrategy and spec.conditions.conditionMatchedLog to MonitoringAlertPolicy resource.

Anthos clusters on VMware 1.9.3-gke.4 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.9.3-gke.4 runs on Kubernetes v1.21.5-gke.1200.

The supported versions offering the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on VMware are 1.10, 1.9, and 1.8.

Anthos clusters on VMware 1.8.6-gke.4 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.8.6-gke.4 runs on Kubernetes 1.20.12-gke.1500.

Dialogflow CX has a new agent design best practices guide.

Released the RDB Snapshots (Preview) feature for Memorystore for Redis. For more details, see RDB Snapshots.

Web Security Scanner, a built-in service of Security Command Center, released the SQL_INJECTION and STRUTS_INSECURE_DESERIALIZATION finding types.

For more information, see Web Security Scanner findings.

Private network access in Service Directory is available in GA.

Private network access helps supported Google Cloud services to connect directly to VPC networks.

Accessing supported regional service endpoints using Private Service Connect with consumer HTTP(S) controls is available in General Availability.

Accessing managed services using Private Service Connect with consumer HTTP(S) controls is available in Preview.

Callback endpoints are now generally available (GA).

Managing routing policies in Cloud DNS is available in GA.

The Intelligent Document Quality Processor is now publicly accessible and now supports 3 more defect types:

• quality/defect_document_cutoff
• quality/defect_text_cutoff
• quality/defect_glare

Notable features

Beta: PodSecurity admission

PodSecurity replaces the deprecated PodSecurityPolicy admission controller (which will be removed in 1.25). PodSecurity is an admission controller that enforces Pod Security Standards on Pods in a Namespace based on specific namespace labels that set the enforcement level. In 1.23, the PodSecurity feature is enabled by default, and applies to namespaces that opt into enforcement. Refer to the PodSecurity documentation and PodSecurityPolicy migration guide for more information.

The Retail console is now available to all Recommendations AI users. The Retail Console is a new way to manage both Recommendations AI and Retail Search seamlessly in one project through a unified onboarding and admin console experience.

We recommend switching to the Retail console and using the Retail documentation, which documents Recommendations AI, the Retail console, and Retail Search.

To switch, go to the new console and click Enable the Retail API. You can then view and manage your project from the new console.

You can now use reCAPTCHA Enterprise account defender to detect and prevent account-related fraudulent activities. This feature is in Public Preview.

A Carbon Footprint Looker block is available to create custom dashboards using Looker.

The following resource types are now publicly available through the Analyze Policy APIs (AnalyzeIamPolicy and AnalyzeIamPolicyLongrunning):

• Game Services
  • gameservices.googleapis.com/GameServerCluster
  • gameservices.googleapis.com/Realm
  • gameservices.googleapis.com/GameServerConfig
  • gameservices.googleapis.com/GameServerDeployment

Public preview: Creating rich-text overview and adding data stewards to your data entries is rolled out to all Data Catalog regions with minimal disruption and in a controlled way. For more information, see Create tag templates, tags, overviews, and data stewards.

VPC-scoped DNS for GKE using Cloud DNS is now generally available for GKE versions 1.21 and later. This allows for seamless VPC-wide DNS resolution of GKE Services. Note that cluster-scoped DNS using Cloud DNS is still in public preview.

A new kubernetes metric, Network policy event count (kubernetes.io/pod/network/policy_event_count), is available (beta) for GKE Dataplane V2 clusters in GKE versions 1.22.3-gke.700 and later.

This metric can be viewed in the Metrics Explorer in Cloud Monitoring for resource type, Kubernetes Pod.

This metric provides visibility into network policy events and shows the Change in the number of network policy events seen in the dataplane, each event has the following metric labels:

• verdict: Policy verdict, possible values: [allow, deny].
• workload_kind: Kind of the workload, policy-enforced-pod belongs to, for example, "Deployment", "Replicaset", "StatefulSet", "DaemonSet", "Job", or "CronJob".
• workload_name: Name of the workload, policy-enforced-pod belongs to.
• direction: Direction of the traffic from the point of view of policy-enforced-pod, possible values: [ingress, egress].

In addition to these metric labels, customers can also see usual resource labels for resource type, Kubernetes Pod: project_id, location, cluster_name, namespace_name, and pod_name.

This metric can be used for setting up automated alerts for specific behaviors (denials higher than a threshold), identifying security issues, gaining better understanding of traffic flow, and troubleshooting.

Managed Anthos Service Mesh now supports GKE Autopilot in the Regular and Rapid channels. For more information, see Configure managed Anthos Service Mesh.

Managed Anthos Service Mesh control plane now displays its provisioning status in the ControlPlaneRevision API. For more information, see Verify the control plane has been provisioned.

Managed Anthos Service Mesh now supports deploying a proxy built on the distroless base image. Note that distroless proxy images do not work with managed data plane.

The distroless base image ensures that the proxy image contains the minimal number of packages required to run the proxy. This improves security posture by reducing the overall attack surface of the image and gets cleaner results with CVE scanners. See Distroless proxy image for more information.

Users of the App Engine Bundled Services for Python 3 can now access Blobstore, Deferred, and Mail handlers in preview, through language-idiomatic libraries.

Generally available: You can now use the SSH troubleshooting tool to help you determine the cause of failed SSH connections.

Generally Available: Configure commitments to renew automatically. For more information, see Renew commitments automatically.

Added support for LoggingLogMetric resource.

Added support for NetworkConnectivitySpoke resource.

Added regional support for ComputeTargetHTTP(S)Proxy resource(s).

Added spec.build.availableSecrets to CloudBuildTrigger resource.

Added spec.nodeConfig.nodeGroupRef and spec.nodeConfig.spot to ContainerCluster and ContainerNodePool resources.

Added spec.readReplicaMode, spec.replicaCount and status.nodes to RedisInstance resources.

Added spec.settings.ipConfiguration.allocatedIpRange to SQLInstance resource.

Added spec.publicAccessPrevention to StorageBucket resource.

Added spec.identityServiceConfig to ContainerCluster resource.

Dialogflow CX now provides an IDENTITY system function, which is useful to copy a composite parameter object in a parameter preset field.

You can now roll back targets from the delivery pipeline visualization in Google Cloud Console.

Preview support for the following integration:

• Image streaming for container images stored in Artifact Registry.

Added support for Dataproc Metastore's beta NetworkConfig field. Beta services using this field can now be used in conjunction with v1 Dataproc clusters.

Google Cloud Deploy now automatically applies provenance labels to deployed resources.

Now available in Preview: Use a compact placement policy to specify that nodes within the node pool should be placed in closer physical proximity to each other within a zone. Having nodes closer to each other can reduce network latency between nodes, which can be useful for tightly-coupled batch workloads.

The following resource types are now publicly available through the Export APIs (ExportAssets and BatchGetAssetsHistory) and the Feed API:

• Cloud SQL
  • sqladmin.googleapis.com/BackupRun

The following resource types are now publicly available through the resource search API (SearchAllResources) and policy search API (SearchAllIamPolicies):

• Cloud OS Config
  • osconfig.googleapis.com/VulnerabilityReport
• Cloud Dataplex
  • dataplex.googleapis.com/Asset
  • dataplex.googleapis.com/Zone
  • dataplex.googleapis.com/Task
  • dataplex.googleapis.com/Lake

Support for VPC Service Controls is now available in Preview.

Support for VPC Service Controls is now available in Preview.

You can now view deployment metrics on the Google Cloud Deploy page in Google Cloud Console.

You can now view a list of documentation relevant to your current screen, including tutorials, by clicking the Learn button on the Google Cloud Deploy page in Google Cloud Console.

Preview stage support for the following integrations:

• Firestore
• Datastore

Unattended project recommender now reports the reduction in gross carbon footprint associated with your workloads. If there is a recommendation, it is now displayed directly in the Carbon Footprint dashboard.

Generally available: Compute Engine now supports machine images in General Availability. You can use machine images to store configuration, metadata, permission, and data required to create a VM instance.

Web Security Scanner, a built-in service of Security Command Center, released the INSECURE_ALLOW_ORIGIN_ENDS_WITH_VALIDATION, INSECURE_ALLOW_ORIGIN_STARTS_WITH_VALIDATION, and XXE_REFLECTED_FILE_LEAKAGE finding types.

For more information, see Web Security Scanner findings.

Windows connection strings

Migrate for Anthos and GKE supports connection strings at the site and global scopes. See Setting connection strings for a data provider for more information.

Reducing system container image size

New alerts have been added to the migration plan to alert you when files may be too large for a successful image. See Specifying content to exclude from the migration for more information.

You can now collect MariaDB metrics and logs from the Ops Agent, starting with version 2.8.0. For more information, see Monitoring third-party applications: MariaDB.

You can now collect MySQL metrics from the Ops Agent, starting with version 2.8.0. For more information, see Monitoring third-party applications: MySQL.

You can now collect Memcached metrics from the Ops Agent, starting with version 2.8.0. For more information, see Monitoring third-party applications: Memcached.

Starting with version 2.8.0, the Ops Agent supports Ubuntu 21.10. For more information, see Linux operating systems.

You can now collect MariaDB metrics and logs from the Ops Agent, starting with version 2.8.0. For more information, see Monitoring third-party applications: MariaDB.

Added support for MonitoringUptimeCheckConfig resource.

Added support for RunService (alpha) resource.

Added support for NetworkServicesGateway (alpha), NetworkServicesMesh (alpha), NetworkServicesGRPCRoute (alpha), NetworkServicesHTTPRoute (alpha), and NetworkServicesTCPRoute (alpha) resources.

Added field spec.networkInterface.queueCount to ComputeInstance and ComputeInstanceTemplate resources.

Added fields spec.bfd.minReceiveInterval, spec.bfd.minTransmitInterval, spec.bfd.multiplier, and spec.bfd.sessionInitializationMode to ComputeRouterPeer resource.

Added fields spec.nodeConfig.gcfsConfig and spec.managedInstanceGroupUrls to ContainerNodePool resource.

Added field spec.nodeConfig.gcfsConfig (deprecated) to ContainerCluster resource. spec.nodeConfig is a deprecated field that we recommend not using in your configuration.

Added field spec.messageRetentionDuration to PubSubTopic resource.

Supported referencing Workload Identity principals in IAMPolicyMember. (Issue #583)

BigQuery standard SQL now supports the JSON data type for storing JSON data. The JSON data type is in Preview. For more information, see Working with JSON data in Standard SQL.

You can allow your Organization's members to request Cloud Marketplace SaaS products for procurement.

Learn more about enabling procurement requests.

Cloud HSM is now available in the Melbourne (australia-southeast2) region.

Preview: You can now disable VM instance creation retries during resizing of both regional and zonal managed instance groups.

Workflows can invoke Cloud Functions or Cloud Run services that have ingress restricted to internal traffic.

Dataflow now fully supports using Identity and Access Management (IAM) custom roles. You can create a custom IAM role and assign it to a user-managed service account used in Dataflow instead of assigning the Dataflow Worker role.

You can now use a pre-built container to perform custom training with PyTorch 1.10.

Anthos component releases for December 2021

Anthos clusters on VMware:

• December 22, 2021: 1.10.0-gke.194 quarterly minor release

Anthos clusters on bare metal:

• December 09, 2021: 1.7.7 patch release
• December 10, 2021: 1.10.0 quarterly minor release
• December 14, 2021: 1.8.7 patch release
• December 22, 2021: 1.9.3 patch release

Anthos clusters on AWS:

• December 14, 2021: aws-1.10.0-gke.5 quarterly minor release (previous generation)
• December 02, 2021: GA release (current generation)

Anthos clusters on Azure:

• December 02, 2021: GA release

Anthos Config Management:

• December 09, 2021: 1.10.0 quarterly minor release

Anthos Service Mesh:

• December 09, 2021: 1.12.0-asm.3 quarterly minor release
• December 13, 2021: 1.11.5-asm.3 patch release
• December 16, 2021: 1.12.0-asm.4 patch release

Connect:

• N/A

Cloud Run for Anthos:

• December 2021: release updates

Migrate for Anthos and GKE:

• December 08, 2021: 1.10 quarterly minor release

Cloud Logging:

• N/A

Cloud Monitoring:

• December 2021: release updates

Security Health Analytics, a built-in service of Security Command Center, launched the DATAPROC_IMAGE_OUTDATED detector to General Availability. This detector finds clusters created with Dataproc image versions that are affected by security vulnerabilities in the Apache Log4j 2 utility (CVE-2021-44228 and CVE-2021-45046). For more information, see Dataproc vulnerability findings.

Anthos clusters on VMware 1.10.0-gke.194 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.10.0-gke.194 runs on Kubernetes v1.21.5-gke.1200.

The supported versions offering the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on VMware are 1.10, 1.9, and 1.8.

Cluster lifecycle Improvements:

• An admin cluster upgrade is resumable after a previous failed admin cluster upgrade attempt.

• GA: Admin cluster registration during new cluster creation is generally available.

• Preview: Admin cluster registration when updating existing clusters is available as a preview feature.

Platform enhancements:

• Preview: A new load balancer option, MetalLB, is available as another bundled software load balancer in addition to Seesaw.This will be the default load balancer choice instead of Seesaw when GA.

• GA: Support for user cluster node pool autoscaling is generally available.

• Preview: You can create admin cluster nodes and user cluster control-plane nodes with Container-Optimized OS by specifying the osImageType as cos in the admin cluster configuration file.

• Windows Node Pool:

  • Preview: The containerd runtime is now available for Windows node pools when Dataplane V2 for Windows is enabled.
  • Node Problem Detector checks containerd service health on the nodes and surfaces problems to the API Server. For version 1.10.0, NPD does not attempt to repair the containerd service.

  • Containerd logs are exported to the Cloud Console.

  • CSI proxy is deployed automatically onto Windows nodes. You can install and use a Windows CSI driver of your choice, such as the SMB CSI driver.

• GA: The multi-NIC capability to provide additional network interfaces to your Pods is generally available.

• GA: You can upgrade to Ubuntu 20.04 and containerd 1.5.

Security enhancements:

• User cluster control plane certificates are automatically rotated at each cluster upgrade. 

Simplify day-2 operations:

• Preview: gkectl update admin supports the enabling and disabling of Cloud Monitoring and Cloud Logging in the admin cluster. 

• Changed the collection of application metrics to use a more scalable monitoring pipeline based on OpenTelemetry. This change significantly reduces the amount of resources required to collect metrics.

• Updated the parser of containerd and kubelet node logs to extract severity level.

• Introduced the --share-with optional flag in the gkectl diagnose snapshot command to share the read permission after uploading the snapshot to a Google Cloud Storage bucket.

Release 1.9.3

Anthos clusters on bare metal 1.9.3 is now available for download. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.9.3 runs on Kubernetes 1.21.

Support for Cloud Storage triggers is now generally available (GA).

Event Threat Detection, a built-in service of Security Command Center, launched the Active Scan: Log4j Vulnerable to RCE rule to General Availability. This rule detects active Log4j vulnerabilities by identifying DNS queries for unobfuscated domains that were initiated by supported Log4j vulnerability scanners. For more information, see Event Threat Detection rules.

System Event audit logs for Cloud Bigtable autoscaling are now generally available (GA).

Internal TCP/UDP Load Balancing now allows you to configure a connection tracking policy for the load balancer's backend service. A connection tracking policy introduces the following new properties to let you customize your load balancer's connection tracking behavior:

• Tracking mode
• Connection persistence on unhealthy backends
• Idle timeout

To learn about how connection tracking works, see Traffic distribution.

This feature is available in General Availability.

• Starting with this release, the Python packages that are installed on each container image are listed in files that are available on Cloud Storage. For example, the lists for this release are available at gs://deeplearning-platform-release/installed-dependencies/containers/m88/.

• Starting with this release, the Python packages that are installed on each image are listed in files that are available on Cloud Storage. For example, the lists for this release are available at gs://deeplearning-platform-release/installed-dependencies/images/m88/.

Managed Anthos Service Mesh now supports Locality Load Balancing and Consistent Hash Load Balancing in the regular and rapid channels.

Anthos Service Mesh now supports Locality Load Balancing and Consistent Hash Load Balancing.

Cloud Composer 2 is now generally available (GA).

Private Service Connect support is available in Preview for Cloud Composer 2.

Authorized networks support is available in Preview.

Database Migration Service now supports creating Cloud SQL for MySQL, Cloud SQL for PostgreSQL, and Cloud SQL for SQL Server instances with customer-managed encryption keys (CMEK) enabled. Click here to access the documentation.

You can now see the database minor version when viewing information about an instance. See Database versions and version policies for a list of the latest supported versions.

You can now set or upgrade your minor version for Cloud SQL for MySQL 8.0.

Cloud SQL now supports MySQL 8.0.26. To upgrade your existing instance to the new version, see how to upgrade your minor version.

MySQL 5.7.35 has been upgraded to 5.7.36.

Cloud SQL for MySQL now supports point-in-time recovery using a timestamp. See Point-in-time recovery.

Cloud SQL for MySQL now supports database auditing. Database auditing lets you track specific user actions in the database, such as table updates, read queries, user privilege grants, and others. To learn more, see MySQL database auditing.

You can now see the database minor version when viewing information about an instance. See Database versions and version policies for a list of the latest supported versions.

You can now see the database minor version when viewing information about an instance. See Database versions and version policies for a list of the latest supported versions.

Preview: Compute-optimized C2D machine types are now available in preview. C2D machine types are built on top of third generation AMD EPYC Milan processors and are a great fit for high-performance computing (HPC) workloads. For more information, see Compute-optimized machine family.

Starring is now generally available (GA). You can now mark your favorite entries as starred and see them on the Data Catalog's main page in the corresponding list. The list of starred entries is private to you.

Eventarc for Cloud Run for Anthos is now available in Preview.

The following Filestore features are now generally available (GA):

• Enterprise tier
• Customer-managed encryption key support
• Snapshots
• Private services access support

Filestore snapshots now support reverting to a snapshot.

Event Threat Detection, a built-in service of Security Command Center, launched the Initial Access: Log4j Compromise Attempt rule to General Availability. This rule detects Java Naming and Directory Interface (JNDI) lookups within headers or URL parameters. These lookups may indicate attempts at Log4Shell exploitation. For more information, see Event Threat Detection rules.

Traffic Director authorization policies are generally available for gRPC deployments with Java and Go clients.

PostgreSQL version 14 is now generally available. To start using PostgreSQL 14, see Create instances.

Generally available: When rolling out configuration or application updates to a stateful or stateless managed instance group, use the minimum and most disruptive allowed actions to control disruption to your workload.

Public preview: You can use the gcloud tool or API to configure stateful IP addresses in a managed instance group. Stateful IP addresses are preserved when VM instances in the group are autohealed, updated, and recreated.

Key Visualizer for Datastore is now generally available (GA).

Dialogflow CX auto sync for agent collaboration is now GA (generally available).

Dialogflow CX change history is now GA (generally available).

The Dialogflow CX simulator now allows you to specify flow versions when interacting with the simulator.

New Lending Processors (Preview)

The following new processors are now available in limited preview:

• 1040 Schedule D Parser
• HOA Statement Parser
• HUD-92900B Parser
• Retirement/Investment Statement Parser
• SSA-89 Parser
• VBA26-0551 Parser

New Versions of Lending Processors

We have launched new versions of the following lending processors.

• 1003 Parser
• 1040 Parser
• 1040 Schedule C Parser
• 1040 Schedule E Parser
• 1099-DIV Parser
• 1099-G Parser
• 1099-INT Parser
• 1099-MISC Parser
• Bank Statement Parser
• Pay Slip Parser
• W2 Parser
• W9 Parser

These new versions use a new lending document splitting and classification model with improved quality and support for more document types. For more information, see the Document types identified by the Lending Splitter & Classifier.

Key Visualizer for Firestore is now generally available (GA).

Added ability to forward syslog messages of a desired severity (like Error or Warning) to Cloud Logging from NSX-T. You can set up alerts and dashboards based on those messages in Google Cloud's operations suite.

For details about this feature, see Configure a private cloud for syslog forwarding.

When you create a custom mode VPC network, you can select predefined firewall rules which address common use cases for connectivity to instances. This feature is available in General Availability.

Access Transparency supports Secret Manager in Preview stage. For the complete list of services that Access Transparency supports, see Supported services.

This release supports creating instances in the c5a, c5ad, i3en, m5a, m5ad, r5a, r5ad, and t3a families.

Release 1.8.7

Anthos clusters on bare metal 1.8.7 is now available for download. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.8.7 runs on Kubernetes 1.20.

BigQuery BI Engine SQL interface is now generally available.

Role-based access control (RBAC)

Role-based access control (RBAC) enables you to tailor access to Chronicle features based on an employee's role in the organization. Assigning a role to a user grants that user the permissions associated with the role, which enables the user to access role-appropriate Chronicle features.

The ability to configure Cloud Run services to have CPU allocated for the entire lifetime of container instances is now at general availability (GA).

Generally available: You can now share reservations of Compute Engine zonal resources between multiple projects. Learn about shared reservations and creating a shared reservation.

Added support for VPCAccessConnector resource

Added support for ComputePacketMirroring resource

Added support for PrivateCACAPool resource

Added support for IAMWorkloadIdentityPool resource

Added support for IAMWorkloadIdentityPoolProvider resource

Added support for CloudIdentityMembership resource

Rollout support for state-into-spec: absent to ContainerCluster resource (Issue #576)

Add billgProject flag in ConfigConnectorContext to specify a quota project to send along with user_project_override header, used for all requests sent from Config Connector. If set on a resource that supports sending the resource project, this value will supersede the resource project. This field can only be set if requestProjectPolicy takes BILLING_PROJECT value

Datastream now supports customer-managed encryption keys (CMEK). Click here to access the documentation.

Added buildDisableIstioSidecar configuration feature.

Added buildPodResources configuration feature.

Added controllerCACerts configuration feature.

Added buildRetentionCount configuration feature.

Added V3 Google stack as build option.

Added V3 kf-v2-to-v3-shim stack as build option.

Access Transparency supports Secret Manager in Preview stage.

Configuring an external TCP/UDP load balancer in Service Directory is available in Preview.

Access Approval provides a preview feature that uses a cryptographic key to sign an access request. Access Approval uses this signature to verify the integrity of the approval.

You can either use the default Google-managed signing key or bring your own signing key. To bring your own signing key, you must meet certain requirements. Using the default Google-managed signing key doesn't require any additional configuration.

For more information about how signing of an access request works, see Overview of Access Approval.

Managed Anthos Service Mesh now supports VPC Service Controls (VPC-SC) as a preview feature in the rapid channel. For more information, see Configure VPC Service Control for Managed Anthos Service Mesh.

A new feature enables more flexibility for integrating Cloud SQL for SQL Server with Managed Service for Microsoft Active Directory. You can integrate your SQL Server instance with a managed AD domain located in a different project.

Dialogflow CX now supports the asia-southeast1 (Jurong West, Singapore) and europe-west3 (Frankfurt, Germany) regions.

A dedicated user interface is now available in Preview.

Event Threat Detection, a built-in service of Security Command Center, launched the Persistence: New API Method rule to Preview. This rule detects anomalous API behavior by examining Cloud Audit Logs for requests to Google Cloud services that a principal has not seen before. For more information, see Event Threat Detection rules.

Integration with AWS Security Token Service is now generally available (GA) for Storage Transfer Service.

Security conscious customers can use Storage Transfer Service to perform transfers from AWS S3 without passing long-term AWS S3 credentials, which have to be rotated or explicitly revoked when they are no longer needed. Refer to the Amazon S3 > Federated Identity tab when setting up access to your data source.

Creating and managing data transfers with the gcloud command-line tool is now available in Preview. You can use gcloud commands to perform agent installation, manage agent pool lifecycles, and orchestrate transfer jobs. This launch simplifies writing scripts to automate transfer workflow.

Control plane observability is now in Preview. This lets you view logs and metrics for the Traffic Director control plane. For more information, see Control plane observability.

Accessing published services using a Private Service Connect endpoint from on-premises hosts that are connected to a VPC network using Cloud VPN is now available in General Availability.

Release 1.10.0

Anthos clusters on bare metal 1.10.0 is now available for download. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.10.0 runs on Kubernetes 1.21.

Improved cluster lifecycle functionalities:

• GA: Enabled Node Problem Detector to run by default on all nodes. You can check if a problem was detected on a node by running the kubectl describe command for the node. Then look for NodeConditions or Events reported by Node Problem Detector.

• GA: Added bmctl backup cluster and bmctl restore cluster commands to facilitate disaster recovery for clusters.

• Preview: Added the ability to reset individual nodes using the SSH key.

• Updated the bmctl check cluster command so that the snapshot of a cluster includes the cluster's YAML file and logs that are in the bmctl-workspace directory.

• Added a new status field cluster.gkeHubRegistrationStatus. The command kubectl get cluster now shows information about the cluster's membership to GKE Hub.

Networking:

• Preview: Enabled Anthos multi-cluster connectivity to provide Anthos clusters a way to connect to another Anthos cluster in the same data center (intra-site, cluster-to-cluster). Pods in connected clusters can reach each other over pod IP addresses without using native address translation (NAT) in between.

• Preview: Enabled IPv4/IPv6 dual stack support. Customers can deploy clusters in a dual-stack network, where IPv4 and IPv6 addresses can be assigned to both nodes and pods.

• Preview: Enabled "flat mode" (a simplified network topology) for IPv4 , where the pod's IPv4 address is visible and routable without masquerading as node IP within the same Layer 2 domain.

• Preview: Enabled SR-IOV. This feature lets you configure Virtual Functions (VFs) on the supported devices on the nodes of their cluster. This feature also lets you define the kernel module you want to bind to the VF.

Observability:

• GA: Added ability to show severity level of an issue in Cloud Logging. Severity level is extracted from containerd and kubelet node logs.

• GA: Changed collection of application metrics to use a more scalable monitoring pipeline based on OpenTelemetry. This change significantly reduces the amount of resources required to collect metrics.

Security:

• GA: Enhanced the capability to rotate cluster certificate authorities (CAs). Updates include support for all cluster types, rotation of front-proxy and etcd CAs, and changes to the bmctl command syntax.

• Preview: Enabled installation of Anthos clusters on bare metal using a short-lived Google Service Account token instead of using Google Service Account keys.

• Enabled Kubernetes control plane and most Anthos system containers to run as non-root users. For details, see Don't run containers as root user.

VM Runtime:

• Preview: Supported enabling or disabling Anthos VM Runtime on user clusters.

• Preview: Enabled Anthos VM Runtime to support QEMU Copy On Write (QCOW2) format, which is a storage format for virtual disks on virtual machines. Some benefits of virtual disk capabilities are independent thin provisioning, better compression, and encryption at rest.

• Preview: Enabled VMRuntime custom resource and the Network custom resource, which let you create VMs on either the node network with a static IP address or the default pod network.

• Preview: Enabled VM pods audit logs for VM runtime resources.

• Preview: Expanded guest OS versions that can run on the virtual machine. We support Windows Server 2019, 2016, Windows 10, Red Hat Enterprise Linux (RHEL) 8, Centos 8, and Ubuntu 20.04 as guest OS.

• Preview: Enabled virtual machine high availability to provide greater uptime for virtual machines instances (VMIs) by automatically detecting and recovering from a range of host machine failures.

The following resource types are now publicly available through the Analyze Policy APIs (AnalyzeIamPolicy and AnalyzeIamPolicyLongrunning):

• Dialogflow
  • dialogflow.googleapis.com/Agent
  • dialogflow.googleapis.com/LocationSettings
• Artifactregistry
  • artifactregistry.googleapis.com/DockerImage
• GKE
  • networking.k8s.io/NetworkPolicy
• DLP
  • dlp.googleapis.com/DlpJob
  • dlp.googleapis.com/DeidentifyTemplate
  • dlp.googleapis.com/InspectTemplate
  • dlp.googleapis.com/JobTrigger
  • dlp.googleapis.com/StoredInfoType
• Service Management
  • servicemanagement.googleapis.com/ManagedService

The following resource types are now publicly available through the resource search API (SearchAllResources) and policy search API (SearchAllIamPolicies):

• Game Service
  • gameservices.googleapis.com/GameServerCluster
  • gameservices.googleapis.com/GameServerConfig
  • gameservices.googleapis.com/GameServerDeployment
  • gameservices.googleapis.com/Realm

The n2-node-128-864 sole-tenant node type is now available in Preview.

Event Threat Detection, a built-in service of Security Command Center, launched the Evasion: Access from Anonymizing Proxy rule to General Availability. This rule detects Google Cloud service modifications that originated from anonymous proxy IP addresses, like Tor IP addresses. For more information, see Event Threat Detection rules.

Dynamic keys are now supported.

Use nomos migrate to easily enable the RootSync and RepoSync APIs in the cluster. These APIs provide you with additional features and gives you the flexibility to sync to a single repository, or multiple repositories.

Added a new metric pipeline_error_observed to capture if there is any error from different stages: rendering, sync, source, readiness.

nomos status surfaces messages from resource conditions when the managed resources are not ready or healthy.

Anthos Service Mesh now supports installations and upgrades on Microsoft Azure Kubernetes Service (AKS) clusters.

Anthos Service Mesh now supports the Certificate Authority Service integration on on-premises platforms (both Anthos on VMware and bare metal). See install and upgrade with default features and CA Service.

Anthos Service Mesh now supports deploying a proxy built on the distroless base image. The distroless base image ensures that the proxy image contains the minimal number of packages required to run the proxy. This improves security posture by reducing the overall attack surface of the image and gets cleaner results with CVE scanners. See Distroless proxy image for more information.

Release 1.7.7

Anthos clusters on bare metal 1.7.7 is now available for download. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.7.7 runs on Kubernetes 1.19.

Autoscaling for Cloud Bigtable is now generally available (GA). Autoscaling helps prevent over-provisioning or under-provisioning by letting Cloud Bigtable automatically add or remove nodes to a cluster when usage changes. In addition, new metrics are available to help you understand how autoscaling is working.

You can now use customer managed encryption keys (CMEK) in Cloud Bigtable instances that are replicated across multiple regions. Previously, CMEK was limited to instances that had clusters in a single region. This feature is generally available (GA).

Cloud SQL now limits the rate for backup and restore operations on the data disk. For more information, see Backup rate limitations and Restore rate limitations.

Cloud SQL now limits the rate for backup and restore operations on the data disk. For more information, see Backup rate limitations and Restore rate limitations.

Runtime version 2.7 is available. You can use runtime version 2.7 to train with TensorFlow 2.7, scikit-learn 1.0.1, or XGBoost 1.5.0. Runtime version 2.7 supports training with CPUs, GPUs, or TPUs.

See the full list of updated dependencies in runtime version 2.7.

You can view Access Transparency logs for Google Workspace services in the Google Cloud Console. For more information, see Viewing Access Transparency logs for Google Workspace.

Dashboards

Chronicle provides a set of default dashboards to monitor data ingestion status, health, rule detection context, IOC matches and alert prioritization, and user sign-ins. Reporting is available by converting a dashboard to a shareable file (PDF, Excel, CSV, etc.). You can also create custom personal and shared dashboards.

Cloud Functions has added support for customer-managed encryption keys, available at the Preview release level.

The Dialogflow CX simulator now provides page lifecycle navigation to help you understand the execution steps taken for each conversational turn.

Replatform Tomcat applications to containers

Version 1.10 introduces a new public offering for replatforming VMs based on Tomcat applications into containers using Apache Tomcat OSS community images. Migrate for GKE now enables: * Detect VMs that host Tomcat web servers and indicate their fit level for containerization. * Discover Tomcat applications as part of the migration processing and their breakdown into individual containers over the Tomcat community images.

See Migrating Tomcat Workloads Migrating Tomcat Workloads.

Fit assessment for Tomcat application servers workloads

The fit assessment tool now supports assessments of Linux workloads running Tomcat application servers. The new assessment capability allows users to inspect their Tomcat applications for automated containerization to GKE, GKE Autopilot, or Cloud Run

Migrate to GKE Autopilot clusters and Cloud Run now in GA

Simplified Linux service manager, which lets you deploy containers to GKE Autopilot clusters and to Cloud Run, is now the default service manager for any migrations performed with Migrate for Anthos and GKE.

See Migrating to Autopilot clusters and Cloud Run for more on these new features.

Assessment of workloads for Shift to Google Compute Engine

Added support for assessing Lift & Shift migrations to Google Compute Engine. The fit assessment tool is enhanced with additional assessment capabilities which can indicate a VM's fit score toward a Lift & Shift migration using Migrate for Compute Engine. The fit assessment report provides users recommended actions based on conditions that can impact automated migration. With the advanced details users can choose the best workloads for migration and fix issues before they impact the automated migration process.

Fit assessment of AWS EC2 workloads

The fit assessment tool now supports assessments of AWS EC2 workloads by running the collection scripts directly on the assessed AWS EC2 VM, or through a remote SSH from the CLI. The new assessment feature enables users to inspect their workload for automated containerization to GKE, GKE AutoPilot and Cloud Run using Migrate For Anthos and GKE.

Fit assessment of Google Compute Engine VM workloads

The fit assessment tool now supports assessment of Google Compute Engine VM workloads by running the collection scripts directly on the assessed Google Compute Engine VM, or through a remote SSH from the CLI. The new assessment feature enables users to inspect their workload for automated containerization to GKE, GKE AutoPilot and Cloud Run using Migrate For Anthos and GKE.

Source platform indication and VM path on Fit Assessment reports

The fit assessment reports in HTML and Cloud Console include information on the source platform of the assessed VM, and a unique ID per platform. This allows users to compare and view information on their assessed workloads from various platforms.

Assessment for containerization on Cloud Run

The fit assessment tool now supports assessments of workloads for containerization to Google Cloud Run - A Google cloud fully managed serverless platform. The new assessment allows users to inspect their workloads for automated containerization using Migrate for Anthos and GKE.

Assessment for containerization on GKE Auto Pilot

The fit assessment tool now supports assessments of workloads for containerization to GKE Auto Pilot - A new mode of operation in Google Kubernetes Engine (GKE) that is designed to reduce the operational cost of GKE clusters. The new assessment capability allows users to inspect their workloads for automated containerization to GKE Auto Pilot using Migrate for Anthos and GKE.

Using RVTools output as a data source for fit assessment

The fit assessment tool now supports analyzing the RVTools .xlsx report file from a single VMware vCenter by running $./mfit discover rvtools name.xlsx. RVTools utilities are used to retrieve VMWare VSphere management data. With the RVTools data source users can easily generate detailed fit assessment reports based on their existing RVTools export.

Fit assessment automatic version checks

The fit assessment tool now checks for the availability of a new version by probing a version check Google Cloud Storage resource.

Extended topic retention allows you to retain published messages for a maximum of 31 days.

For more information, see Topic message retention.

To facilitate the flow of information between Security Command Center and third-party systems, a resource called ExternalSystems was added under the Finding object. A finding can contain multiple ExternalSystems fields.

The ExternalSystems resource can contain any of the following:

• Third-party SIEM/SOAR fields within Security Command Center
• External system information
• External system finding fields

A caller with the Security Center External Systems Editor (roles/securitycenter.externalSystemsEditor) IAM role can update an ExternalSystems object using the organizations.sources.findings.externalSystems.patch API.

Event Threat Detection, a built-in service of Security Command Center, released the Exfiltration: BigQuery Data Extraction rule. This rule is available in Preview. It detects events where an organization's BigQuery data is exported to an externally visible Cloud Storage bucket. For more information, see Event Threat Detection rules.

Anomaly detection in BigQuery ML is now generally available (GA). You can use the ML.DETECT_ANOMALIES function with the ARIMA_PLUS model to detect anomalies in time-series data. You can also use this function with the K-means, Autoencoder, or PCA models to detect anomalies in independent and identically distributed (IID) data.

M87 Release

• Added Artifact Registry's Python keyring authentication library to deep learning vm environments.

M87 Release

• Added Artifact Registry's Python keyring authentication library to deep learning vm environments.

Storage Transfer Service now offers Preview support for detailed logging for objects copied between AWS S3, Azure Blob, ADLS Gen 2, and Cloud Storage. With detailed logs of individual objects available in Cloud Logging, you can verify what was transferred and perform additional data integrity checks. This launch simplifies monitoring, reporting, and troubleshooting. Read Configure transfer logs for details.

A Workflows Service Level Agreement (SLA) is now available and applicable.