How Long Does Your Data Stay On Google Storage

Data deletion on Google Cloud

Overview

CIO-level Summary

Google takes a principled arroyo to the storage and deletion of Customer Data. Google Cloud is engineered to attain a high caste of speed, availability, durability, and consistency, and the design of systems optimized for these operation attributes must be balanced carefully with the need to attain timely data deletion.
When you delete your Customer Data, Google'southward deletion pipeline begins by confirming the deletion asking and eliminating the information iteratively from application and storage layers, from both active and backup storage systems. This process is described generally in Google's argument on deletion and retention.
Logical deletion occurs in phases, beginning with marking the data for deletion in active storage systems immediately and isolating the data from ordinary processing at the awarding layer. Successive compaction and mark-and-sweep deletion cycles in Google'south storage layers serve to overwrite the deleted information over time. Cryptographic erasure is also used to render the deleted data unrecoverable. Finally, backup systems containing snapshots of Google'south active systems are retired on a standard bicycle.
Deletion from application and storage layers may occur immediately depending on how storage of the data has been configured and the timing of ongoing deletion cycles in the relevant storage layers and data centers. Deletion from active systems typically completes within about two months of the deletion request. Finally, Client Data is removed from Google'due south long-term backup systems, which preserve snapshots of Google systems for up to six months (180 days) to guard confronting natural disasters and catastrophic events.

Introduction

This document gives you lot an overview of the secure process that occurs when y'all delete your Client Data (equally defined in the Google Cloud Terms of Service) stored in Google Cloud. Ensuring rubber deletion of Customer Information at the end of its life cycle is a basic attribute of working with data on any computing platform.

Working with data in whatever deject platform that commits to high levels of availability, speed and accessibility from any location, and durability against information loss or disasters requires technical innovation to achieve prompt deletion at scale. Google, equally an early on player in technology storage platforms for products that process trillions upon trillions of data elements, brings more than a decade of industry feel to conduct on optimizing high operation storage systems for this job.

This whitepaper volition start with an overview of how Customer Data is stored in Google Cloud. Side by side, we will describe Google's deletion pipeline and the period of time it by and large takes to complete deletion at each stage. Finally, we will describe how we prevent any reconstruction of information stored in our platform through a secure hardware decommissioning and sanitization process.

Data Storage and Replication

Our description of how Google Cloud deletes Customer Information necessarily begins with a brief overview of how information storage works inside Google's infrastructure. Google Cloud offers storage services, such as Cloud Bigtable and Cloud Spanner. Most Google Deject applications and services admission Google's storage systems indirectly via these Cloud storage services or through other internal storage services used past Google.

Google Deject is designed to provide low latency, highly available, scalable, and durable solutions. Information replication is disquisitional to achieve these cardinal performance goals. Redundant copies of Customer Information could be stored locally and regionally and even globally, depending on your configuration and the demands of customer projects. Actions taken on data in Google Cloud may be simultaneously replicated in multiple information centers, and so that Client Data is highly available. When performance-impacting changes occur in the hardware, software, or network surroundings, Customer Information is automatically shifted from one system or facility to another, subject field to customers' configuration settings, so that customer projects continue performing at scale and without interruption.

At the physical storage level, Customer Information is stored at rest in two types of systems: active storage systems and backup storage systems. These two types of systems process information differently. Active storage systems are Google Deject Platform'southward production servers running Google'due south application and storage layers. Active systems are mass arrays of disks and drives used to write new data equally well as shop and call up data in multiple replicated copies. Active storage systems are optimized to perform alive read / write operations on Customer Information at speed and calibration.

Google's backup storage systems house full and incremental copies of Google'south active systems for a defined flow of time to help Google recover information and systems in the event of a catastrophic outage or disaster. Unlike agile systems, backup systems are designed to receive periodic snapshots of Google systems and backup copies are retired after a limited window of time as new backup copies are made.

Throughout the storage systems described to a higher place, Customer Data is encrypted when stored at rest. The details of Google's encryption techniques are discussed in greater detail in Google's Cloud Security Whitepapers. Encryption of data at residuum occurs at the application and storage layers, on both active and backup storage media.

Secure and Effective Data Deletion

Data Deletion Pipeline

Once Customer Information is stored in Google Cloud, our systems are designed to store the data deeply until it completes the stages of Google's information deletion pipeline. This section describes this process in detail.

Stage 1 - Deletion asking

The deletion of Customer Data begins when the customer initiates a deletion request. By and large, a deletion request is directed to a specific resource, a Google Cloud project, or the customer'southward Google account. Deletion requests may exist handled in different ways depending on the scope of the client's request:

Resource Deletion: Individual resources containing Customer Information, such as Google Deject Storage buckets, can be deleted in a number of ways from the Cloud Console or via API. For example, Customers may outcome a remove saucepan or rm -r command to delete a storage bucket through the command line or customers may select a storage bucket and delete it from the Cloud Storage Browser.
Project Deletion: As a Google Deject project owner, you tin shut downward a project. Deleting a projection acts every bit a bulk deletion request for all resources tied to the corresponding project_number.
Account Deletion: When you delete your Google business relationship, information technology deletes all Google Deject projects that are solely owned by you. Note that when in that location are multiple owners for a project, the projection is not deleted until all owners are removed from the project or delete their Google accounts. This ensures that Google Deject projects go along so long as they accept an possessor.

While deletion requests are designed primarily to be used by Customers to manage their information, Google may issue deletion requests automatically, for instance when a customer terminates their relationship with Google.

Stage 2 - Soft Deletion

Soft deletion is the natural point in the process to provide a brief internal staging and recovery period to ensure that in that location is fourth dimension to recover any data that has been marked for deletion by accident or mistake. Individual Google Cloud Platform products may adopt and configure such a defined recovery period before the data is deleted from the underlying storage systems and so long as it fits inside Google's overall deletion timeline.

To illustrate, when projects are deleted , Google Cloud first identifies the unique project_number, so it broadcasts a interruption betoken to the Google Deject Platform products containing that project_number, for example App Engine and Deject Bigtable. In this case, App Engine will immediately append operations keyed to that project_number and relevant tables in Cloud Bigtable volition enter an internal recovery menses for up to xxx days. At the finish of the recovery menstruum, Google Cloud broadcasts a indicate to the aforementioned products to begin logical deletion of resources tied to the unique project_number. Then Google waits (and, when necessary, rebroadcasts the signal) to collect an acknowledgement bespeak (ACK) from the applicable products to complete project deletion.

When a Google account is closed, Google Cloud may impose an internal recovery period up to thirty days, depending on past account action. In one case that grace period expires, a signal containing the deleted billing account user_id is broadcasted to Google products and Google Cloud resources tied solely to that user_id are marked for deletion.

Stage iii - Logical Deletion from Active Systems

After the data is marked for deletion and any recovery menses has expired, the data is deleted successively from Google's agile and backup storage systems. On active systems, data is deleted in two ways.

In all Cloud products under Compute, Storage & Databases, and Big Data except Google Cloud Storage, copies of the deleted information are marked as bachelor storage and overwritten over time. In an active storage system, like Deject Bigtable, deleted data is stored as entries inside a massive structured table. Compacting existing tables to overwrite deleted data can be expensive, as information technology requires re-writing tables of existing (non-deleted) information, so marker-and-sweep garbage collection and major compaction events are scheduled to occur at regular intervals to reclaim storage space and overwrite deleted information.

In Google Cloud Storage, Client Data is also deleted through cryptographic erasure. This is an industry standard technique that renders data unreadable by deleting the encryption keys needed to decrypt that data. One advantage of using cryptographic erasure, whether information technology involves Google-supplied or customer-supplied encryption keys, is that logical deletion can exist completed even earlier all deleted blocks of that information are overwritten in Google Cloud'south active and backup storage systems.

Phase 4 - Expiration from Fill-in Systems

Like to deletion from Google'south active systems, deleted data is eliminated from backup systems using both overwriting and cryptographic techniques. In the case of backup systems, however, Customer Information is typically stored within large aggregate snapshots of agile systems that are retained for static periods of time to ensure business organization continuity in the upshot of a disaster (e.chiliad., an outage affecting an entire data center), when the fourth dimension and expense of restoring a system entirely from backup systems may become necessary. Consistent with reasonable business continuity practices, total and incremental snapshots of agile systems are made on a daily, weekly, and monthly cycles and retired after a predefined flow of time to make room for the newest snapshots.

When a backup is retired, information technology is marked as available infinite and overwritten as new daily / weekly / monthly backups are performed.

Note that whatsoever reasonable fill-in cycle imposes a pre-divers filibuster in propagating a data deletion request through backup systems. When Customer Data is deleted from agile systems, it is no longer copied into fill-in systems. Backups performed prior to deletion are expired regularly based on the pre-defined backup bike.

Finally, cryptographic erasure of the deleted information may occur before the backup containing Customer Data has expired. Without the encryption key used to encrypt specific Customer Information, the Customer Data volition be unrecoverable even during its remaining lifespan on Google's backup systems.

Deletion Timeline

Google Cloud is engineered to achieve a high degree of speed, availability, durability, and consistency, and the blueprint of systems optimized for these performance attributes must be balanced carefully with the demand to accomplish timely information deletion. Google Cloud commits to delete Customer Data within a maximum period of near vi months (180 days). This delivery incorporates the stages of Google's deletion pipeline described above, including:

Phase ii - One time the deletion request is made, data is typically marked for deletion immediately and our goal is to perform this step within a maximum menstruum of 24 hours. Afterwards the data is marked for deletion, an internal recovery period of up to 30 days may utilise depending on the service or deletion asking.
Stage 3 - The time needed to complete garbage collection tasks and attain logical deletion from agile systems. These processes may occur immediately later on the deletion request is received, depending on the level of data replication and the timing of ongoing garbage collection cycles. From deletion request, it generally takes about 2 months to delete data from active systems, which is typically plenty time to complete two major garbage drove cycles and ensure that logical deletion is completed.
Stage 4 - Google backup wheel is designed to expire deleted data within data heart backups within half-dozen months of the deletion request. Deletion may occur sooner depending on the level of data replication and the timing of Google's ongoing fill-in cycles.

Deletion Pipeline Diagram Figure 1: The Stages of Google Deject's Deletion Pipeline

In add-on to Google Cloud'southward deletion pipeline, a disciplined media sanitization plan enhances the security of the deletion process past preventing forensic or laboratory attacks on the physical storage media one time it has reached the stop of its life cycle.

Google meticulously tracks the location and condition of all storage equipment within our data centers, through conquering, installation, retirement, and destruction, via barcodes and asset tags that are tracked in Google's asset database. Various techniques such as biometric identification, metal detection, cameras, vehicle barriers, and laser-based intrusion detection systems are used to prevent equipment from leaving the data heart floor without dominance. Learn more in the Google Infrastructure Security Design Overview.

Physical storage media may exist decommissioned for a range of reasons. If a component fails to pass a functioning test at whatsoever point during its life cycle, it is removed from inventory and retired. Google likewise upgrades obsolete hardware to ameliorate processing speed and energy efficiency, or increase storage capacity. Whether hardware is decommissioned due to failure, upgrade, or any other reason, storage media is decommissioned using appropriate safeguards. Google hard drives use technologies like total disk encryption (FDE) and drive locking to protect data at rest during decommission. When a hard drive is retired, authorized individuals verify that the disk is erased by overwriting the drive with zeros and performing a multi-step verification process to ensure the drive contains no data.

If the storage media cannot be erased for whatsoever reason, information technology is stored deeply until it can exist physically destroyed. Depending on available equipment, we either beat and deform the drive or shred the drive into small pieces. In either instance, the disk is recycled at a secure facility, ensuring that no one will be able to read data on retired Google disks. Each data center adheres to a strict disposal policy and uses the techniques described to achieve compliance with NIST SP 800-88 Revision 1 "Guidelines for Media Sanitization" and DoD 5220.22-M "National Industrial Security Program Operating Manual.".