Lab / 8 steps

A private bucket behind a load balancer.

The bucket stays private. The load balancer signs every request with a service account’s HMAC key, so only it can read the objects.

Lab / A private bucket behind a load balancer
Google Cloud GET /image.jpg signed object reader HMAC curl the internet HTTPS LB static IP + cert Internet NEG port 443 Bucket private Service account HMAC key
  1. Step 01 · Console · Cloud Storage Private

    Lock the bucket

    Create the bucket with public access prevention and uniform access control, then upload a file. Nobody outside the project can read it.

    Public access prevention: Enforced
    Access control:           Uniform
    Object:                   image.jpg
  2. Step 02 · Console · IAM and Cloud Storage Key ready

    Give a service account a key

    Create a service account, give it an HMAC key under Cloud Storage → Settings → Interoperability, and grant it Storage Legacy Object Reader on the bucket.

    Service account: PRIVATE_GCS_SA
    HMAC key:        HMAC_ACCESS_KEY + HMAC_SECRET
    Bucket role:     Storage Legacy Object Reader
  3. Step 03 · Console · Network endpoint groups Backend ready

    Point a NEG at the bucket

    An internet network endpoint group gives the load balancer a backend outside the VPC: the bucket’s own hostname on port 443.

    Name:        private-gcs-neg
    Type:        Internet NEG (Global)
    Endpoint:    [BUCKET_NAME].storage.googleapis.com
    Default port: 443
  4. Step 04 · Console · Load balancing 403 Forbidden

    Put a load balancer in front

    An HTTPS load balancer with a static IP and certificate, the NEG as backend, Cloud CDN on, and a Host header set to the bucket’s hostname. Requests now reach the bucket, unsigned.

    Frontend:   HTTPS, static IP, certificate
    Backend:    Internet NEG (private-gcs-neg), HTTPS
    Cloud CDN:  enabled
    Header:     host: [BUCKET_NAME].storage.googleapis.com
  5. Step 05 · Cloud Shell 403 Forbidden

    Export the backend service

    Private origin authentication is not in the console. Export the backend service to YAML and add the HMAC key under securitySettings.

    Terminal window
    gcloud compute backend-services export --global [Backend Service Name] \
        --destination=private-gcs.yaml \
        --project [PROJECT_ID]
  6. Step 06 · private-gcs.yaml 403 Forbidden

    Add the HMAC key

    awsV4Authentication tells the load balancer to sign each request to the bucket with the service account’s key. Nothing changes until it is imported.

    securitySettings:
        awsV4Authentication:
            accessKeyId: [HMAC_ACCESS_KEY]
            accessKey: [HMAC_SECRET]
            originRegion: [REGION]
  7. Step 07 · Cloud Shell Signing

    Import it

    Import the edited file. From now on every request the load balancer sends to the bucket is signed.

    Terminal window
    gcloud compute backend-services import --global [Backend Service Name] \
        --source=private-gcs.yaml \
        --project [PROJECT_ID]
  8. Step 08 · Any terminal 200 OK

    Prove it

    Ask the load balancer for the file. The bucket is still private, but the signed request gets 200 OK.

    Terminal window
    curl -X GET -I https://[LOAD_BALANCER_IP]/image.jpg
    

    HTTP/1.1 200 OK
    x-goog-storage-class: STANDARD
    via: 1.1 google
Read the note: How to Access a Private GCS Bucket Using a Load Balancer All traces