How to Access a Private GCS Bucket using a Load Balancer

Abdul Aziz
Cat Administrator

Follow these steps to create a private Google Cloud Storage (GCS) bucket and configure access through a load balancer.
Create a Private GCS Bucket
- Create a GCS Bucket: Make sure it’s private by selecting Enforce public access prevention on this bucket and choose Uniform for Access control.
- Upload Your Content: Add your files to the bucket, for example
image.jpg
.
Create a Service Account
- Navigate to IAM & Admin -> Service Accounts.
- Create a Service Account: Add a name for the service account (referred to as PRIVATE_GCS_SA) and leave other settings as default.
Generate HMAC for the Service Account
- Go to Cloud Storage -> Settings.
- Switch to the Interoperability Tab.
- Create an Access Key: Under “Access keys for service accounts,” click
+ Create a key for a Service Account
. - Select the Service Account: Choose the previously created service account (PRIVATE_GCS_SA) and click
Create Key
. - Save the Access Key and Secret: Note the “Access key” (HMAC_ACCESS_KEY) and “Secret” (HMAC_SECRET) for later use.
Add Service Account Permissions to the Bucket
- Go to Cloud Storage -> [BUCKET_NAME].
- Switch to the PERMISSIONS Tab.
- Grant Access: Click
+ GRANT ACCESS
and type the service account email (PRIVATE_GCS_SA). Assign the “Storage Legacy Object Reader” role and clickSave
.
Create Network Endpoint Groups
- Navigate to Compute Engine -> Network Endpoint Groups.
- Create a Network Endpoint Group: Name it like ”private-gcs-neg”
- Configure the Endpoint Group:
- Type: Select “Internet NEG (Global, Regional).”
- Default Port: 443.
- Add Through: Choose “Fully qualified domain name and port.”
- Domain Name: Input “[BUCKET_NAME].storage.googleapis.com.”
- Create the Endpoint Group.
Create an HTTP(S) Load Balancer
- Go to Network Services -> Load Balancing.
- Create a Load Balancer: Click
+ Create Load Balancer
. - Configure the Load Balancer:
- Type: Select Application Load Balancer (HTTP/HTTPS).
- Frontend Configuration:
- Protocol: HTTPS (includes HTTP/2 and HTTP/3).
- IP Address: Static IP address (or create a new one).
- Certificate: Select or create a new one.
- HTTP to HTTPS Redirect: Enable it.
- Backend Configuration:
- Create a Backend Service: Name it.
- Backend Type: Select Internet network endpoint group.
- Protocol: HTTPS.
- Backend Selection: Choose the previously created Network Endpoint Group.
- Cloud CDN: Ensure Cloud CDN is enabled.
- Security: Select None for Cloud Armor backend security policy.
- Custom Request Headers: Add a header with ”host” as the name and ”[BUCKET_NAME].storage.googleapis.com” as the value.
- Create the Load Balancer: Set the name and finalize the creation.
Configure Private Origin Authentication
- Open Cloud Shell or gcloud CLI.
- Export Backend Configuration: Use the command:
Terminal window 1gcloud compute backend-services export --global [Backend Service Name] \2--destination=private-gcs.yaml \3--project [PROJECT_ID] - Edit the YAML File: Add the following to the
securitySettings
section:Replace1securitySettings:2awsV4Authentication:3accessKeyId: [HMAC_ACCESS_KEY]4accessKey: [HMAC_SECRET]5originRegion: [REGION][HMAC_ACCESS_KEY]
,[HMAC_SECRET]
, and[REGION]
with the appropriate values. - Import the Configuration: Use the command:
Terminal window 1gcloud compute backend-services import --global [Backend Service Name] \2--source=private-gcs.yaml \3--project [PROJECT_ID]
Testing
Run the following cURL command
1curl -X GET -I https://[LOAD_BALANCER_IP]/image.jpg
IF success, it will return 200 similar like this
1HTTP/1.1 200 OK2x-goog-generation: 17213737863820233x-goog-metageneration: 14x-goog-stored-content-encoding: identity5x-goog-stored-content-length: 14566x-amz-meta-x-goog-reserved-source-generation: 17212832409531927x-goog-hash: crc32c=Bbj32A==8x-goog-hash: md5=Gz8lupMp0rjzxJSMOWfbhQ==9x-amz-checksum-crc32c: Bbj32A==10x-goog-storage-class: STANDARD11accept-ranges: bytes12Content-Length: 145613x-guploader-uploadid: ACJd0NqW9GOPmKlURsNWyGpDyxgA34g-BB9psFcQFai8XdB52HuwBbDEiO4B3ZJPOB91_ROPluE14server: UploadServer15alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=259200016via: 1.1 google17Date: Sat, 20 Jul 2024 06:37:12 GMT18Cache-Control: public,max-age=360019Age: 1720Last-Modified: Fri, 19 Jul 2024 07:23:06 GMT21ETag: "1b3f25ba9329d2b8f3c4948c3967db85"22Content-Type: image/png23Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000