Getting started with google-cloud-storage

Remarks#

Google Cloud Storage is a service for durable and highly available storage of objects of any size. You can use Google Cloud Storage for a range of scenarios including serving website content, storing data for archival and disaster recovery, or distributing large data objects to users via direct download.

Google’s official documentation for GCS (Google Cloud Storage) is at https://cloud.google.com/storage/docs/.

Initial Setup

Google maintains documentation on getting started here: https://cloud.google.com/storage/docs/quickstart-console

Getting ready to use GCS:

Create a Google Cloud project, if you don’t have one already.
Enable billing for your project to allow buckets to be created.
(Optional) Install the Google Cloud SDK, which includes gsutil, GCS’s command line utility. As an alternative, you can use gsutil directly from the Google Cloud UI by using the Google Cloud Shell.

HTTP Download (public objects)

If you want to download an object from GCS that is publicly viewable, the simplest way is to use a web browser or a command line tool to fetch a URL with this pattern: https://storage.googleapis.com/bucketName/objectName.

Example: https://storage.googleapis.com/pub/someOfTheTeam.jpg

Upload Files using Python

Import needed libraries:

from gcloud import storage

Define needed variables:

Client: Bundles the configuration needed for API requests

client = storage.Client()

Optional params for Client():

project: the project which the client acts on behalf of. Will be passed when creating a topic. If not passed, falls back to the default inferred from the environment.
credentials: OAuth2 Credentials used for the connection. If not passed, falls back to the default inferred from the environment.
http: HTTP object to make requests. If not passed, an http object is created that is bound to the credentials for the current object.

Bucket: Selects the bucket created in the project through the Google Cloud Console

bucket = client.get_bucket('<your-bucket-name>')

For more detailed information about the Client functions refer to Storage Client

Blob: File name that will be saved.

blob = bucket.blob('my-test-file.txt')

You can also define directories like this:

filename = "%s/%s" % (folder, filename)
blob = bucket.blob(filename)

There are several methods to upload a file. You can be expecting a file in the payload of a POST or PUT request, or have it locally on your file system. You can even send text directly to a text file.

# Uploading string of text
blob.upload_from_string('this is test content!')

# Uploading from a local file using open()
with open('photo.jpg', 'rb') as photo:
    blob.upload_from_file(photo)

# Uploading from local file without open()
blob.upload_from_filename('photo.jpg')

For more detailed information about the upload functions refer to Blob/Objects

If you need your blob to be public, you can set the privacy of the file public:

blob.make_public()
url = blob.public_url