Cloud function

We’ve written a Google Cloud Function (a serverless deployed app) that, when a window is uploaded to the storage ingress bucket, pre-processes/cleans it before moving it to a more permanent home in a different bucket. The ingress bucket is currently set to aerosense-ingress-eu and the output bucket is set to data-gateway-processed-data. Both are part of the aerosense-twined Google Cloud project. You can view the deployed Cloud Function here - it’s called ingress-eu.

There is no need to read further about this if you are only working on data collection from the serial port.

Developing the cloud function

The entrypoint for the cloud function is cloud_functions.main.clean_and_upload_window and it must accept event and context arguments in that order. Apart from that, it can do anything upon receiving an event (the event is an upload of a file to the ingress bucket). It currently uses the window_handler module.

Dependencies

Dependencies for the cloud function must be included in the requirements.txt file in the cloud_functions package.

More information

More information can be found at https://cloud.google.com/functions/docs/writing

Manual redeployment

The cloud function package is included in this (data-gateway) repository in cloud_functions, which is where it should be edited and version controlled. When a new version is ready, it must be manually deployed to the cloud for it to be used for new window uploads (there is no automatic deployment enabled currently):

cd cloud_functions

gcloud functions deploy ingress-eu \
    --runtime python38 \
    --trigger-resource <name_of_ingress_bucket> \
    --trigger-event google.storage.object.finalize \
    --memory 1GB \
    --region <name_of_region> \
    --set-env-vars SOURCE_PROJECT_NAME=<source_project_name>,DESTINATION_PROJECT_NAME=<destination_project_name>,DESTINATION_BUCKET_NAME=<destination_bucket_name>