On-demand ELK Stack on Google Cloud

On demand ELK Stack

This pipeline will provision and configure Elasticsearch, Logstash and Kibana on the Google Cloud platform. 

The pipeline is meant to be used for on-demand data analysis. There are two configurations included. The first is to provision the stack and the second is to tear it down when it’s no longer in use. The tear down configuration first stores a snapshot of the data in a GCP Storage Bucket. The next time the stack is provisioned, it will restore the data from the snapshot and also ingest any new CSV or JSON files that were added to the GCP bucket.