Functions as a service (FaaS) is a great way to architect event-driven systems, whether it is in response to a RESTful HTTP event, responding to an external event, such as a content changes to a storage bucket, or a Pub/Sub message.
Cloud functions can be written in a myriad of languages, Java, Python, NodeJS to name just some, however I have chosen Java as it was a client’s first-choice in languages, and IntelliJ provides a nice debugging experience. …
For this experiment I wanted to assess how difficult it would be to create a small Apache Spark cluster using an Ubuntu Desktop machine as a master, and a single Raspberry Pi as one of the worker nodes. The desktop instance also hosts a worker node as well. The plan is to eventually expand the number of Raspberry Pi nodes over time, since they are quite inexpensive — less than USD $100.00 for a Raspberry Pi 4.
My goal was to add a user-commenting capabilities to a blog, without having to pay monthly fees like Disqus. I also wanted to keep things as inexpensive as possible, so where possible no dedicated servers or traditional databases. Lastly, I wanted to do it all on Google Cloud Platform — I have had a good experience with deploying infrequently-used containerized applications using Cloud Run. Like Cloud Run, both BigQuery and Cloud Functions have a free-tier:
Cloud Architect, AI/ML Scientist, Big Data Practitioner