Atmora Tech

Cloud & Platform

Google Cloud, chosen
for a specific reason

Enterprises that pick Google Cloud normally do so for one of two reasons: BigQuery, or Kubernetes the way a platform team wants it. We build both properly, with the organisation policy and VPC Service Controls that regulated data requires.

Overview

Google Cloud engineering centred on BigQuery, GKE and the data that justifies the move.

Google Cloud is rarely the incumbent, and that changes the shape of the engagement. There is usually one specific workload driving the decision: an analytics estate that has outgrown its warehouse, or a container platform where GKE's operational maturity matters more than provider consolidation. We build for that workload first and generalise afterwards.

BigQuery rewards a different discipline from other warehouses. Partitioning and clustering decide the bill more than anything else, and a query scanning a full table because someone omitted the partition filter costs real money at four in the morning on a schedule. We set slot reservations, enforce required partition filters, and put a query cost budget into CI.

For GKE, Autopilot removes a genuine operational burden and takes away controls that some workloads need: DaemonSets, specific node taints, GPU topology. We choose per cluster rather than per organisation, and we write down the reasoning, because that decision gets revisited every time somebody new joins the platform team.

44%
lower BigQuery slot spend after partition and clustering rework
8.4 min
p95 latency on a 2 TB daily Dataflow batch pipeline
3.1 s
median dashboard query time after materialised views, from 22 s

Capabilities

What this covers

Six areas we staff properly. If your problem sits outside them, the honest note at the foot of this page says so.

  • Organisation and resource hierarchy

    Folder structure, organisation policy constraints, and VPC Service Controls perimeters around data that must not leave. Shared VPC design that avoids a network ticket for every deployment.

  • BigQuery data platform

    Partitioning and clustering design, slot reservations with autoscaling, materialised views, and cost controls including required partition filters and per-project query budgets.

  • GKE and container platform

    Autopilot or Standard chosen per workload, Workload Identity Federation, Gateway API ingress, and a release channel matched to how much upgrade risk each service can reasonably absorb.

  • Data pipelines and streaming

    Dataflow, Pub/Sub and Datastream for change data capture, with schema evolution handled deliberately rather than by whatever the pipeline happens to do when a new column appears.

  • Vertex AI and model serving

    Model deployment on Vertex AI endpoints, feature store design where it earns its keep, and direct advice on when a managed API costs less than hosting your own model weights.

  • Fleet management and hybrid

    Config Sync and Policy Controller across fleets spanning Google Cloud and on-premises clusters, with policy written once and applied everywhere it is supposed to apply.

Deliverables

What you get

  • Organisation hierarchy, policy constraints and Service Controls perimeters
  • BigQuery dataset design with partitioning, clustering and reservation plan
  • GKE cluster baselines in Terraform with a release channel policy
  • Data pipeline definitions with schema contracts and a replay procedure
  • Query cost budgets enforced in the CI pipeline
  • Runbooks for slot exhaustion, pipeline backlog and cluster upgrade

Stack

What we build it with

  • BigQuery
  • Google Kubernetes Engine
  • Dataflow
  • Pub/Sub
  • Datastream
  • Cloud Run
  • Vertex AI
  • Cloud Spanner
  • Looker
  • Config Connector
  • Terraform
  • dbt

Process

How the engagement runs

Two-week increments against a written definition of done. You can stop at any increment boundary and keep everything built so far.

  1. Workload framing

    We establish which workload justifies Google Cloud and what success looks like for it in numbers. Provider choice without that is preference rather than architecture.

  2. Foundation and perimeter

    Organisation hierarchy, IAM, Shared VPC and Service Controls perimeters are built before data lands, because retrofitting a perimeter around live datasets is painful.

  3. Data or platform build

    BigQuery modelling or GKE platform delivery, in Terraform, with cost controls attached from the first deployment rather than added after the first surprise.

  4. Migration and parallel run

    Existing pipelines keep running in parallel until output reconciles for an agreed period, typically two to four weeks of real production data.

  5. Handover

    Cost review cadence, on-call runbooks, and a written list of the decisions we made together with the conditions under which you should reverse them.

When this is the wrong engagement

If your analytics volumes sit under a few terabytes and your team already runs Snowflake or Databricks well, moving to BigQuery will consume a quarter of engineering time for savings you could get by tuning what you have.

FAQ

Questions we get asked

Is BigQuery cheaper than what we run now?

Only if you control scanning. On-demand pricing punishes unpartitioned tables and SELECT star queries, and one badly written scheduled job can cost more in a month than the migration saved in a year. With reservations and enforced partition filters, most estates land 30 to 50 per cent lower.

Autopilot or Standard GKE?

Autopilot for stateless services and teams without dedicated platform engineers: you stop managing nodes and pay a per-pod premium. Standard when you need DaemonSets, custom kernels, particular GPU topologies or fine-grained bin packing. Deciding per cluster is normal and correct.

We are mostly on AWS. Does adding Google Cloud make sense?

It can, for a defined workload with a defined reason, usually analytics. What does not work is running two clouds as equal peers, because you then double the identity model, the network engineering and the on-call knowledge. Treat the second provider as a specialised tool.

How do you keep sensitive data inside the perimeter?

VPC Service Controls perimeters around the projects holding regulated data, with explicit ingress and egress rules and a documented exception process. It is genuinely restrictive, since copying a dataset to a personal project stops working, and that is the intended behaviour.

Related

More in Cloud & Platform

Start a project

Tell us what is
breaking.

We reply within one working day, and the first call is with an engineer who would actually work on it — not an account manager. If we are not the right studio for the problem, we will say so on that call.

Start a project