Skip to content

Cloud Strategy & Multi-Cloud Portability

Purpose

This document outlines the cloud strategy for BitVelocity, focusing on multi-cloud portability using Pulumi abstractions, cost optimization, and learning-oriented infrastructure management.

Strategic Objectives

Learning Goals

  1. Multi-Cloud Expertise: Hands-on experience with GCP, AWS, and Azure
  2. Infrastructure as Code: Pulumi Java SDK for cloud-agnostic deployments
  3. Cost Management: Leverage free tiers and optimize for minimal expenses
  4. Portability: Design for seamless migration between cloud providers
  5. Production Patterns: Implement enterprise-grade infrastructure patterns

Business Constraints

  • Budget: <$200 USD monthly infrastructure costs
  • Learning Focus: Technology mastery over business complexity
  • Team Size: 2-3 developers with 10-15 hours/week each
  • Timeline: 12+ months for comprehensive implementation

Cloud Provider Strategy

Primary Cloud: Google Cloud Platform (GCP)

Rationale: Generous free tier, strong Kubernetes support, excellent data services

Free Tier Benefits:

  • Compute Engine: 1 f1-micro instance (always free)
  • Cloud Storage: 5 GB (always free)
  • Cloud Firestore: 1 GiB storage + 50K reads/20K writes daily
  • Cloud Functions: 2M invocations per month
  • Cloud Run: 2M requests per month
  • BigQuery: 1 TB queries per month

Secondary Clouds: AWS and Azure

Purpose: Migration learning, multi-cloud patterns, disaster recovery

AWS Free Tier:

  • EC2: 750 hours of t2.micro instances
  • RDS: 750 hours of db.t2.micro instances
  • S3: 5 GB storage
  • Lambda: 1M requests per month

Azure Free Tier:

  • Virtual Machines: 750 hours B1S instances
  • Storage: 5 GB LRS hot block storage
  • Functions: 1M requests per month
  • App Service: 10 web apps

Pulumi Abstraction Strategy

Cloud-Agnostic Resource Definitions

// Abstract resource definitions
public abstract class CloudStorage {
    protected String bucketName;
    protected String region;
    protected Map<String, String> tags;

    public abstract Output<String> getBucketUrl();
    public abstract Output<String> getBucketArn();
}

// GCP implementation
public class GcpCloudStorage extends CloudStorage {
    private final Bucket bucket;

    public GcpCloudStorage(String name, CloudStorageArgs args) {
        this.bucket = new Bucket(name, BucketArgs.builder()
            .location(args.region())
            .uniformBucketLevelAccess(true)
            .labels(args.tags())
            .build());
    }

    @Override
    public Output<String> getBucketUrl() {
        return Output.format("gs://%s", bucket.name());
    }
}

// AWS implementation
public class AwsCloudStorage extends CloudStorage {
    private final Bucket bucket;

    public AwsCloudStorage(String name, CloudStorageArgs args) {
        this.bucket = new Bucket(name, BucketArgs.builder()
            .region(args.region())
            .tags(args.tags())
            .build());
    }

    @Override
    public Output<String> getBucketUrl() {
        return Output.format("s3://%s", bucket.id());
    }
}

Infrastructure Component Factory

@Component
public class InfrastructureFactory {

    public enum CloudProvider {
        GCP, AWS, AZURE
    }

    public CloudStorage createStorage(CloudProvider provider, String name, CloudStorageArgs args) {
        return switch (provider) {
            case GCP -> new GcpCloudStorage(name, args);
            case AWS -> new AwsCloudStorage(name, args);
            case AZURE -> new AzureCloudStorage(name, args);
        };
    }

    public DatabaseCluster createDatabase(CloudProvider provider, String name, DatabaseArgs args) {
        return switch (provider) {
            case GCP -> new GcpCloudSql(name, args);
            case AWS -> new AwsRds(name, args);
            case AZURE -> new AzureDatabase(name, args);
        };
    }

    public KubernetesCluster createK8sCluster(CloudProvider provider, String name, K8sArgs args) {
        return switch (provider) {
            case GCP -> new GkeCluster(name, args);
            case AWS -> new EksCluster(name, args);
            case AZURE -> new AksCluster(name, args);
        };
    }
}

Configuration Management

# environments/local.yaml
cloud:
  provider: local
  region: local
database:
  type: postgresql
  instance_type: local
  storage_gb: 10
kubernetes:
  node_count: 1
  machine_type: local

# environments/gcp-dev.yaml
cloud:
  provider: gcp
  project: bitvelocity-dev
  region: us-central1-a
database:
  type: cloud-sql
  instance_type: db-f1-micro
  storage_gb: 10
kubernetes:
  node_count: 2
  machine_type: e2-micro

# environments/aws-prod.yaml
cloud:
  provider: aws
  region: us-east-1
database:
  type: rds
  instance_type: db.t3.micro
  storage_gb: 20
kubernetes:
  node_count: 3
  machine_type: t3.small

Local Development Strategy

Local Infrastructure with Kind

public class LocalInfrastructure {

    public void deployLocalCluster() {
        // Kind cluster configuration
        var kindConfig = """
            kind: Cluster
            apiVersion: kind.x-k8s.io/v1alpha4
            nodes:
            - role: control-plane
              kubeadmConfigPatches:
              - |
                kind: InitConfiguration
                nodeRegistration:
                  kubeletExtraArgs:
                    node-labels: "ingress-ready=true"
              extraPortMappings:
              - containerPort: 80
                hostPort: 80
              - containerPort: 443
                hostPort: 443
            - role: worker
            - role: worker
            """;

        // Deploy PostgreSQL
        deployPostgreSQL();

        // Deploy Redis
        deployRedis();

        // Deploy Kafka
        deployKafka();

        // Deploy monitoring stack
        deployMonitoring();
    }

    private void deployPostgreSQL() {
        var postgres = new Chart("postgres", ChartArgs.builder()
            .chart("postgresql")
            .version("12.1.9")
            .fetchOpts(FetchOpts.builder()
                .repo("https://charts.bitnami.com/bitnami")
                .build())
            .values(Map.of(
                "auth.enablePostgresUser", true,
                "auth.postgresPassword", "postgres",
                "auth.database", "bitvelocity",
                "primary.persistence.size", "1Gi"
            ))
            .build());
    }
}

Docker Compose for Development

# docker-compose.dev.yml
version: '3.8'
services:
  postgres:
    image: postgres:15
    environment:
      POSTGRES_DB: bitvelocity
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
    ports:
      - "5432:5432"
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - ./scripts/init-db.sql:/docker-entrypoint-initdb.d/

  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    volumes:
      - redis_data:/data

  kafka:
    image: confluentinc/cp-kafka:latest
    depends_on:
      - zookeeper
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
    ports:
      - "9092:9092"

  zookeeper:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181

volumes:
  postgres_data:
  redis_data:

Cloud Migration Strategy

Blue-Green Deployment Across Clouds

public class CrossCloudMigration {

    public void migrateToAws() {
        // Step 1: Deploy to AWS (Green environment)
        var awsInfra = infraFactory.createInfrastructure(AWS, "bitvelocity-aws");
        awsInfra.deploy();

        // Step 2: Migrate data
        var dataMigration = new DataMigration()
            .from(gcpDatabase)
            .to(awsDatabase)
            .withValidation(true)
            .withRollbackPlan(true);
        dataMigration.execute();

        // Step 3: Switch traffic gradually
        var trafficSplitter = new TrafficSplitter()
            .addDestination("gcp", 90)
            .addDestination("aws", 10);
        trafficSplitter.apply();

        // Step 4: Monitor and validate
        var validation = new EnvironmentValidator()
            .validateHealthChecks(awsInfra)
            .validateDataConsistency()
            .validatePerformanceMetrics();

        if (validation.allPassed()) {
            // Gradually increase AWS traffic
            updateTrafficSplit("aws", 50);
            // Eventually: updateTrafficSplit("aws", 100);
        } else {
            dataMigration.rollback();
        }
    }
}

Data Migration Patterns

@Service
public class DataMigrationService {

    public void performCrossCloudMigration(MigrationPlan plan) {
        // 1. Schema migration
        migrateDatabaseSchema(plan.getSourceDb(), plan.getTargetDb());

        // 2. Historical data migration
        migrateHistoricalData(plan);

        // 3. Real-time sync setup
        setupContinuousReplication(plan);

        // 4. Validation and cutover
        validateDataConsistency(plan);
        performCutover(plan);
    }

    private void setupContinuousReplication(MigrationPlan plan) {
        // Debezium connector for real-time sync
        var connector = KafkaConnector.builder()
            .sourceDatabase(plan.getSourceDb())
            .targetDatabase(plan.getTargetDb())
            .conflictResolution(ConflictResolution.TIMESTAMP_BASED)
            .build();

        connector.start();
    }
}

Cost Optimization Strategies

Auto-Scaling Policies

public class CostOptimization {

    @Scheduled(cron = "0 0 22 * * *") // 10 PM daily
    public void scaleDownForNight() {
        if (isWeekend() || isDevelopmentEnvironment()) {
            kubernetesService.scaleDeployment("bitvelocity-services", 0);
            cloudService.stopNonCriticalInstances();
        }
    }

    @Scheduled(cron = "0 0 8 * * MON-FRI") // 8 AM weekdays
    public void scaleUpForWork() {
        kubernetesService.scaleDeployment("bitvelocity-services", 2);
        cloudService.startDevelopmentInstances();
    }

    public void implementSpotInstanceStrategy() {
        var spotConfig = SpotInstanceConfig.builder()
            .maxPrice("0.01") // $0.01 per hour
            .onInterruption(SpotInterruptionAction.HIBERNATE)
            .instanceTypes(List.of("e2-micro", "t3.micro", "B1s"))
            .build();

        kubernetesService.configureSpotInstances(spotConfig);
    }
}

Resource Monitoring and Alerting

# Cost monitoring alerts
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: cost-monitoring
spec:
  groups:
  - name: cost.rules
    rules:
    - alert: HighCloudCosts
      expr: monthly_cloud_cost > 150
      for: 1h
      labels:
        severity: warning
      annotations:
        summary: "Monthly cloud costs approaching budget limit"
        description: "Current monthly cost: ${{ $value }}"

    - alert: UnusedResources
      expr: cpu_utilization < 5
      for: 2h
      labels:
        severity: info
      annotations:
        summary: "Low resource utilization detected"
        description: "Consider scaling down or terminating unused resources"

Storage Optimization

@Component
public class StorageOptimization {

    @Scheduled(fixedRate = 86400000) // Daily
    public void optimizeStorage() {
        // Move old data to cheaper storage tiers
        archiveOldData();

        // Compress infrequently accessed data
        compressArchivalData();

        // Delete temporary data
        cleanupTemporaryData();
    }

    private void archiveOldData() {
        var archivalPolicy = ArchivalPolicy.builder()
            .archiveAfterDays(90)
            .storageClass(StorageClass.COLD)
            .compressionEnabled(true)
            .build();

        storageService.applyArchivalPolicy(archivalPolicy);
    }
}

Security Considerations

Cross-Cloud Security

public class CrossCloudSecurity {

    public void setupCrossCloudNetworking() {
        // VPN connections between clouds
        var vpnGcpToAws = new VpnConnection("gcp-aws-tunnel")
            .setSourceProvider(GCP)
            .setTargetProvider(AWS)
            .setEncryption(VpnEncryption.IPSEC)
            .setSharedKey(vaultService.getSecret("vpn-shared-key"));

        // Service mesh for secure communication
        var serviceMesh = new IstioMesh()
            .enableMtls(true)
            .setCertificateProvider(CertProvider.VAULT)
            .setSecurityPolicies(loadSecurityPolicies());
    }

    public void setupSecretManagement() {
        // Vault cluster for cross-cloud secret management
        var vaultCluster = new HashiCorpVault()
            .setHighAvailability(true)
            .setBackendStorage(StorageBackend.RAFT)
            .setCloudKmsAutoUnseal(true);

        // Configure cloud-specific secret engines
        vaultCluster.enableEngine(SecretEngine.GCP_SECRETS);
        vaultCluster.enableEngine(SecretEngine.AWS_SECRETS);
        vaultCluster.enableEngine(SecretEngine.AZURE_SECRETS);
    }
}

Monitoring and Observability

Cross-Cloud Monitoring

@Configuration
public class MonitoringConfiguration {

    @Bean
    public PrometheusRegistry crossCloudMetrics() {
        var registry = new PrometheusRegistry();

        // Cloud cost metrics
        registry.register(new CloudCostCollector());

        // Cross-cloud latency metrics
        registry.register(new CrossCloudLatencyCollector());

        // Resource utilization across clouds
        registry.register(new ResourceUtilizationCollector());

        return registry;
    }

    @Bean
    public GrafanaDashboard crossCloudDashboard() {
        return GrafanaDashboard.builder()
            .addPanel("Cloud Costs by Provider")
            .addPanel("Cross-Cloud Network Latency")
            .addPanel("Resource Utilization Comparison")
            .addPanel("Service Health Across Clouds")
            .build();
    }
}

Implementation Roadmap

Phase 1: Local Development (Weeks 1-2)

  • Set up Kind cluster with local services
  • Implement basic Pulumi abstractions
  • Deploy core services locally

Phase 2: GCP Deployment (Weeks 3-4)

  • Deploy to GCP using free tier
  • Implement monitoring and alerting
  • Set up CI/CD pipeline

Phase 3: Multi-Cloud Abstractions (Weeks 5-8)

  • Implement AWS and Azure abstractions
  • Test deployment across clouds
  • Implement cost monitoring

Phase 4: Migration Capabilities (Weeks 9-12)

  • Build data migration tools
  • Implement blue-green deployment
  • Test full cloud migration

Phase 5: Production Readiness (Weeks 13-16)

  • Implement security hardening
  • Set up disaster recovery
  • Complete documentation

Success Metrics

Technical Metrics

  • Deployment Time: <10 minutes for full stack deployment
  • Migration Time: <4 hours for complete cloud migration
  • Cost Efficiency: Stay within $200 monthly budget
  • Uptime: >99% availability during active development

Learning Metrics

  • Multi-Cloud Competency: Successful deployment on all three clouds
  • Automation: Fully automated deployment and migration processes
  • Documentation: Complete runbooks for all cloud operations
  • Knowledge Transfer: Team members can independently manage cloud operations

This cloud strategy provides a comprehensive approach to multi-cloud learning while maintaining cost efficiency and production-ready patterns.