How Do I Score Migration Candidates by Savings and Risk?

Migrating workloads between cloud environments or upgrading existing infrastructure often promises cost savings and performance improvements. However, the path to realizing these benefits is riddled with pitfalls — from hidden costs in always-on small services to misunderstandings about shared CPU definitions between cloud providers.

In this post, we’ll methodically tackle how to score your migration candidates by savings vs risk, integrating insights about migration complexity and telemetry confidence. We’ll reference industry tools like AWS Compute Optimizer and Azure Advisor, emphasize why averages won’t cut it, and edge you towards smarter cloud cost decisions.

The Problem: Always-On Small Services Hide Cloud Waste

You might already know that in large distributed architectures, dozens (if not hundreds) of small, always-on services quietly build up your cloud bill. These services rarely get the attention given to your big applications, but cumulatively, they often represent a significant portion of cloud waste.

Why? Because these services tend to be:

Provisioned conservatively: Too large or overpowered “just in case” to avoid latency or downtime
Always running: Night or day, they consume resources continuously
Monitored poorly: Using average metrics that mask peak usage and spiky behavior

This lack of precise attention creates hidden pockets of inefficiency that blunt your migration savings or even increase your costs post-migration if not accounted for.

Understanding Shared CPU Definitions: The AWS and Azure Difference

Another layer of complexity: cloud providers define “shared CPU” differently — and this affects performance guarantees and reliability more than most people realize.

Cloud Provider Shared CPU Definition Impact on Migration Scoring AWS Credits-based bursts (e.g., T-series, with CPU credits to accumulate usage)

Performance can degrade predictably when credits run out
Workloads with spiky CPU must be mapped carefully

Azure Shared vCPU with potential noisy neighbor effects but less explicit credit system

CPU throttling more opaque, risk of unpredictable peaks
Requires deeper telemetry to avoid false confidence

The key takeaway here: never treat vCPU counts as rigid performance guarantees. Shared CPU means you need to observe and measure actual peak behavior and account for potential noisy neighbors or throttling.

computingforgeeks.com

Measure Peaks with the Right Observation Window

Misleading cost-saving or risk assessments happen when you base decisions on incomplete or averaged telemetry. Suppose you pull a monthly average CPU utilization from your monitoring dashboard and declare “We are only at 15% CPU — so we can downsize safely.” That misses the 99th percentile peaks that potentially last 5-10% of the time but are critical for performance.

Instead, get exact about your observation window and metrics:

Choose time granularities that reveal spikes: Use 1-minute or 5-minute data points rather than 1-hour averages.
Analyze peak percentiles: Compute 95th (P95) and 99th (P99) utilization values instead of mean or median.
Consider spike duration: Are those peaks sustained for minutes or just seconds? Long spikes are more influential in sizing decisions.
Don’t ignore I/O, memory, and network spikes: CPU alone isn’t the whole story.

Before adjusting instance types or scaling targets, ask: What do the P95 and P99 CPU loads look like over the past 7, 14, or 30 days? How long do those peaks last? If we downsize to a smaller instance, can it handle those bursts without throttling or errors?

Use Percentiles and Spike Duration, Not Averages

Averages are useful for broad trend visualization but deadly for decision-making during migrations.

Here’s why:

Averages can mask occasional high CPU usage or memory pressure that triggers latency or failed requests.
Sizing to average utilization risks under-provisioning and increases operational risk.
Migration complexity rises if post-migration SLAs are violated due to undersizing.
Percentile-based sizing aligns better with actual workload needs by prioritizing peak performance characteristics.

For example, if a service peaks at 85% CPU for 10 minutes every day but averages only 20%, sizing based on average might choose an instance that fails during essential spikes. Using the 95th or 99th percentile with spike duration helps you size accurately considering risk.

Introducing AWS Compute Optimizer and Azure Advisor into Your Scoring Approach

Both AWS Compute Optimizer and Azure Advisor provide recommendations to optimize instance types and performance — but only when used carefully.

AWS Compute Optimizer

AWS Compute Optimizer analyzes historical utilization metrics across CPU, memory, disk I/O, and network I/O, then recommends rightsizing targets (up, down, or sideways scaling). It also estimates potential cost savings.

Strengths: Rich data integration, percentile-based metrics, cross-instance family recommendations, and clear risk flags.
Limitations: Doesn’t always capture telemetry outside monitored AWS services, may understate burst behavior in credit-based instances.
Best practice: Use Compute Optimizer recommendations as a starting point — independently validate peak usage, spike durations, and SLA impact before committing.

Azure Advisor

Azure Advisor provides best practices and optimization suggestions related to cost, performance, security, and reliability. For migration candidates, its cost optimization recommendations highlight underutilized VMs and potential savings.

Strengths: Integrates with Azure Monitor logs and metrics, offers actionable migration insights and consolidation opportunities.
Limitations: May rely heavily on average CPU utilization unless configured rigorously; shared CPU nuances require custom observations.
Best practice: Use Azure Advisor cost recommendations with careful telemetry tuning — combine with customized queries to incorporate P95/P99 analysis and spike duration measurements.

Scoring Migration Candidates: Savings vs Risk Framework

Now let's put all these pieces together. How do you score which workloads are worth migrating, and which ones pose too much risk or complexity?

Factor Description Scoring Criteria Savings Potential Estimated reduction in cloud spend using right-sizing, cheaper instance families, or volume discounts

High: >30% cost savings including storage and egress
Medium: 10%–30% savings
Low: <10% savings or uncertain estimates

Risk Level Likelihood of performance degradation or downtime post-migration

High: High peak CPU/memory spikes, long spike duration, or telemetry gaps
Medium: Occasional spikes, moderate telemetry confidence
Low: Stable utilization, high telemetry confidence, no shared CPU throttling risk

Migration Complexity Technical and operational effort needed including code changes, refactoring, and validation

High: Stateful workloads, legacy dependencies, complex network setups
Medium: Some configuration dependencies, moderate testing needed
Low: Stateless, containerized, easily validated

Telemetry Confidence Quality, granularity, and completeness of monitoring data

High: Fine-grained (1-minute or better), includes P95/P99 and spike duration, multiple metrics
Medium: Mostly reliable, some gaps in coverage or coarser granularity
Low: Sparse data, averages only, no peak monitoring

Weight these factors according to your organizational priorities (example: put more emphasis on risk for customer-facing systems, prioritize savings on internal infrastructure).

Example: Applying the Scoring Framework

Consider a web service you want to migrate from a m5.large AWS instance down to a t3.medium:

Savings Potential: Compute Optimizer reports 35% savings including right-sizing and reserved instance discounts → High
Risk Level: P99 CPU bursts peak at 90% for 5 minutes per day, 2x/month sustained spikes → Medium
Migration Complexity: Stateless API, containerized → Low
Telemetry Confidence: 1-minute granularity logging and monitoring with 99% data completeness → High

The risk isn’t negligible — consider running a pilot on t3.medium with detailed monitoring and rollback criteria before rolling this out broadly.

Tips Before Running Pilots and Rollbacks

Write rollback criteria: Define clear SLOs and error budgets. If latency spikes by >20% or error rate exceeds your threshold, rollback immediately.
Focus on P95 and P99 metrics during pilots: Track tail latency and resource spikes closely.
Watch for correlated events: CPU alone isn’t enough; watch memory, network, disk I/O.
Test shared CPU burstability assumptions: If moving from dedicated to burstable or burstable to dedicated, test peak cases carefully.
Validate telemetry completeness: Ensure no blackout window or missed data periods.

Conclusion

Scoring migration candidates by savings and risk is a nuanced exercise — one that demands you go beyond rough averages and simplistic assumptions. Always-on small services hide cloud waste, shared CPU definitions vary, and spikes define performance, not averages.

Integrate tools like AWS Compute Optimizer and Azure Advisor for data-driven insights, but ground their recommendations in your telemetry’s P95/P99 percentiles with attention to spike duration. Build a scoring rubric that incorporates savings potential, migration complexity, risk level, and telemetry confidence, backed by concrete rollback plans.

This approach will ensure you maximize cloud cost savings while controlling risk — turning migration campaigns from guesswork into well-calibrated engineering projects.