# On-demand-only configuration for non-production Databricks clusters Canonical: https://www.pointfive.co/efficiency-hub/inefficiencies/on-demand-only-configuration-for-non-production-databricks-clusters In non-production environments - such as development, testing, and experimentation-many teams default to on-demand nodes out of habit or caution. By: Benjamin van der Maas Updated: 2026-09-28 [Cloud Efficiency Hub](https://www.pointfive.co/efficiency-hub) # On-Demand-Only Configuration for Non-Production Databricks Clusters The short version In non-production environments - such as development, testing, and experimentation-many teams default to on-demand nodes out of habit or caution. [Benjamin van der Maas](https://www.linkedin.com/in/benjamin-van-der-maas-71168466/) Manager, Cloud FinOps @ Johnson & Johnson Databricks service [Databricks Clusters](https://www.pointfive.co/efficiency-hub/cloud-services/databricks-clusters) Category [Compute](https://www.pointfive.co/efficiency-hub/service-category/compute) Reference CER-0098 Type Suboptimal Pricing Model ## Explanation Why the waste happens and who it affects. However, Databricks offers built-in support for using spot instances safely. Its job scheduler and cluster management system are designed to detect spot instance evictions and automatically replace them with on-demand nodes when necessary, making the use of spot compute relatively low-risk. Failing to enable spot for non-critical or short-lived workloads leads to unnecessary overspend. The inefficiency often arises because spot usage is not enabled by default and must be explicitly selected in cluster settings. In teams that don't revisit infrastructure defaults or where FinOps guardrails are missing, this results in a persistent cost gap between actual usage and what could be safely optimized. ## Billing model The pricing dimensions that drive this cost. Databricks clusters incur two charges: DBUs billed by Databricks, and the underlying cloud provider virtual machines used for driver and worker nodes. When on-demand instances are selected, VM charges are based on standard cloud provider rates. If spot instances are enabled (where available), the VM portion can be significantly lower (AWS advertises up to 90% off On-Demand prices), but the spot discount does not apply to DBU charges, so total cluster savings are smaller. Databricks includes native failover capabilities that automatically replace preempted spot nodes with on-demand nodes to maintain job continuity, minimizing the impact of eviction risk. ## How to detect 5 checks to find it in your estate. - Identify Databricks clusters that are not configured to use spot instances - Filter for non-production environments (e.g., dev, test, staging) where eviction risk is acceptable - Review the duration and criticality of jobs; short-lived or interruptible workloads are ideal candidates - Check compute configurations and compute policies for spot availability settings (for example, SPOT\_WITH\_FALLBACK and first\_on\_demand on AWS) - Evaluate whether cost differences between current on-demand usage and spot alternatives are material ## How to fix 4 ways to remove the waste. - Enable spot instance usage for non-production clusters where workloads are resilient to interruption - Leverage Databricks' native fallback-to-on-demand capabilities to preserve job continuity - Establish workspace-level defaults or templates that promote spot usage in dev/test clusters - Periodically audit compute configurations to detect persistent on-demand usage in non-critical environments ## Documentation Vendor references for pricing and configuration. - https://docs.databricks.com/clusters/configure.html Reference - https://docs.databricks.com/clusters/instance-pools.html Reference - https://docs.databricks.com/clusters/clusters-manage.html Reference ## Related inefficiencies [Browse the library](https://www.pointfive.co/efficiency-hub) - Databricks Clusters CER-0159 ### [Inefficient Use of Photon Engine in Databricks Compute](https://www.pointfive.co/efficiency-hub/inefficiencies/inefficient-use-of-photon-engine-in-databricks-compute) Photon is enabled by default on many Databricks compute configurations. While it can accelerate certain SQL and DataFrame operations, its performance benefits are workload-specific and may not justify the increased DBU cost. Many... Compute - Databricks Clusters CER-0022 ### [Missing Auto-Termination Policy for Databricks Clusters](https://www.pointfive.co/efficiency-hub/inefficiencies/missing-auto-termination-policy-for-databricks-clusters) In many environments, users launch Databricks clusters for development or analysis and forget to shut them down after use. When no auto-termination policy is configured, these clusters remain active indefinitely, incurring unnecessary... Compute - Databricks Clusters CER-0116 ### [Lack of Graviton Usage in Databricks Clusters](https://www.pointfive.co/efficiency-hub/inefficiencies/lack-of-graviton-usage-in-databricks-clusters) Databricks supports AWS Graviton-based instances for most workloads, including Spark jobs, data engineering pipelines, and interactive notebooks. These instances offer significant cost advantages over traditional x86-based VMs, with... Compute --- Source: the public page above. Product screenshots and illustrative interfaces are examples, not live customer data.