# Inefficient use of job clusters in Databricks workflows Canonical: https://www.pointfive.co/efficiency-hub/inefficiencies/inefficient-use-of-job-clusters-in-databricks-workflows When multiple tasks within a workflow are executed on separate job clusters - despite having similar compute requirements - organizations incur... By: Matt Weingarten Updated: 2026-09-28 [Cloud Efficiency Hub](https://www.pointfive.co/efficiency-hub) # Inefficient Use of Job Clusters in Databricks Workflows The short version When multiple tasks within a workflow are executed on separate job clusters - despite having similar compute requirements - organizations incur unnecessary overhead. Matt Weingarten Databricks service [Databricks Workflows](https://www.pointfive.co/efficiency-hub/cloud-services/databricks-workflows) Category [Compute](https://www.pointfive.co/efficiency-hub/service-category/compute) Reference CER-0114 Type Inefficient Configuration ## Explanation Why the waste happens and who it affects. Each cluster must initialize independently, adding latency and cost. This results in inefficient resource usage, especially for workflows that could reuse the same cluster across tasks. Consolidating tasks onto a single job cluster where feasible reduces start-up time and avoids duplicative compute charges. ## Billing model The pricing dimensions that drive this cost. Databricks is billed based on Databricks Units (DBUs) consumed by the compute resources provisioned. Each job cluster accrues DBU charges while running, and additional cost arises from repeated cluster start-up times and redundant resource allocation. ## How to detect 4 checks to find it in your estate. - Review whether multiple clusters are being provisioned within a single workflow for tasks with similar compute requirements - Assess cluster start-up overhead across tasks to determine if repeated initialization is adding latency and cost - Examine job execution logs to confirm whether resource utilization across clusters is consistent and could be consolidated - Evaluate whether the workflow is creating clusters unnecessarily instead of leveraging a single reusable job cluster ## How to fix 3 ways to remove the waste. - Configure a shared job cluster to run multiple tasks within the same workflow when compute requirements are similar - Leverage cluster reuse settings to reduce start-up overhead and improve efficiency - Validate that consolidation does not impact workload performance or isolation requirements before implementing ## Documentation Vendor references for pricing and configuration. - [Configure compute for jobs docs.databricks.com](https://docs.databricks.com/aws/en/jobs/compute) ## Related inefficiencies [Browse the library](https://www.pointfive.co/efficiency-hub) - Databricks Workflows CER-0528 ### [Always-On Structured Streaming Jobs for Batch-Tolerant Databricks Workloads](https://www.pointfive.co/efficiency-hub/inefficiencies/always-on-structured-streaming-jobs-for-batch-tolerant-databricks-workloads) Structured Streaming jobs are often deployed as 24x7 jobs - a continuous job trigger or a run that never ends - so that ingestion from Kafka, Auto Loader or Delta sources is processed as soon as data lands. That keeps job compute running... Compute - Databricks SQL CER-0225 ### [Underuse of Serverless for Short or Interactive Workloads](https://www.pointfive.co/efficiency-hub/inefficiencies/underuse-of-serverless-for-short-or-interactive-workloads) Many organizations continue running short-lived or low-intensity SQL workloads - such as dashboards, exploratory queries, and BI tool integrations - on traditional clusters. This leads to idle compute, overprovisioning, and high baseline... Compute - Databricks Compute CER-0189 ### [Lack of Workload-Specific Cluster Segmentation](https://www.pointfive.co/efficiency-hub/inefficiencies/lack-of-workload-specific-cluster-segmentation) Running varied workload types (e.g., ETL pipelines, ML training, SQL dashboards) on the same cluster introduces inefficiencies. Each workload has different runtime characteristics, scaling needs, and performance sensitivities. When mixed... Compute --- Source: the public page above. Product screenshots and illustrative interfaces are examples, not live customer data.