Explanation
Why the waste happens and who it affects.
In these non-production contexts, job runtimes are typically shorter, SLAs are relaxed or nonexistent, and performance gains offer little business value.
Enabling Photon in these environments can inflate DBU costs substantially without meaningful runtime improvements. By not differentiating cluster configurations between production and non-production, organizations may pay a premium for workloads that could run just as efficiently on standard compute.
Cluster policies can be used to restrict Photon usage to explicitly tagged production workloads, helping enforce cost-conscious defaults and reduce unnecessary spend.
Billing model
The pricing dimensions that drive this cost.
Photon-enabled compute clusters incur a higher Databricks Unit (DBU) rate compared to standard compute. While the performance boost can be significant for complex workloads, the DBU multiplier - which varies by cloud, instance type, and compute type - may not be justified in development, staging, or ad-hoc environments where performance is less critical. This results in unnecessary cost when Photon is enabled for non-production workloads that don't benefit from its capabilities.
How to detect
5 checks to find it in your estate.
- Identify compute clusters or job configurations with Photon enabled in dev, test, or sandbox environments
- Review naming conventions, tags, or workspace metadata to isolate non-production workload
- Analyze job characteristics: short runtimes, low concurrency, small data volumes, or non-SLA workloads using Photon
- Compare cost and runtime of similar jobs executed with and without Photon enabled
- Review cluster policies and workspace defaults to determine whether Photon is disabled by default for non-prod
How to fix
5 ways to remove the waste.
- Disable Photon by default in dev/test environments using workspace settings or cluster policies
- Create separate cluster templates or policies for production and non-production workloads
- Use tagging or automation to flag or block Photon usage in low-priority environments
- Educate teams on the cost-performance tradeoffs of Photon and when its use is justified
- Periodically review Photon usage by environment and update governance rules accordingly
Documentation
Vendor references for pricing and configuration.