# Inefficient use of Photon engine in Azure Databricks Canonical: https://www.pointfive.co/efficiency-hub/inefficiencies/inefficient-use-of-photon-engine-in-azure-databricks Photon is optimized for SQL workloads, delivering significant speedups through vectorized execution and native C++ performance. By: Mathijs Hendriks Updated: 2026-09-28 [Cloud Efficiency Hub](https://www.pointfive.co/efficiency-hub) # Inefficient Use of Photon Engine in Azure Databricks The short version Photon is optimized for SQL workloads, delivering significant speedups through vectorized execution and native C++ performance. Mathijs Hendriks Azure service [Databricks](https://www.pointfive.co/efficiency-hub/cloud-services/databricks) Category [Compute](https://www.pointfive.co/efficiency-hub/service-category/compute) Reference CER-0115 Type Inefficient Configuration ## Explanation Why the waste happens and who it affects. However, Photon only accelerates workloads that use compatible operations and data patterns. If a workload includes unsupported functions, unoptimized joins, or falls back to interpreted execution, Photon may be silently bypassed - even on a Photon-enabled cluster. In this case, Photon instances on classic compute still consume DBUs at a higher rate than non-Photon instances while delivering no meaningful speed or efficiency gain. This inefficiency typically arises when teams enable Photon globally without validating workload compatibility or updating their pipelines to follow Photon best practices. The result is higher costs with no corresponding benefit - a classic case of configuration drift outpacing optimization discipline. ## Billing model The pricing dimensions that drive this cost. Azure Databricks compute charges are based on Databricks Units (DBUs). - On classic all-purpose, jobs, and pipeline compute, Photon instance types consume DBUs at a higher rate than the same instance type without Photon (Photon is enabled by default there); SQL warehouses and serverless compute always run Photon, with no separate Photon option - If workloads are not optimized for Photon, users may incur higher costs without realizing any performance benefits ## How to detect 4 checks to find it in your estate. - Analyze job execution plans to determine whether Photon is being used end-to-end - Review workloads that run on Photon-enabled clusters but show no runtime improvement over standard execution - Identify SQL operations or UDFs that are unsupported by Photon - Check for UDFs, RDD or Dataset API calls, stateful streaming, or other unsupported operations that force a fallback to the Spark runtime ## How to fix 5 ways to remove the waste. - Ensure that Photon is only enabled for workloads structured to benefit from vectorized execution - Replace UDFs and RDD or Dataset API usage with built-in SQL or DataFrame functions that Photon can execute natively - Use the Spark UI (classic compute) or the query profile (SQL warehouses and serverless) to check which operators and how much task time run in Photon - Monitor DBU consumption alongside job duration to track whether Photon is delivering net cost/performance gains - Collaborate with data engineering teams to continuously tune high-volume pipelines for Photon compatibility ## Documentation Vendor references for pricing and configuration. - [What is Photon? - Azure Databricks learn.microsoft.com](https://learn.microsoft.com/en-us/azure/databricks/compute/photon) - [Azure Databricks pricing azure.microsoft.com](https://azure.microsoft.com/en-us/pricing/details/databricks/) ## Related inefficiencies [Browse the library](https://www.pointfive.co/efficiency-hub) - Databricks CER-0332 ### [Missing Partition Pruning in Delta Lake Table Queries](https://www.pointfive.co/efficiency-hub/inefficiencies/missing-partition-pruning-in-delta-lake-table-queries-on-databricks) When Delta Lake tables are partitioned by specific columns - such as date, region, or tenant identifier - the query engine can use partition pruning to limit data scans to only the relevant subset of files. However, when queries against... Databases - Databricks CER-0020 ### [Lack of Functional Cost Attribution in Databricks Workloads](https://www.pointfive.co/efficiency-hub/inefficiencies/lack-of-functional-cost-attribution-in-databricks-workloads) Databricks cost optimization begins with visibility. Unlike traditional IaaS services, Databricks operates as an orchestration layer spanning compute, storage, and execution - but its spend is often hard to attribute by workload, job, or... Other - Azure Virtual Machine Scale Sets CER-0314 ### [Fixed Instance Count on Virtual Machine Scale Set Without Autoscaling](https://www.pointfive.co/efficiency-hub/inefficiencies/fixed-instance-count-on-virtual-machine-scale-set-without-autoscaling) Azure Virtual Machine Scale Sets can operate in two modes: manual scaling with a fixed instance count, or autoscaling with dynamic instance counts that respond to demand. When a scale set is configured with manual scaling, it maintains the... Compute --- Source: the public page above. Product screenshots and illustrative interfaces are examples, not live customer data.