Explanation
Why the waste happens and who it affects.
Inefficient pipeline refresh scheduling occurs when data refresh operations are executed more frequently, or with more compute resources, than the actual downstream business usage requires.
Without aligning refresh frequency and resource allocation to true data consumption patterns (e.g., report access rates in Tableau or Sigma), organizations can waste substantial Snowflake credits maintaining underutilized or rarely accessed data assets.
Billing model
The pricing dimensions that drive this cost.
- Tasks that run on a user-managed virtual warehouse consume credits at an hourly rate set by the warehouse size (for example, Small is 2 credits per hour and Medium is 4), billed per second with a 60-second minimum each time the warehouse starts or resumes. Frequent short refreshes on a warehouse that suspends between runs pay that minimum on every resume.
- Serverless tasks are billed for the compute resources Snowflake assigns to each run, at the rates in the Snowflake Service Consumption Table.
- Dynamic tables refresh on a warehouse according to their target lag, and every refresh cycle also runs change-detection checks that consume Cloud Services credits; Cloud Services credits are billed only when the account's daily Cloud Services usage exceeds 10% of that day's warehouse compute credits.
- This inefficiency is driven by refresh frequency and warehouse size: every refresh consumes credits whether or not anyone reads the refreshed data.
How to detect
3 checks to find it in your estate.
- Review the execution frequency and resource consumption (warehouse size, task duration) of scheduled pipelines and tasks
Map data lineage to understand which downstream assets (e.g., dashboards, reports) depend on each refreshed dataset
- Analyze BI tool usage metrics (e.g., Tableau, Sigma) to assess the frequency of access to downstream data consumers
- Identify pipelines where the refresh cost is high relative to the actual business consumption of the refreshed data
How to fix
4 ways to remove the waste.
- Adjust pipeline refresh frequencies to better align with actual data access patterns (e.g., move from hourly to daily refresh if applicable)
- Right-size the warehouse resources used for pipeline executions to minimize overprovisioning
- Implement usage monitoring frameworks that continuously correlate refresh costs with downstream consumption
- Periodically review pipeline operational costs and business value to optimize refresh schedules proactively
Documentation
Vendor references for pricing and configuration.