Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The FabCon + SQLCon recap series starts April 14th at 8am Pacific. If you’re tracking where AI is going inside Fabric, this first session is a can't miss. Register now

0

Enable CI/CD for Dataflow Gen2 across all Fabric SKUs (including F2/F4/F64)

Description

Dataflow Gen2 currently does not support true CI/CD scenarios for organizations running Fabric capacities below F8. The core issue is that:

  • Dataflow Gen2 cannot access the current workspace context (e.g., workspaceId="."),

  • the Lakehouse connector always requires an explicit workspace ID, and

  • parameters are only available starting from F8.

As a result, developers must manually modify M‑code after every deployment to Test or Production to update the workspace ID or Lakehouse reference. This is error‑prone, not scalable, and contradicts the CI/CD principles that Fabric aims to support.

For small and medium‑sized organizations, F8 is often not financially feasible, yet these organizations still require Dev/Test/Prod environments and automated deployments. Without parameters or workspace‑independent references, this is currently impossible.

Impact on users

  • CI/CD only works for customers with F8 or higher.

  • Manual M‑code edits are required after every deployment.

  • Deployment pipelines incorrectly mark dataflows as “changed”.

  • The risk of production errors increases.

  • Adoption of Fabric is hindered for smaller organizations.

Requested solution

Microsoft should implement one of the following options (any of them would solve the problem):

  1. Support workspaceId="." in Dataflow Gen2 so the dataflow automatically targets the workspace it runs in.

  2. Make parameters available across all Fabric SKUs, enabling dynamic workspace IDs and Lakehouse names.

  3. Introduce an M‑function such as Fabric.CurrentWorkspace() to expose workspace context to Dataflow Gen2.

Why this matters

This change would enable CI/CD for all customers, eliminate manual code changes, reduce deployment risk, and make Dataflow Gen2 consistent with other Fabric artifacts (Pipelines, Notebooks, SQL). It would also significantly lower the barrier for professional DevOps practices in Fabric, especially for smaller organizations. Most organisations evaluate Fabric on a lower capacity like F2 or F4. If they are happy with the experience, they gradually upgrade the capacity. If CI/CD is not available on the lower capacities, this can be a show stopper and have your custumers walk away. 

Status: Completed

Relative references is actually something that's being rolled out to production and should be globally available by the end of next week. More information about this feature from the link below:

Relative references with Fabric connectors in Dataflow Gen2 - Microsoft Fabric | Microsoft Learn

 

In the meantime, we do support other mechanisms to have a CI/CD ready solution through parameters or variables. You can read more about these features and solution architectures from the link below:

CI/CD and ALM solution architectures for Dataflow Gen2 - Microsoft Fabric | Microsoft Learn

Comments
miguel
Community Admin
Status changed to: Completed

Relative references is actually something that's being rolled out to production and should be globally available by the end of next week. More information about this feature from the link below:

Relative references with Fabric connectors in Dataflow Gen2 - Microsoft Fabric | Microsoft Learn

 

In the meantime, we do support other mechanisms to have a CI/CD ready solution through parameters or variables. You can read more about these features and solution architectures from the link below:

CI/CD and ALM solution architectures for Dataflow Gen2 - Microsoft Fabric | Microsoft Learn

GertjanIdema
Regular Visitor
That's good news. Thanks for the update. I'm looking forward to this functionality.