Empowering Power BI users: version control in custom scenarios

Introduction

Version control as a concept came from the software development world and became de facto a standard when working with the code. It is also available in Business Intelligence tooling, although only some BI tools support it. Since its launch, Power BI was not among those tools and model/report developers were deprived of this functionality, but this changed in the summer of 2023.

Let’s begin by providing a short definition of version control. At its core, version control is a system that tracks changes in files over time. Sounds simple but it is very powerful and helpful. It allows you to revert to previous versions of files, look for differences between code blocks, see who made the changes and easily collaborate on files with others. Typically, the system used for version control is Git.

In the past, Power BI users could only version whole files, usually in the form of .pbit or .pbix, using either Git or SharePoint integration. Those Power BI file formats are binary, meaning we couldn’t see the human readable code that is used to build them. Versioning those files was only partially useful – it was not possible to have all the benefits of using Git. Ultimately, it led to a lack of observability, difficulties in further development, and multiple .pbix versions being saved on a desktop or published to Power BI Service. Thankfully, it is no longer relevant due to the new file format .pbip – more on this below.

Git integration is tightly coupled with product or feature deployment, and this whole concept is called product lifecycle management. In the case of Power BI, this is implemented through version control and deployment pipelines, although the availability of some of the options heavily depends on license type.

In this article, we will focus on custom integration with Git – by custom, we mean not the ‘out of the box’ integration that came with Microsoft Fabric.

Recent technical advancements in Power BI file formats

A new file extension – .pbip (Power BI Project or PBIP) was released in 2023, along with a new modelling language – TMDL (Tabular Model Definition Language) that serves for Power BI semantic model definition. TMDL joined (and will probably replace) TMSL (Tabular Model Scripting Language) as it has a more human-friendly syntax.

This change was followed by the June 2024 update, which includes PBIR – the Enhanced Report Format for Power BI Project Files (.pbip) which, in turn, defines the Power BI report layer. Together with TMDL, they provide a friendly file format, which improves development efficiency. These changes allow report/model creators to work with Power BI through code editors (like Visual Studio Code).

Power BI Project can be enabled in Power BI Desktop Options: File > Options and settings > Options > Preview features.

This setting allows you to save your report as a Power BI project file (.pbip).

Once it’s done, new folders filled with files are created. What’s important is that those are text files suitable for version control. So instead of a monolith file format (.pbix or .pbit) we get a well-structured folder with text files. This opens a new world for us, Power BI developers.

With the newest updates, we can create Power BI reports and semantic models by only modifying code. It is called ‘developer mode’ and is actively being developed by Microsoft. 

It’s worth pointing out that TMDL is generally available, but both PBIP and PBIR are currently in preview.  

Let’s open PBIP in Visual Studio Code (in this article, we will be using a very simple report using data from SQLBI’s Contoso data generator). 

As visible in the picture, files are divided into ones relevant to Semantic Model (Dataset) and Report. Both model and report objects can be found in the ‘definition’ subfolders. In addition, the .gitignore file is created, which indicates what should be excluded when creating a commit. As a default, two files are added here, localSettings.json and cache.abf .The latter one is important as it holds data itself.  

The .pbip file (in yellow) is used to open Power BI Desktop. The experience is 1:1 as with .pbix. For now .pbix is the only file format that can be published to Power BI Service. 

Power BI Desktop project semantic model folder

TMDL, unlike TMSL, provides semantic model definitions that are easy to read, document and modify.  TMDL has a folder structure with files defining each table, perspective, role, and culture. Files stored in the semantic model folder are described in detail here.  

If you already have a PBIP using TMSL as a semantic model format, you can convert it to TMDL. You need to open the .pbip file in Power BI Desktop and save it. A prompt asking you to upgrade into TMDL will pop up:

If you upgrade to TMDL, you can’t revert back to TMSL. If you think you might need to go back to TMSL format, save a copy of your project files beforehand.

Power BI Desktop project report folder

Before the June 2024 update, the report definition was saved as a low-readable JSON file (report.json) that didn’t support modifications from non-Power BI applications.

As we mentioned earlier, PBIR is a very recent feature that has more limitations than PBIP itself. For example, publishing to Power BI Service is currently impossible when using PBIR. Still, it is worth to follow recent updates on this topic as it opens a ton of productivity improvements when creating reports. Here you can find a list of limitations. 

With the introduction of Power BI Enhanced Report Format (PBIR), the readability of report definition files has significantly improved. Unlike the legacy file (report.json), PBIR is a publicly documented format. With this format, the report is split into multiple files, where each visual, bookmark, page etc. have their own .json file. Each file documents all properties and lets developers perform syntax validation and use IntelliSense feature while editing the code in editors such as Visual Studio Code. Moreover, the public JSON schema is described at the top of each .json file. 

Similarly to semantic model folder, upgrading your existing project files to the new PBIR format is possible. However, conversion to PBIR cannot be reverted. 

Files stored in the report folder are described in detail here. Schemas of .json files can be found on GitHub repository. 

The report definition is stored inside the definition\ folder with the structure shown in the following example: 

As visible above, all names of visuals, pages and bookmarks follow a default naming convention. These object names are initially a 20-character unique identifier, such as '8f1e206eb6ad1e8a421d’. Note that the 'name’ property within each JSON file can be modified but might break external references inside and outside the report. After renaming any PBIR files or folders, you must restart Power BI Desktop to reflect the new naming convention in the file. 

Here are some example scenarios where using PBIR format can come in handy: 

• Ensuring visuals consistency across pages – it can be done by copying & pasting the visual files across page folders 
• Batch editing across all visuals using a script (for example, hide visual level filters) 
• Ensuring default report configuration – specifying default page and default slicer selection. 

All the above changes can be done manually or using a script. 

Changes visibility

With the addition of .pbip and the fact that both the model and report are in the form of text files, we can now easily introduce version control and see changes between report versions. In our Contoso report, we currently have one measure and two visuals: a slicer to choose year and a column chart showing Total Sales across months and countries: 

Now we will add a new measure (change in the semantic model) and a new visual + modify existing visuals (change in report layer) and save it.

The changes are immediately reflected in VS Code: 

In case of a new measure, the difference is visible in TMDL format: 

The visual layer is updated in visual.json file: 

Git integration process

As for now, local development is usually done in Power BI Desktop, so for all mentioned scenarios, we would have the same initial set-up: 

• Power BI Desktop Project file format enabled in settings 
• Report saved locally so that a folder is created, which is then opened in VS Code 
• Local repository initialized in VS Code 
• Now, anytime developer makes changes in Power BI Desktop and hits ‘save’, it will be reflected in VS Code (it also works in the other direction with ‘developer mode’) 
• Next changes are committed to the local repository 

The next step is to extend the local repository with a remote one – you can create it on GitHub, Azure DevOps or another provider depending on the license and technology stack of your organization we will see differences in the architecture of our solution. 

Before we start to split the hair, we need to mention what are the benefits of having a remote repository: 

• Safety (you have full version history that would not get lost in case of loss of your laptop or deletion of VM) 
• Availability (you can connect to the repository from anywhere, clone it, and do you work) 
• Collaboration (multiple developers can work on a report simultaneously and commit to the same remote repo – with .pbip it is possible to solve/merge conflicts 
• Implementation of branching strategy – best practices created in software development can be used to control and precisely control the whole development experience. 

Branching strategy is an important part of working with Git. Basically, it is a plan of how to organize your repository and especially how to use branches. Many strategies were developed, and there are plenty of blogs and videos describing them, so we will limit our description to one we use in our client’s environment called Gitflow. Below is a basic diagram of how Gitflow looks like: 

Here, we will be leveraging both Git and deployment pipelines. Our remote repository can be created on GitLab, CodeCommit, or another service that is not available as out-of-the-box integration in PBI Premium/MS Fabric. In case your organization is working with Azure Devops or Github (which currently are supported in MS Fabric) but is reluctant to use Fabric git integration as it is still in preview, the method described in this article is a good way to lay some foundations for the version control process.
 

So, the initial setup is the same in all scenarios, the difference lies in the deployment part. By leveraging deployment pipelines, we can easily change parameters between workspaces and have a full deployment history.

How it works: 

• Remote repository is cloned, and a feature branch is created from develop branch 
• Report is developed on this local feature branch 
• When the solution is ready, it is published to the DEV workspace, some initial checks are done, and then it is pushed to the remote repository and a PR to QA branch is also created 
• When PR is accepted, the report is pushed through the deployment pipeline to the UAT workspace, where business users will test the solution 
• If everything is ok, then another PR to develop branch is created, and after it is accepted, the report is pushed through pipelines to PROD workspace, and the version form development is merged to main/master branch 

There is a clear separation of developer tasks and Support tasks which allows the company to have full observability and control over their Power Bi assets. The whole solution can be adjusted to seamlessly integrate with the already adopted branching strategy or deployment rules, e.g. support can deploy only from TEST to PROD workspace, after UATs are accepted.  

Summary

New file formats of Power BI files brought tons of new possibilities for developers that are open to using the code editor. Semantic models can be extended or adjusted, reports can be easily modified with almost no clicking at all, and all the assets can be easily documented and versioned. Implementation of version control can greatly improve the development experience, secure the code and business logic it contains, and allow developers to collaborate. At BitPeak, we have the experience and skills to integrate current solutions with Git and introduce proper deployment processes. If you would like to know more, contact us. 

All content in this blog is created exclusively by technical experts specializing in Data Consulting, Data Insight, Data Engineering, and Data Science. Our aim is purely educational, providing valuable insights without marketing intent.