We previously announced the release of the Publish and Share Kedro-Viz feature which enables you to easily share your pipeline visualisation by publishing your Kedro-Viz project on Amazon S3. The feature is designed to ease project debugging and collaboration across teams and stakeholders.
What is Kedro-Viz?
Kedro-Viz is an interactive development tool for building and visualising data science pipelines with Kedro. It enables you to monitor the status of your ML project, present it to stakeholders, and easily bring new team members up to speed. You can try it out using our hosted demo and find out more from these blog posts:
We are excited to announce that we have now added support to automate publishing to Github pages through the release of the publish-kedro-viz Github Action. Here's how to use it!
Step 1: Create a Github repository with your Kedro project
The first step is to create a Github repository on github.com to store your Kedro project.
Check that you can run and visualise Kedro-Viz using the following command: kedro viz run
Then follow the Github tutorial to enable GitHub Pages.
Note: While configuring your repository for GitHub Pages, you have two publishing source options. You can choose a branch, or a custom Github Actions workflow. If you choose a branch, the build artifacts will be uploaded to the publish_branch you pass as an input to the action, which defaults to gh-pages. If you choose a custom GitHub Actions workflow, you need to indicate that in the input publishing_source to the action. In this case, no branch will be created and the artifacts are deployed at run-time. You can find more information on configuring a publishing source for Github pages site in the official docs.
The following tutorial below is based on configuring your repository for GitHub Pages, using a branch as your publishing source. See publish-kedro-viz for the option of using a custom Github Actions workflow.
Step 2: Add/Update a GitHub Actions workflow file
To use the publish-kedro-viz Github Action, you need to create a new workflow file ( eg., .github/workflows/publish-kedro-viz.yml) in your project.
You should do this if there is no existing workflow file to update, or you want to use the action in a separate workflow.
As shown in publish-kedro-viz, update the workflow file with the following code:
1- name: publish-kedro-viz
2 uses: kedro-org/publish-kedro-viz@v1Step 3: Trigger the Github workflow
Once this configurations is complete you can trigger the workflow based on the workflow trigger condition. Once triggered, the GitHub workflow “Publish and share Kedro Viz” will automatically start deployment and can be found in the Actions tab of your repository, with your commit message.
For example our test workflow has:
1on:
2 push:
3 branches:
4 - "*"
5 paths-ignore:
6 - "*.md"
7 pull_request:
8 branches:
9 - main
10 workflow_dispatch:This is to ensure the workflow gets triggered on all pushes, PRs to main and can also be triggered manually. This depends completely on your use-case.
When the action is executed successfully, under Pages in the Settings tab, select the newly created branch: gh-pages in Branch.
GitHub Pages will automatically deploy your site from the chosen branch: gh-pages.
Note: As mentioned earlier, alternatively you can select Github Actions as your publishing source.
Step 4: Access your site
In the Actions tab, under pages-build-deployment, select the URL by deploy to access the site.
You can use this URL in any browser to access your Kedro-Viz static site. You can now share this link and collaborate with others.
What are users saying?
“…By the way, the new ability to easily host Kedro-Viz on GitHub Pages is amazing! ... Seems like one of those things every open-source Kedro project should do...”
Deepyaman Datta, Senior Staff Software Engineer at Voltron Data
Recently, a Kedro user who wrote the kedro blog post Building scalable data pipelines with Kedro and Ibis, used this feature to deploy and share his project.
You can view and interact with this project at https://deepyaman.github.io/jaffle-shop/.
Summary
Publish and Share Kedro-Viz with Github Pages enables you to easily share your pipeline visualisation with others. This post explains the steps you need to use it: create a GitHub repository with your Kedro project, add and trigger the workflow files, which deploy the project to the repo URL. You can now access your site. For more information, check out our documentation.
If you are new to the Kedro framework you can learn more about them from our YouTube course.
