In the pipeline

From the latest news to upcoming events and interesting topics, “In the Pipeline” is overflowing with updates for the Kedro community.

In the pipeline: July 2024

Glass v6 resin

Kedro and Ibis streamline the management of ML pipelines and SQL queries within a Python project, leveraging Google BigQuery for efficient execution and storage.

Streamlining SQL Data Processing in Kedro ML Pipelines

In the pipeline: May 2024

Glass v6 black

Creating data platforms is a challenging task. A guest author explains how Kedro reduces the learning curve and enables data science teams.

A practical guide to team topologies for ML platform teams

We have added support to automate publishing to Github pages through the publish-kedro-viz Github Action. Learn how to configure and use the feature!

Share a Kedro-Viz with Github pages

In the pipeline: March 2024

In the pipeline: February 2024

From production-ready to production. Bring the flexibility and familiarity of Python, and the scale and performance of modern SQL, to Kedro pipelines.

Building scalable data pipelines with Kedro and Ibis

Voxel v1 black

We've made changes to Kedro in the new 0.19 release to tackle one of the most commonly perceived pain points. Find out more!

Your new Kedro project. Your way.

Glass v8

Check out the official video course to teach you the skills you need to create  data and machine learning pipelines for future-friendly, platform-agnostic data science code 

Learn Kedro from a new video course

Voxel v1 white

This blog post gives details of the enhancements and improvements to Kedro in the recent 0.19 release.

Exploring our new release: Kedro 0.19 

Find out more about how Kedro's award-winning documentation is created using a docs-as-code workflow.

How the Kedro team built award-winning documentation

How to set up and use "publish and share" to host a Kedro-Viz project on Amazon S3.

How to share a Kedro-Viz pipeline visualisation

From the latest news to upcoming events and interesting topics, “In the Pipeline” is overflowing with interesting updates for the Kedro community.

In the pipeline: November 2023

This article describes the “code smells” typical to a data science project developed in a Jupyter notebook. It explains how some of Kedro's features can improve the code within your notebook.

Supercharge your data science notebook with Kedro

In the pipeline: October 2023

"In the Pipeline" is overflowing with the September's Kedro news.

In the pipeline: September 2023

In this blog post, Diego Lira explains how to use Databricks Connect with Kedro for a development experience that works completely inside an IDE. 

How to integrate Kedro and Databricks Connect

"In the Pipeline" is overflowing with the latest Kedro news, upcoming events, and interesting topics.

In the Pipeline: July 2023

This post illustrates the extensibility of Kedro with a new dataset for realtime data processing using Spark Structured Streaming. 

A new Kedro dataset for Spark Structured Streaming

Glass v9

This post explains how to use a newly-released dataset for managed Delta tables in Databricks within your Kedro project.

How to use Databricks managed Delta tables in a Kedro project

We explain how to write an efficient custom Kedro pipeline runner that continues executing a set of nodes after encountering non-catastrophic failure points.

Get up to speed: how to build a custom Kedro runner

Fourth anniversary balloon

As we commemorate the 4th anniversary of our open-source journey, we look back on some key milestones with gratitude to the Kedroid community who have supported us. 

Kedro celebrates 4 years as an open-source project

We are thrilled to announce that the Kedro brand has had a makeover. This article shares the inspiration that went into the rebrand and explores the exciting changes that lie ahead.


Unveiling Kedro's striking new branding

small square tie fighter

We've launched collaborative experiment tracking in Kedro-Viz 6.2.0 to enable a team of users to log and compare each others' experiments. This tutorial explains how to configure and use the feature.

Collaborative experiment tracking in Kedro-Viz

small square violet pringles

On World Product Day 2023, Yetunde Dada explains how to build your technical skills as a product owner to enhance your effectiveness and success in the role.


How to become a more technical product owner

small square ice dunes

Polars is an open-source library that provides fast dataframes for Python. This blog post explains how can you use Polars instead of pandas in Kedro for your data catalog and data manipulation.

A Polars exploration into Kedro

small square planet spin

Amazon EMR works with open-source big data frameworks like Apache Spark to help you tackle vast amounts of data. This post explains how to combine Amazon EMR, Kedro, and Apache Spark. 

Seven steps to deploy Kedro pipelines on Amazon EMR

small square glass waves 2

In the Pipeline: May 2023

small square exploding sugarcubes

In this blog post, we introduce a set of principles that summarise our development philosophy and steer our decision-making. 

Seven development principles for opinionated teams

small square yellow neon

We announce a new member of the Kedro Technical Steering Committee as we move towards Kedro's 4th open source anniversary.

News from the Kedro Technical Steering Committee 

small square mint knot

In recent research, we learned that Databricks is the dominant machine-learning platform used by Kedro users. We investigated aspects of our users' data science workflows to prioritise seamless Kedro usage, and we are collaborating with Databricks on our findings.

How do data scientists combine Kedro and Databricks?

This post explains how Kedro can guide an analytics team to follow best practices and avoid technical debt.

Introducing your new team lead…Kedro

small square ball and spoke

In this post, we describe recent releases to Kedro, Kedro-Viz and some new Kedro datasets. We also show off our new blog design.

A new home for the Kedro blog and some recent releases

small square mint holes

This article focuses on experiment tracking, explaining why it is valuable to data science projects and describing the features recently released for experiment tracking in Kedro projects. 

kedro

In the pipeline: July 2024

Recent blog posts

Your new Kedro project. Your way.

Exploring our new release: Kedro 0.19

All blog posts