Kedro newsletter — 5 min read

In the Pipeline: May 2023

"In the Pipeline" is overflowing with the latest Kedro news, upcoming events, and interesting topics.

4 May 2023 (last updated 20 Nov 2023)
small square glass waves 2

We're launching a new monthly blog post that'll keep you updated on all the exciting things happening in the Kedro community. From the latest Kedro news to upcoming events and interesting topics, “In the Pipeline” has got you covered.

This month: a new pair of releases, Technical Steering Committee news, upcoming events, and our top picks from recent articles and podcasts.

The latest releases of Kedro and Kedro-Viz are here

Earlier this week, Merel announced on Slack that Kedro 0.18.8 has been released.

Here are the headlines. You can see the full set of release notes on GitHub.

🚀 Major features and changes

  • Added  KEDRO_LOGGING_CONFIG environment variable, which can be used to configure logging from the beginning of the kedro process.

  • Removed logs folder from the Kedro new project template. File-based logging will remain but just be level INFO and above and go to project root instead.

  • A set of bug fixes and other changes 🪲

✍️ Documentation changes

  • Improvements to Sphinx toolchain including incrementing to use a newer version.

  • Improvements to documentation on visualising Kedro projects on Databricks, and additional documentation about the development workflow for Kedro projects on Databricks.

  • Improvements to documentation about configuration.

  • Updated table of contents for documentation to reduce scrolling.

  • And more!

Note that using kedro.extras.datasets has been officially deprecated, and will be removed from Kedro in 0.19. Installing kedro_datasets is now the preferred approach.

We would like to thank community contributors tomasvanpottelbergh and debugger24 for their input to this release.

In the last week of April, Nero announced the release of Kedro-Viz 6.1.0

Kedro-Viz is an interactive development tool for building and visualising data science pipelines with Kedro. It enables you to monitor the status of your ML project, present it to stakeholders, and smoothly bring new team members onboard. It also offers experiment tracking, and the ability to preview code and datasets.

How do I get Kedro-Viz?

  • Python: 

    pip install kedro-viz==6.0.0

  • React: 

    npm install @quantumblack/kedro-viz@latest

🚀 What can you expect in this release?

  • Experiment tracking updates allowing users to filter (show/hide) metrics in the time series & parallel coordinates metrics plots.📈

  • A set of bug fixes and other changes 🪲

You can see the full Kedro-Viz release notes on GitHub.

🔮 What's coming next?

  • Collaboration features within Kedro-Viz.

  • Create your own reports.

Technical Steering Committee news

We’ve recently welcomed @marrrcin to the Kedro Technical Steering Committee! You can read more about this fantastic news on our blog.

We’d also like to share some numbers that we collected recently:

Upcoming events

4th May 2023

The Kedro team is organising a 2-hour virtual training session on Thursday, May 4th, 2023 that is open to everyone. The session introduces you to Kedro and explains how to turn a Jupyter notebooks into reusable Python libraries. You’ll learn the benefits of Kedro pipelines and how to visualise them using Kedro-Viz in an interactive session with plenty of Q&A.

Register now to reserve your slot on 4th May 2023 at 4:00pm–6:00pm CEST (which is 10:00am–12:00pm EDT).

18th May 2023

Juan Luis, Kedro’s Developer Advocate, is giving a talk on 18th May at PyCon Lithuania. His talk is titled “Analyze your data at the speed of light with Polars and Kedro” and presents how to combine Kedro with Polars, a new dataframe library backed by Arrow and Rust, for lightning fast data manipulation and exploratory data analysis

In the pipeline: top picks from the Kedro team

  • Towards Data Science recently published a pair of nice posts by Jõao Pedro about writing a data pipeline with Airflow and AWS Tools (S3, Lambda & Glue) and automatically managing data pipeline infrastructures With Terraform

  • GetInData | Part of Xebia publish a weekly newsletter on LinkedIn called Data Pill and it hit its 50th edition this week, so celebrated with a compilation of the most popular case studies from previous editions.

  • The NerdOut@Spotify podcast is always a must-listen. It’s produced by the nerds at Spotify, and made for the nerds inside all of us. You get to hear from Spotify engineers about challenging tech problems and get a firsthand look into what they’re doing. The most recent episode is a fascinating look into building at scale.

  • Speaking of podcasts and Spotify, the R&D Engineering team recently blogged about the generation of podcast previews using Google Dataflow. The result: a neat way of providing users with audio teasers so they can make listening decisions that aren’t based just on static content, such as cover art and descriptions.

  • In last month’s virtual Kedro update meeting, we walked the community through the new OmegaConfigLoader, described user research and ongoing collaboration with Databricks, and discussed experiment tracking in Kedro-Viz. If you missed the session, you can catch up with a recording on the Kedro YouTube channel.

That’s it for May 2023

And that’s a wrap for this month. But if you can’t wait for next month’s In the Pipeline news, we also toot out regular updates onto Mastodon ( and across the popular channels of the Slack community.

Spoiler alert! Next month, we might unveil a fresh new look. But shh, let's keep it between us for now. Make sure to bookmark this blog or add our RSS feed to your favorite reader to stay in the loop and join us in the first week of June for another update from the Kedro team.

On this page:

Photo of Jo Stichbury
Jo Stichbury
Technical Writer, QuantumBlack
Share post:
Mastodon logoLinkedIn logo

All blog posts

cover image alt

Kedro newsletter — 5 min read

In the pipeline: May 2024

From the latest news to upcoming events and interesting topics, “In the Pipeline” is overflowing with updates for the Kedro community.

Jo Stichbury

7 May 2024

cover image alt

Best practices — 5 min read

A practical guide to team topologies for ML platform teams

Creating data platforms is a challenging task. A guest author explains how Kedro reduces the learning curve and enables data science teams.

Carlos Barreto

30 Apr 2024

cover image alt

Kedro-Viz — 6 min read

Share a Kedro-Viz with Github pages

We have added support to automate publishing to Github pages through the publish-kedro-viz Github Action. Learn how to configure and use the feature!

Nero Okwa

4 Apr 2024

cover image alt

Kedro newsletter — 5 min read

In the pipeline: March 2024

From the latest news to upcoming events and interesting topics, “In the Pipeline” is overflowing with updates for the Kedro community.

Jo Stichbury

12 Mar 2024

cover image alt

Kedro newsletter — 5 min read

In the pipeline: February 2024

From the latest news to upcoming events and interesting topics, “In the Pipeline” is overflowing with updates for the Kedro community.

Jo Stichbury

7 Feb 2024