Kedro newsletter — 5 min read

In the Pipeline: July 2023

"In the Pipeline" is overflowing with the latest Kedro news, upcoming events, and interesting topics.

12 Jul 2023 (last updated 11 Aug 2023)
In the pipeline

From the latest news to upcoming events and interesting topics, “In the Pipeline” is overflowing with interesting updates for the Kedro community.

This month: a roundup of June’s Kedro news, some release updates, and our top picks from recent articles.

Kedro news

June was an exciting month for Kedro as we celebrated four-years as an open-source project. In a blog post, we looked back to the launch in June 2019 and reflected on the milestones we have passed since then. Part of our celebration unveiled Kedro's new branding, with changes to the identity that align it with our vision of simplicity and collaboration.

Besides looking back over our achievements to date, we also looked forward, with the news that Kedro will sit at the heart of the QuantumBlack Horizon, an integrated and flexible suite of AI development tools announced in June 2023.

We love this photo of the Kedro team, taken at our open-source anniversary celebrations

We want to express our gratitude to the community, contributors, and users who have supported and shaped our software over the last few years. As we enter a dynamic and exciting period in the field of data science and machine learning, we can together embrace the future and strive to innovate and create lasting impact.

Release news

There have been a few releases since the last issue of In the Pipeline.

We’re working to a new release cadence so Kedro versions 0.18.9, 0.18.10 and 0.18.11 have all shipped in recent weeks.

The headline news is that we have reworked the micropackaging workflow to use standard Python packaging practices. We made a batch of changes for Kedro’s rebranding and some additions for OmegaConfigLoader (such as support for variable interpolation in the catalog and some bugfixes). In those new releases, we also added substantial improvements to our deployment documentation. In particular, we’ve added a brand new set of pages about working with Kedro and Databricks.

Mirroring the cadence of Kedro framework, we’ve also released released a number of updates to Kedro-Viz; versions 6.2.0, 6.3.0, 6.3.1 and 6.3.2 have been released.

The headline news in Kedro-Viz 6.2.0 was the addition of collaborative experiment tracking so multiple users can store their experiment data in centralised remote storage and share it. For more details, check out the documentation and blog post. Kedro-Viz 6.3.0 introduced the new Kedro brand, support for a new way to configure data layers, and data preview. The later minor releases have fixed bugs and made small improvements.

Finally, kedro-datasets versions 1.3.0, 1.4.0, 1.4.1 and 1.4.2 were also shipped in recent weeks. We introduced a SparkStreamingDataSet and a ManagedTableDataSet for managed delta tables on Databricks, plus pandas 2.0 support, and SQLAlchemy 2.0 support (dropping support for versions below 1.4). We also fixed some bugs and tidied up some docstrings. Check out the full description available in the release notes and the list of datasets in our documentation. It's worth pointing out that kedro-datasets is a separate PyPI package for Kedro datasets and that kedro.extras.datasets is deprecated (and will be removed in Kedro 0.19 when that release arrives later in 2023).

Recently on the Kedro blog

What we’ve learned

We asked on one of our community Slack channels where Kedroids get their news and updates. Some of the useful resources shared include these newsletters:

  • The Gradient — collects articles, interviews, and news coverage about AI.

  • The Sequence — a weekly digest about developments in machine learning, artificial intelligence, and data science.

We also revisited some blog posts about Kedro as we updated our fork of the awesome-kedro GitHub repo to collect articles, videos and podcasts about Kedro. We liked “Deploying a recommendation system the Kedro way” and “From concept to production in 2 months: sales forecasting Machine Learning model for dema.ai” by GetinData | Part of Xebia.

If you have an article about Kedro that should be on the awesome-kedro list, please do add it!

Coming up soon

Over the next few weeks, we hope to take delivery of a big bundle of Kedro swag that showcases our new brand. Look out for opportunities to collect stickers, t-shirts and more! We also expect to have news of upcoming Kedro releases and some virtual meetings in the coming month.

Kedroid swag is on the way!

That’s it for this edition

And that’s a wrap for this month. Make sure to bookmark this blog or add our RSS feed to your favorite reader to stay in the loop and join us in the first week of August for another update from the Kedro team.

If you can’t wait for next month’s In the Pipeline news, we also toot out regular updates onto Mastodon (https://social.lfx.dev/@kedro) and across the popular channels of the Slack community. Let us know on Slack if there's something you'd like us to feature on this blog or newsletter!


On this page:

Photo of Jo Stichbury
Jo Stichbury
Technical Writer, QuantumBlack
Share post:
Mastodon logoLinkedIn logo

All blog posts

cover image alt

Kedro newsletter — 5 min read

In the pipeline: February 2024

From the latest news to upcoming events and interesting topics, “In the Pipeline” is overflowing with updates for the Kedro community.

Jo Stichbury

7 Feb 2024

cover image alt

Ibis — 10 min read

Building scalable data pipelines with Kedro and Ibis

From production-ready to production. Bring the flexibility and familiarity of Python, and the scale and performance of modern SQL, to Kedro pipelines.

Deepyaman Datta

29 Jan 2024

cover image alt

Kedro news — 10 min read

Your new Kedro project. Your way.

We've made changes to Kedro in the new 0.19 release to tackle one of the most commonly perceived pain points. Find out more!

Jo Stichbury

24 Jan 2024

cover image alt

Kedro news — 4 min read

Learn Kedro from a new video course

Check out the official video course to teach you the skills you need to create data and machine learning pipelines for future-friendly, platform-agnostic data science code

cover image alt

Kedro news — 5 min read

Exploring our new release: Kedro 0.19

This blog post gives details of the enhancements and improvements to Kedro in the recent 0.19 release.