Kedro newsletter — 5 min read

In the pipeline: October 2024

From the latest news to upcoming events and interesting topics, “In the Pipeline” is overflowing with updates for the Kedro community.

2 Oct 2024 (last updated 2 Oct 2024)
In the pipeline

Release news

Since our last In the Pipeline post, we have made a few releases.

We released a new version of the Kedro VS Code Extension (version 0.2.2), which integrates Kedro-Viz into your development workflow. The additions enable you to visualise and explore your pipelines directly within VS Code. Within Kedro-Vizr, you can now jump to the source code or the dataset definitions in your Kedro project by clicking on the nodes and datasets directly. While it's still in an experimental stage, we plan to add some more powerful capabilities soon, so stay tuned! You can find the full set of release notes on the Slack archive.

We also released Kedro-Viz version 10.0.0 to align with the addition to the extension. This new version also now includes Visual Pipeline Slicing to make slicing your pipelines more intuitive by doing it visually in Kedro-Viz. Select the start and end nodes and Kedro-Viz will show the corresponding kedro run command that you can drop directly in the terminal. Take a look at the documentation for more information or watch Huong’s video!

In Kedro-Viz version 10.0.0, we also added a new command (kedro viz --lite) for when you need to view and explore your Kedro pipelines without installing any project dependencies.

Last, but not least, we also made two new Kedro releases: version 0.19.7 and version 0.19.8.

In Kedro version 0.19.8 we added another improvement to the Databricks and Kedro development experience. We waved farewell to databricks_run.py and Kedro projects now come with a single entrypoint that supports the full Kedro CLI even in interactive environments such as Databricks. This means you can do python -m <package_name> run anywhere. Check out our updated Databricks deployment docs for more details.

In Kedro version 0.19.7 we made some changes so that Kedro commands are now lazily loaded to add performance gains when being run. We implemented key completion support for accessing datasets in the DataCatalog, and added Dataset and DataCatalog pretty printing.

We’ve now moved to an opt-out model for telemetry, enabling it by default without requiring prior consent. To disable it, set the DO_NOT_TRACK or KEDRO_DISABLE_TELEMETRY environment variables.

Kedro in conference

Since our last post, Merel has been presenting on behalf of QuantumBlack Labs at a few events. She was at the September 2024 Kickstart AI event, co-organized with PyData Eindhoven, where she shared awesome insights on embedding software engineering best practices into ML projects with Kedro. Her tips on making ML projects more scalable and manageable were super helpful!

Merel and Elena both presented at PyLadies London, hosted at at QuantumBlack, AI by McKinsey in mid night, where attendees explored exciting topics on AI and machine learning.

Coming up in October and November!

Merel will be at PyCon NL in Utrecht đŸ‡łđŸ‡± on 10th of October to present “Efficient Python Project Setup: Showing Cookiecutter's Potential within Kedro”. Check out the event website for more details of the speakers and presentations.

Later in November, there will be two talks about Kedro at PyCon France đŸ‡«đŸ‡· one about its use in precision agriculture and another one about its integration with MLFlow.

Hacktoberfest

It’s October already! And that means
Hacktoberfest is upon us. Hacktoberfest is a global celebration of open source that takes place during the month of October. Why not become a Kedroid notch up some contributions to Kedro or Kedro-Viz?

Find more information and a list of issues ready for you to tackle. You may also want to take a look at the guidance for contributors, and ask us if you’ve any questions, either on the individual GitHub issues, or over on Slack.

That’s it for this edition!

Don’t forget that we also make regular Kedro updates on LinkedIn, Bluesky, Mastodon, and on our increasingly popular Slack community. Keep an eye on QuantumBlack’s LinkedIn feed too!

Bookmark this blog or add our RSS feed to your favorite reader to stay in the loop and join us next month for another update.

Recently on the Kedro blog

PS


Don’t forget, if you spot Kedro in an article or video, don’t forget to share it on the #resources channel on Slack. And if you make a video about Kedro, or write an article, don’t be shy! Add it to the channel and to the “Awesome Kedro” repository so others can find it!


On this page:

Photo of Jo Stichbury
Jo Stichbury
Technical Writer, QuantumBlack
Share post:
Mastodon logoLinkedIn logo

All blog posts

cover image alt

News — 5 min read

Introducing a Kedro extension for VS Code

We're launching a Kedro extension for VS Code that offers enhanced code navigation and autocompletion.

Nok Lam Chan

1 Aug 2024

cover image alt

Kedro newsletter — 5 min read

In the pipeline: July 2024

From the latest news to upcoming events and interesting topics, “In the Pipeline” is overflowing with updates for the Kedro community.

Jo Stichbury

1 Jul 2024

cover image alt

SQL in Python — 7 min read

Streamlining SQL Data Processing in Kedro ML Pipelines

Kedro and Ibis streamline the management of ML pipelines and SQL queries within a Python project, leveraging Google BigQuery for efficient execution and storage.

Dmitry Sorokin

5 Jun 2024

cover image alt

Kedro newsletter — 5 min read

In the pipeline: May 2024

From the latest news to upcoming events and interesting topics, “In the Pipeline” is overflowing with updates for the Kedro community.

Jo Stichbury

7 May 2024

cover image alt

Best practices — 5 min read

A practical guide to team topologies for ML platform teams

Creating data platforms is a challenging task. A guest author explains how Kedro reduces the learning curve and enables data science teams.

Carlos Barreto

30 Apr 2024