Next Generation Apache NiFi

Next Generation Apache NiFi | NiFi 2.0.0 is GA

by | November 4, 2024

Apache NiFi, Big Data

Apache NiFi is about to turn 10 years old as an Apache Software Foundation (ASF) project and it is in use by over 8,000 enterprises around the globe. No better time for this incredibly flexible and powerful framework to finalize its 2.0.0 version. Welcome to the Next Generation Apache NiFi.

Over 2000 Jira issues and 4 milestone releases later, today’s announcement declares the official release of Apache NiFi 2.0.0. Datavolo’s CEO, Joe Witt, shared the following with our team: 

“When Datavolo was founded we leaned into the NiFi 2.x line exclusively as it offers a far better direction for cloud native deployments, far stronger security, and we intentionally invested in the community in a number of other areas, not least of which includes a completely rebuilt from the ground up UI using modern front-end libraries and frameworks.”

The project release notes for 2.0.0 are here. Listed below are some of the most important features, enhancements, and changes in this significant release.

Python support

The Python API for Processors allows Python programmers to build custom Processors for NiFi and MiNiFi. Try it out for yourself with our Build a NiFi Python transform processor hands-on tutorial.

Kubernetes

While NiFi can continue to be deployed as a bare-metal solution, 2.0 now allows for native k8s integration which removes the need for ZooKeeper. Visit our Constructing Apache NiFi Clusters on Kubernetes blog post for more information. 

Rebuilt UI

An entirely newly implemented UI that remains faithful to the core of the existing user experience but brings many improvements.  It provides a far more modern stack to build on and includes various niceties such as a dark mode.

Stateless flows

Developers can define the transactional boundary for processing data. If a failure occurs, the entire transaction can be rolled back. The is a replacement for the previous ExecuteStateless Processor.

Tighter Git integration

Flows can now use a Flow Registry client where the ‘registry’ is simply a GitHub or GitLab repository. This can simplify SDLC processes and now allows support for development branches. Check out our hands-on Versioning NiFi flows with GitHub tutorial, too.

S3 enhancements

Additional Processors have been added to allow retrieval of S3 metadata and copying of files between buckets.

Kafka 3 support

Support for Kafka 3 in both consumption and publishing.

Framework updates

NiFi 2.0 upgraded underlying frameworks to modern versions such as Java 21, Spring 6, Jetty 12, Servlet 6, Angular 18, and OpenAPI 3.

Summary

New features, modern frameworks, first-class Python support, and native k8s integration are just a few of the reasons that the next generation of Apache NiFi will be around for many years to come. Pop the cork and as members in the greater Apache NiFi community, let’s all drink our own champagne!

Next Generation Apache NiFi

Top Related Posts

Continuous Integration for NiFi Flows in GitHub

Datavolo is proud to announce the release of a GitHub Action designed to help with Continuous Integration of Apache NiFi Flows and make reviewing of changes between two flow versions as easy as possible. At Datavolo, collaboration on the Flow Definitions is done by...

Apache NiFi frontend modernization complete

Apache NiFi's 2.0.0 release included several upgrades that make the platform faster, more secure, and easy to use. One thing that really stands out to us, however, is how transformational Apache NiFi's frontend modernization really is. The platform has been redesigned...

Streaming Data to Iceberg From Any Source

New support for writing to Apache Polaris-managed Apache Iceberg tables enables Datavolo customers to stream transformed data from nearly any source system into Iceberg. Originally created by Snowflake, Polaris allows customers to use any query engine to access the...

How we use the Kubernetes Operator pattern

Organizations using NiFi for business-critical workloads have deep automation, orchestration, and security needs that Kubernetes by itself cannot support. In this second installment of our Kubernetes series, we explore how the Kubernetes Operator pattern alleviates...

Constructing Apache NiFi Clusters on Kubernetes

Introduction Clustering is a core capability of Apache NiFi. Clustered deployments support centralized configuration and distributed processing. NiFi 1.0.0 introduced clustering based on Apache ZooKeeper for coordinated leader election and shared state tracking. Among...

Onward with ONNX® – How We Did It

Digging into new AI models is one of the most exciting parts of my job here at Datavolo. However, having a new toy to play with can easily be overshadowed by the large assortment of issues that come up when you’re moving your code from your laptop to a production...

Survey Findings – Evolving Apache NiFi

Survey of long time users to understand NiFi usage Datavolo empowers and enables the 10X Data Engineer. Today's 10X Data Engineer has to know about and tame unstructured and multi-modal data. Our core technology, Apache NiFi, has nearly 18 years of development,...

Apache NiFi – designed for extension at scale

Apache NiFi acquires, prepares, and delivers every kind of data, and that is exactly what AI systems are hungry for.  AI systems require data from all over the spectrum of unstructured, structured, and multi-modal and the protocols of data transport are as varied...