The Kafka message broker, which was developed under the aegis of the Apache Software Foundation (ASF), is now available in version 3.0.0. Confluent and the community have contributed many new features and bug fixes for the current major release of the open source project, including improvements to KRaft, the self-managed metadata quorum that will be built into Kafka in the future and will replace Apache ZooKeeper. Additionally, version 3.0.0 developers should be prepared for disruptive changes in APIs.

KRaft is not quite ready for production yet

With the release of version 2.8 in the spring, the Kafka team had already given a first impression of how the message broker can be used in the future without ZooKeeper. Since then, a number of improvements have been made to KRaft’s metadata and APIs. Among other things, it is now possible to reassign partitions and KRaft offers support for Exactly-once.

In addition, KRaft controllers and KRaft brokers from version 3.0 can take snapshots for the metadata topic partition. __cluster_metadata create, replicate, and load. The Kafka cluster uses this topic to store and replicate metadata information such as broker configuration or topic partition assignments. Given the still-known shortcomings The team does not yet recommend KRaft for production use.

The list of features and components marked as obsolete (obsolete) includes Java 8, Scala 2.12, and MirrorMaker version 1. While a new LTS version of the programming language has just appeared with Java 17, Kafka users must say goodbye to Java 8. Starting with Kafka 4.0 at the latest, there will no longer be support for the version. deprecated from Java. The same goes for Scala 2.12. While MirrorMaker2 (MM2) can now be used to save offset data records on any cluster to save the source Kafka cluster as a pure read-only instance, the development team is sending MirrorMaker v1 to his advanced age. All development efforts and new functions are reserved to MM2.

Clean up Kafka Streams

As part of the for Kafka Streams in accordance with the Kafka improvement proposal KIP-740 proposed a major renovation of the class TaskId some methods and all internal fields are considered obsolete. The getters subtopology() and partition() replace this topicGroupId and partition-The fields. The proposal made KIP-744 also foresees to separate the implementation of the public API of certain classes. Interfaces TaskMetadata, ThreadMetadata and StreamsMetadata newly introduced, the previous classes with the same name are no longer applicable.

Provide a full overview of all the new features and more details on the Confluent blog announcing the new release, the intro video with Tim Berglund as well as the Apache Kafka 3.0.0 release notes.

(menu)