Workshop online: Getting started with Kafka Connect 

Workshop date & duration: October 9, 2021, 9:30 – 14:00, 30 min break included

Trainer: Valentina Crisan

Location: Online (Zoom). 

Price: 190 RON (including VAT)

Number of places: 10


No doubt you have heard of Apache Kafka, probably the most known distributed events hub that exists nowadays in big data architectures. The fact that is currently used in companies like Apple, Microsoft and Netflix in order to store and make available in near real time billions of events it’s well known. What is not so well known many times is the fact that Apache Kafka vanilla package contains as well Kafka Connect – the framework needed in order to easily connect to different sources and sinks of data and subsequently to store and consume faster data into/from Apache Kafka. Another not so well known fact is that Kafka Connect can be used as well, besides connecting, for transforming and converting the events that we would like to store or consume in/from Apache Kafka. For example: if we would like to add a key to an event prior to saving it to Kafka – this can be done using a transform function, if you would like to convert the format of the message from JSON to AVRO you can do that using a converter class and as well the Schema Registry component (for registering the event schema). We will work on a remote environment consisting of a 3 nodes Apache Kafka cluster, thus no local installations will be required, only an SSH client to connect the remote environment.    

Content of the workshop: 

  • Apache Kafka concepts overview (just a recap mainly of the main concepts: how data is stored, how producers and consumers communicate with Kafka)
  • Kafka connect: concepts and architecture 
  • Using Kafka Connect: 
    • Connectors
      • Start a simple connector: file source and sink (understand the steps)    
      • Start a JDBC connector: configure and start mySQL source and sink connectors 
    •   Transformers
      • Add key to the events stored in Kafka using the mySQL source connector 
    • Converters 
      • Schema Registry role in Kafka ecosystem
      • Using Schema Registry to convert events content to/from AVRO format
    • Adding workers to Kafka Connect cluster 
    • Scaling up/down connectors 

The price for the workshop is 190 RON (including VAT). In order to register for this workshop it is required to complete the registration form and the payment step.

1. Complete registration form:

2. Payment Step (please follow this link)