Advanced Kafka: Tuning and Scaling for High-Performance Systems

Master the art of tuning and scaling Apache Kafka under expert guidance. Enroll in our specialized training program tailored specifically to this course for an in-depth skill set in optimizing performance and ensuring reliability in your Kafka deployments.

Face-to-Face Nov 19-20, 2026 9:00 AM - 5:00 PM Mohammad Mehdi Lotfinejad
updated
advanced
Advanced Kafka: Tuning and Scaling
We price match

Public Pricing

MYR 3500

Corporate Pricing

Pax:

Training Fees: MYR 6500/day
Total Fees: MYR 13000 ++

Training Provider Pricing

Pax:

Training Fees: MYR 3200/day
Material Fees: MYR 800
Total Fees: MYR 7200 ++

Features

2 days
16 modules
1 intake
Full life-time access
English
Technical: 25 pax

Target Audience

engineers

Methodologies

lecture
slides
labs
case studies
q&A

Subsidies

HRDC Claimable logo

What you'll learn

  • Monitor Kafka clusters effectively using JMX metrics and manager tools.
  • Develop scalable solutions by designing from scratch with performance testing.
  • Optimize producer performance through advanced configurations and API usage.
  • Design high-availability Kafka clusters with robust security measures.
  • Enhance consumer efficiency by configuring offsets, groups, rebalancing strategies.
  • Implement advanced stream processing techniques using KStream and KTable.
  • Understand the architecture of Apache Kafka including producers, consumers, brokers, topics, partitions, and replication.
  • Install and configure Kafka using Kraft while mastering CLI commands.

Why should you attend?

In this course, you will delve into the intricacies of Apache Kafka, a powerful distributed event streaming platform. The journey begins with an introduction to big data and Kafka's architecture, covering essential components such as producers, consumers, brokers, topics, partitions, and replication. This foundational knowledge sets the stage for more advanced topics. Next, you will explore Kafka installation using Kraft, learning key concepts and mastering CLI commands. You will also gain hands-on experience with Kafka producers and consumers. As you progress, the course delves deeper into Kafka brokers and partitions, emphasizing replication, reliability, fault tolerance, and guarantees. Producer performance tuning is a critical aspect covered in detail. You will learn about producer configurations, API usage, message acknowledgements, and advanced features like idempotent and transactional producers. Similarly, consumer configurations are explored extensively to optimize performance and ensure efficient stream processing with Kafka Streams. Monitoring and tools are vital for maintaining a healthy Kafka cluster. You will discover techniques for cluster monitoring using JMX metrics and various Kafka manager tools. Real-world case studies provide practical insights into performance tuning scenarios and best practices. The course also covers Kafka cluster design for high availability and security. You will learn about deployment architectures, cluster setup, maintenance, upgrades, troubleshooting, and advanced monitoring techniques. Schema registry management is another crucial topic that ensures seamless schema compatibility and evolution. Finally, you will dive into Kafka streams processing concepts and connectors. The course concludes with implementing a scalable Kafka solution from scratch. By the end of this comprehensive training program, you will be equipped with the skills needed to design, implement, tune, monitor, and secure robust Kafka clusters.

Course Syllabus

Day 1 - Kafka Fundamentals and Setup
Module 1
Module 2
Short Break
15 mins
Module 3
Module 4
Recap and Q&A
15 mins
Lunch
1 hour
Module 5
Module 6
Short Break
15 mins
Module 7
Module 8
Recap and Q&A
15 mins
End of Day 1
Day 2 - Performance Tuning Essentials
Module 9
Module 10
Short Break
15 mins
Module 11
Module 12
Recap and Q&A
15 mins
Lunch
1 hour
Module 13
Module 14
Short Break
15 mins
Module 15
Module 16
Recap and Q&A
15 mins
End of Day 2

Instructor

Loading...
Mohammad Mehdi Lotfinejad Chief Data Officer & Data Science Trainer
Trainer Profile
Trainer Profile
TTT Certificate
TTT Certificate
Mohammad Mehdi Lotfinejad is a distinguished Chief Data Officer and certified HRDF trainer with over 15 years of experience in computer science instruction and professional data science training. Currently serving as Chief Data and Knowledge Officer at Magna.ai, a Florida-based lawtech company, he leads the development of sophisticated graph databases, data warehouses, and API solutions that power AI-driven legal case analysis systems. His expertise spans the entire data ecosystem, from architecture design to workflow automation and team leadership. With a robust background encompassing more than a decade of hands-on experience in data engineering and data science, Mohammad has successfully implemented enterprise-scale data processing pipelines across multiple industries and geographies. His professional journey includes senior roles at Axiata Digital Advertising (ADA) in Malaysia, where he designed and deployed data pipelines using AWS Redshift, Snowflake, Apache Spark, and Apache Airflow, and at The Center of Applied Data Science, where he led teams delivering training solutions to major corporations including CIMB, PETRONAS, SHELL, and TNB. Mohammad's technical proficiency is comprehensive and current, encompassing cloud platforms (AWS, Google Cloud), data warehousing solutions (Redshift, Snowflake), big data technologies (Apache Spark, Hadoop, Hive), workflow orchestration (Apache Airflow), and multiple database systems (MySQL, PostgreSQL, MongoDB). He holds a Harvard Business School certification in Business Analytics and multiple AWS certifications, including specialized credentials in Big Data, Data Warehousing, and Practical Data Science with Amazon SageMaker. As an educator, Mohammad brings exceptional depth to his training delivery. His academic career includes faculty positions at Payame Noor University and Islamic Azad University, where he served in leadership roles including Chancellor and Department Head. He has authored three technical books and published numerous peer-reviewed papers in international journals. His teaching repertoire covers data engineering, data science, machine learning, software development, and computer architecture, delivered through engaging, hands-on methodologies that bridge theoretical concepts with practical industry applications. Mohammad's unique combination of executive leadership, technical expertise, and proven training capabilities makes him an invaluable resource for organizations seeking to develop their data science and engineering capabilities. His ability to translate complex technical concepts into actionable learning outcomes, coupled with his extensive real-world implementation experience, ensures that training participants gain immediately applicable skills that drive business value.
8 Students
108 Courses
18 Years

Instructor Reviews

ML

Mohammad Mehdi Lotfinejad

Chief Data Officer & Data Science Trainer

"Mehdi and I worked on several projects with company such as Petronas , Shell and CIMB Regional ETC. I must say Mehdi's training was highly appreciated by our clients as he was able to exhibit in full display his vast knowledge as a Data professional. I would highly recommend him to anyone looking for a top tier training expert."

"Not only knowledgeable but also having hands dirty on what he knows Friendly and building networks quickly."

"I had the pleasure of working with Mehdi together on some high-level initiatives such as the Petronas data scientist program and Shell's project to become a data-driven organization. During these projects, Mehdi received numerous accolades for his ability to share his knowledge and mentor up-and-coming data scientists. Based on our shared experiences, I have no hesitation in recommending Mehdi for any project or position he may be considered for."

FAQ

Frequently Asked Questions About This Course

Why should you attend?

In this course, you will delve into the intricacies of Apache Kafka, a powerful distributed event streaming platform. The journey begins with an introduction to big data and Kafka's architecture, covering essential components such as producers, consumers, brokers, topics, partitions, and replication. This foundational knowledge sets the stage for more advanced topics. Next, you will explore Kafka installation using Kraft, learning key concepts and mastering CLI commands. You will also gain hands-on experience with Kafka producers and consumers. As you progress, the course delves deeper into Kafka brokers and partitions, emphasizing replication, reliability, fault tolerance, and guarantees. Producer performance tuning is a critical aspect covered in detail. You will learn about producer configurations, API usage, message acknowledgements, and advanced features like idempotent and transactional producers. Similarly, consumer configurations are explored extensively to optimize performance and ensure efficient stream processing with Kafka Streams. Monitoring and tools are vital for maintaining a healthy Kafka cluster. You will discover techniques for cluster monitoring using JMX metrics and various Kafka manager tools. Real-world case studies provide practical insights into performance tuning scenarios and best practices. The course also covers Kafka cluster design for high availability and security. You will learn about deployment architectures, cluster setup, maintenance, upgrades, troubleshooting, and advanced monitoring techniques. Schema registry management is another crucial topic that ensures seamless schema compatibility and evolution. Finally, you will dive into Kafka streams processing concepts and connectors. The course concludes with implementing a scalable Kafka solution from scratch. By the end of this comprehensive training program, you will be equipped with the skills needed to design, implement, tune, monitor, and secure robust Kafka clusters.


What you'll learn

  • Monitor Kafka clusters effectively using JMX metrics and manager tools.
  • Develop scalable solutions by designing from scratch with performance testing.
  • Optimize producer performance through advanced configurations and API usage.
  • Design high-availability Kafka clusters with robust security measures.
  • Enhance consumer efficiency by configuring offsets, groups, rebalancing strategies.
  • Implement advanced stream processing techniques using KStream and KTable.
  • Understand the architecture of Apache Kafka including producers, consumers, brokers, topics, partitions, and replication.
  • Install and configure Kafka using Kraft while mastering CLI commands.

Course Syllabus

Day 1 - Kafka Fundamentals and Setup
Module 1
Module 2
Short Break
15 mins
Module 3
Module 4
Recap and Q&A
15 mins
Lunch
1 hour
Module 5
Module 6
Short Break
15 mins
Module 7
Module 8
Recap and Q&A
15 mins
End of Day 1
Day 2 - Performance Tuning Essentials
Module 9
Module 10
Short Break
15 mins
Module 11
Module 12
Recap and Q&A
15 mins
Lunch
1 hour
Module 13
Module 14
Short Break
15 mins
Module 15
Module 16
Recap and Q&A
15 mins
End of Day 2

Instructor Reviews

Mohammad Mehdi Lotfinejad

Mohammad Mehdi Lotfinejad

Chief Data Officer & Data Science Trainer

"Mehdi and I worked on several projects with company such as Petronas , Shell and CIMB Regional ETC. I must say Mehdi's training was highly appreciated by our clients as he was able to exhibit in full display his vast knowledge as a Data professional. I would highly recommend him to anyone looking for a top tier training expert."

"Not only knowledgeable but also having hands dirty on what he knows Friendly and building networks quickly."

"I had the pleasure of working with Mehdi together on some high-level initiatives such as the Petronas data scientist program and Shell's project to become a data-driven organization. During these projects, Mehdi received numerous accolades for his ability to share his knowledge and mentor up-and-coming data scientists. Based on our shared experiences, I have no hesitation in recommending Mehdi for any project or position he may be considered for."

We price match

Public Pricing

MYR 3500

Corporate Pricing

Pax:

Training Fees: MYR 6500/day
Total Fees: MYR 13000 ++

Training Provider Pricing

Pax:

Training Fees: MYR 3200/day
Material Fees: MYR 800
Total Fees: MYR 7200 ++

Features

2 days
16 modules
1 intake
Full life-time access
English
Technical: 25 pax

Target Audience

engineers

Methodologies

lecture
slides
labs
case studies
q&A

Subsidies

HRDC Claimable logo

Instructor

Loading...
Mohammad Mehdi Lotfinejad Chief Data Officer & Data Science Trainer
Trainer Profile
Trainer Profile
TTT Certificate
TTT Certificate
Mohammad Mehdi Lotfinejad is a distinguished Chief Data Officer and certified HRDF trainer with over 15 years of experience in computer science instruction and professional data science training. Currently serving as Chief Data and Knowledge Officer at Magna.ai, a Florida-based lawtech company, he leads the development of sophisticated graph databases, data warehouses, and API solutions that power AI-driven legal case analysis systems. His expertise spans the entire data ecosystem, from architecture design to workflow automation and team leadership. With a robust background encompassing more than a decade of hands-on experience in data engineering and data science, Mohammad has successfully implemented enterprise-scale data processing pipelines across multiple industries and geographies. His professional journey includes senior roles at Axiata Digital Advertising (ADA) in Malaysia, where he designed and deployed data pipelines using AWS Redshift, Snowflake, Apache Spark, and Apache Airflow, and at The Center of Applied Data Science, where he led teams delivering training solutions to major corporations including CIMB, PETRONAS, SHELL, and TNB. Mohammad's technical proficiency is comprehensive and current, encompassing cloud platforms (AWS, Google Cloud), data warehousing solutions (Redshift, Snowflake), big data technologies (Apache Spark, Hadoop, Hive), workflow orchestration (Apache Airflow), and multiple database systems (MySQL, PostgreSQL, MongoDB). He holds a Harvard Business School certification in Business Analytics and multiple AWS certifications, including specialized credentials in Big Data, Data Warehousing, and Practical Data Science with Amazon SageMaker. As an educator, Mohammad brings exceptional depth to his training delivery. His academic career includes faculty positions at Payame Noor University and Islamic Azad University, where he served in leadership roles including Chancellor and Department Head. He has authored three technical books and published numerous peer-reviewed papers in international journals. His teaching repertoire covers data engineering, data science, machine learning, software development, and computer architecture, delivered through engaging, hands-on methodologies that bridge theoretical concepts with practical industry applications. Mohammad's unique combination of executive leadership, technical expertise, and proven training capabilities makes him an invaluable resource for organizations seeking to develop their data science and engineering capabilities. His ability to translate complex technical concepts into actionable learning outcomes, coupled with his extensive real-world implementation experience, ensures that training participants gain immediately applicable skills that drive business value.
8 Students
108 Courses
18 Years

FAQ

Frequently Asked Questions About This Course

Close menu