AWS Data Engineering: AWS Glue, Athena, and PySpark Training
Master the art of data engineering with our comprehensive training program focused on AWS Glue, Athena, and PySpark. Under expert guidance, you'll build robust ETL pipelines and optimize performance for real-world applications. Enroll now to transform your skills in cloud-based data processing.
Public Pricing
Corporate Pricing
Pax:
Training Provider Pricing
Pax:
Features
Subsidies
What you'll learn
- Set up and configure an AWS environment including S3 Data Lake and IAM roles.
- Understand foundational concepts of data engineering within a banking context.
- Utilize AWS Glue Data Catalogs for effective data management.
- Implement PySpark operations for efficient data processing.
- Develop basic to advanced ETL jobs using AWS Glue.
- Optimize performance of Spark jobs through partitioning strategies.
- Design secure and compliant end-to-end data pipelines.
- Query large datasets using Amazon Athena effectively.
Why should you attend?
This course provides an in-depth exploration of data engineering using AWS Glue, Athena, and PySpark. It begins with foundational concepts, setting the stage for understanding the role of data engineering within a banking context. Participants will gain insights into the AWS Data Services Ecosystem and learn the differences between ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) processes. The course progresses to practical skills such as setting up and configuring an AWS environment. Learners will establish a data lake foundation using Amazon S3 and configure IAM roles for secure access management. Navigation through the AWS Console is also covered to ensure participants can efficiently manage their resources. A significant portion of the training focuses on AWS Glue Data Catalog fundamentals, where learners will explore data catalog architecture and utilize Glue Crawlers for data inspection and validation. The hands-on exercises reinforce these concepts by guiding participants through crawler creation. Further modules introduce PySpark for data processing, covering essential operations and common data preparation tasks. Participants will write basic AWS Glue ETL jobs, execute them, and troubleshoot any issues that arise. Advanced topics include PySpark transformations, performance optimization strategies, and monitoring techniques to ensure efficient job execution. The course also delves into Amazon Athena for querying large datasets directly from S3. Through hands-on labs and case studies like fraud detection pipelines, learners apply their knowledge in real-world scenarios. Finally, the course concludes with discussions on security, governance, data quality monitoring, and end-to-end pipeline design.
Course Syllabus
Day 1 - PySpark Fundamentals & Operations
Short Break
15 minsRecap and Q&A
15 minsLunch
1 hourShort Break
15 minsRecap and Q&A
15 minsEnd of Day 1
Day 2 - Advanced Analytics & Functions
Short Break
15 minsRecap and Q&A
15 minsLunch
1 hourShort Break
15 minsRecap and Q&A
15 minsEnd of Day 2
Day 3 - Production & Best Practices
Short Break
15 minsRecap and Q&A
15 minsLunch
1 hourShort Break
15 minsRecap and Q&A
15 minsEnd of Day 3
Ratings and Reviews
Instructor
Dr. Esther Suria Kala is a highly accomplished ICT consultant and corporate trainer with nearly 30 years of extensive experience in information technology, project management, and organizational development. As the founder of Angel Software Solutions, she has established herself as a versatile and dedicated professional who delivers cutting-edge training solutions across Malaysia and internationally.
Her academic credentials are exceptional, including a PhD in Psychology with a focus on Human Behaviour and Human Resources, an MBA in Finance, and degrees in both Computer Science and Human Resources. Currently pursuing a second PhD in IT Information Security, Dr. Esther demonstrates an unwavering commitment to continuous learning and staying at the forefront of technological advancement. Her educational foundation is complemented by an impressive array of professional certifications, including Microsoft Certified Trainer (MCT), CISSP, ITIL, SAP Certified Associate, Certified Cyber Security Professional, Data Scientist, AI Engineering, and ECBA certifications.
Dr. Esther's technical expertise spans an extensive range of domains, from traditional programming languages like Java, C++, and PHP to modern technologies including Python, AI coding, machine learning, RPA, and IR 4.0 microservices. She is proficient in database management systems including Microsoft SQL Server and Oracle, web development frameworks, and enterprise solutions like SAP Crystal Reports and SharePoint Server. Her knowledge extends to cybersecurity, ethical hacking, network administration, and data analytics, making her uniquely qualified to address both technical and strategic business challenges.
As a trainer, Dr. Esther excels in bridging the gap between technical complexity and practical application. She delivers comprehensive training programs for both IT and non-IT professionals, covering everything from software development and cloud computing to soft skills such as digital marketing, business analysis, customer service, and project management. Her ability to communicate effectively in both English and Bahasa Malaysia enables her to connect with diverse audiences across different organizational levels.
Her impressive client portfolio includes leading organizations such as Maybank, Telekom Malaysia, Malaysian Airlines, Tenaga Nasional Berhad, Bank Negara, Ministry of Education, Ministry of Finance, and numerous multinational corporations. This extensive experience across banking, telecommunications, government, aviation, and technology sectors demonstrates her adaptability and deep understanding of varied industry requirements. Recognized with the Outstanding Achievement Award in Information Technology from University Malaya in 2013, Dr. Esther continues to empower organizations through innovative training solutions that drive digital transformation and professional excellence.
Instructor
Instructor
FAQ
Frequently Asked Questions About This Course
Courses you may like
Why should you attend?
This course provides an in-depth exploration of data engineering using AWS Glue, Athena, and PySpark. It begins with foundational concepts, setting the stage for understanding the role of data engineering within a banking context. Participants will gain insights into the AWS Data Services Ecosystem and learn the differences between ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) processes. The course progresses to practical skills such as setting up and configuring an AWS environment. Learners will establish a data lake foundation using Amazon S3 and configure IAM roles for secure access management. Navigation through the AWS Console is also covered to ensure participants can efficiently manage their resources. A significant portion of the training focuses on AWS Glue Data Catalog fundamentals, where learners will explore data catalog architecture and utilize Glue Crawlers for data inspection and validation. The hands-on exercises reinforce these concepts by guiding participants through crawler creation. Further modules introduce PySpark for data processing, covering essential operations and common data preparation tasks. Participants will write basic AWS Glue ETL jobs, execute them, and troubleshoot any issues that arise. Advanced topics include PySpark transformations, performance optimization strategies, and monitoring techniques to ensure efficient job execution. The course also delves into Amazon Athena for querying large datasets directly from S3. Through hands-on labs and case studies like fraud detection pipelines, learners apply their knowledge in real-world scenarios. Finally, the course concludes with discussions on security, governance, data quality monitoring, and end-to-end pipeline design.
What you'll learn
- Set up and configure an AWS environment including S3 Data Lake and IAM roles.
- Understand foundational concepts of data engineering within a banking context.
- Utilize AWS Glue Data Catalogs for effective data management.
- Implement PySpark operations for efficient data processing.
- Develop basic to advanced ETL jobs using AWS Glue.
- Optimize performance of Spark jobs through partitioning strategies.
- Design secure and compliant end-to-end data pipelines.
- Query large datasets using Amazon Athena effectively.
Course Syllabus
Day 1 - PySpark Fundamentals & Operations
Short Break
15 minsRecap and Q&A
15 minsLunch
1 hourShort Break
15 minsRecap and Q&A
15 minsEnd of Day 1
Day 2 - Advanced Analytics & Functions
Short Break
15 minsRecap and Q&A
15 minsLunch
1 hourShort Break
15 minsRecap and Q&A
15 minsEnd of Day 2
Day 3 - Production & Best Practices
Short Break
15 minsRecap and Q&A
15 minsLunch
1 hourShort Break
15 minsRecap and Q&A
15 minsEnd of Day 3
Public Pricing
Corporate Pricing
Pax:
Training Provider Pricing
Pax:
Features
Subsidies
Ratings and Reviews
Instructors
Dr. Esther Suria Kala is a highly accomplished ICT consultant and corporate trainer with nearly 30 years of extensive experience in information technology, project management, and organizational development. As the founder of Angel Software Solutions, she has established herself as a versatile and dedicated professional who delivers cutting-edge training solutions across Malaysia and internationally.
Her academic credentials are exceptional, including a PhD in Psychology with a focus on Human Behaviour and Human Resources, an MBA in Finance, and degrees in both Computer Science and Human Resources. Currently pursuing a second PhD in IT Information Security, Dr. Esther demonstrates an unwavering commitment to continuous learning and staying at the forefront of technological advancement. Her educational foundation is complemented by an impressive array of professional certifications, including Microsoft Certified Trainer (MCT), CISSP, ITIL, SAP Certified Associate, Certified Cyber Security Professional, Data Scientist, AI Engineering, and ECBA certifications.
Dr. Esther's technical expertise spans an extensive range of domains, from traditional programming languages like Java, C++, and PHP to modern technologies including Python, AI coding, machine learning, RPA, and IR 4.0 microservices. She is proficient in database management systems including Microsoft SQL Server and Oracle, web development frameworks, and enterprise solutions like SAP Crystal Reports and SharePoint Server. Her knowledge extends to cybersecurity, ethical hacking, network administration, and data analytics, making her uniquely qualified to address both technical and strategic business challenges.
As a trainer, Dr. Esther excels in bridging the gap between technical complexity and practical application. She delivers comprehensive training programs for both IT and non-IT professionals, covering everything from software development and cloud computing to soft skills such as digital marketing, business analysis, customer service, and project management. Her ability to communicate effectively in both English and Bahasa Malaysia enables her to connect with diverse audiences across different organizational levels.
Her impressive client portfolio includes leading organizations such as Maybank, Telekom Malaysia, Malaysian Airlines, Tenaga Nasional Berhad, Bank Negara, Ministry of Education, Ministry of Finance, and numerous multinational corporations. This extensive experience across banking, telecommunications, government, aviation, and technology sectors demonstrates her adaptability and deep understanding of varied industry requirements. Recognized with the Outstanding Achievement Award in Information Technology from University Malaya in 2013, Dr. Esther continues to empower organizations through innovative training solutions that drive digital transformation and professional excellence.
Courses you may like
FAQ
Frequently Asked Questions About This Course
Our Offers
Academy for Trainers Academy for Trainers
Teach what you love. Abundent Academy gives you the tools you need to run your own trainings! We provide you with the platform, the students, the materials, and the support you need to succeed!
- Higher trainer payouts
- Ready-made course materials
- Student management system
- AI digital marketing assistant
Academy for Corporates Academy for Corporates
Get unlimited access to all of Abundent Academy's carefully curated courses for your team, all organized according to learning paths and roles! Perfect for companies looking to upskill their workforce and stay ahead in the tech industry.
- Carefully curated courses
- Role-based learning paths
- Team progress tracking
- Gap Identification and Analysis
Academy for Partners Academy for Partners
White-label IT training delivery for training providers. We become your behind-the-scenes delivery arm so you can say yes to more clients without hiring more trainers.
- Expand your training catalog
- 40+ expert trainers ready
- White-label delivery
- You keep client relationships
Top companies choose Abundent Academy
