Big Data Developer Course
About Course
Master the most in-demand big data skills: Hadoop, Sqoop, Hive, Spark, Scala, Cassandra, HBase, NIFI, Kafka and more
Big Data Developers are in high demand and it’s only going to increase as data and AI grows all across the various sectors. Mastering the skills needed to become a Big Data Engineer can be overwhelming. That’s why we created “The Big Data Developer Course” with the help of industry experts. This course provides an end-to-end implementation of the most in-demand Big Data skills, including Hadoop, Spark, Kafka, Cassandra, and more. With more than 40 hours of hands-on training, you’ll start with the basics and work your way up to production-level deployment, troubleshooting, and performance improvement.
We cover everything from local development to integrating with complex data sources, such as NOSQL databases, and even streaming data. Our team is available to address any questions you may have, and our tutorials are all explained with examples. By the end of this course, you’ll be a Big Data expert, ready to take on any job in the industry. Don’t miss this opportunity to join the world of Big Data!
Here is a short description of what you will be learning in this course:
Understand the world of Big Data. What is Big data and why it is important
Understand and learn the concepts behind Hadoop. Understand its architecture
Install the software and start writing code
Learn important Hadoop Commands
Learn the file formats and understand when to use each of the file formats
Dive deep into Sqoop- a tool used for transferring data between RDBMS and HDFS
Dive deep into Hive- a tool used for querying the data on HDFS
Learn Scala – a top programming language
Dive deep into Spark which is very hot in the market
Learn NOSQL Databases – Cassandra and HBase and integrate them with Spark
Work with Complex data and process them effectively
Make your code production ready and deploy them onto the cluster
Learn Apache NIFI- a powerful and scalable open source tool for data routing
Work with Streaming data
Learn Kafka and integrate it with Spark
Learn troubleshooting techniques and performance improvement tips
This is complete end-to-end implementation course and we are very proud to bring this course to you.
Enroll now at this promotional fee and join the world of Big Data !
What Will You Learn?
- Understand the architecture of Hadoop
- Understand file formats and the ability to choose the right format for a given use case
- Develop applications on local system and then deploy them into production
- Parameterize the code and make it production ready
- Import data from mysql database into sqoop. Export data from hdfs to mysql. Get a deep understanding of sqoop
- Query and analyze the data effectively using Hive.
- Get a strong understanding of hive
- Learn Scala - one of the top programming languages
- Learn basic, intermediate and Advance concepts of Spark which is very hot in the market
- Work with complex data and learn how to process them effectively
- Learn Cassandra and integrate it with Spark
- Learn HBase and integrate it with Spark
- Learn Apache NIFI
- Work with Spark Streaming - Learn about Kafka and how it integrates with Spark
- Get a good understanding of end to end big data pipeline
Course Content
Introduction to the Big Data World
What this course is about
How to make the best use of this course
PPT and Pdf notes
-
Introduction to Hadoop
00:00 -
How Map Reduce works
00:00 -
Hadoop 1.0 Architecture
00:00 -
Hadoop 2.0 Architecture
00:00 -
Hadoop 3.0 Architecture
00:00
Setting up cluster and doing hands on with Hadoop
Cloudera Software Installation
[Notes] Cloudera Software Installation
Hadoop Commands
[Notes] Hadoop Commands
Row Storage vs Column Storage
Serialized File Formats
[Notes] Serialized File Formats
Hadoop and Big Data Interview questions and Answers
Setting up cluster and doing hands on with Hadoop
-
Cloudera Software Installation
00:00 -
[Notes] Cloudera Software Installation
00:00 -
Hadoop Commands
00:00 -
Notes : Hadoop commands
00:00 -
Row storage versus Column storage
00:00 -
Serialized file formats
00:00 -
Notes : Serialized file formats
00:00 -
Hadoop and Big Data Interview questions and Answers
00:00
Sqoop
-
Sqoop Introduction
00:00 -
Sqoop Import
00:00 -
[Notes] sqoop import
00:00 -
Sqoop Multiple Mappers
00:00 -
[Notes] Sqoop Multiple Mappers
00:00 -
Import portion of data
00:00 -
[Notes] import portion of data
00:00 -
Sqoop eval and change the file delimiter
00:00 -
Notes : Eval and Change the file delimiter
00:00 -
Incremental Import
00:00 -
Notes : Incremental imports
00:00 -
Password protection
00:00 -
[Notes] Password Protection
00:00 -
Using Last Modified
00:00 -
[Notes] Using Last Modified
00:00 -
Import multiple File Formats
00:00 -
[Notes] Import multiple File Formats
00:00 -
Import multiple Tables
00:00 -
[Notes] Import multiple Tables
00:00 -
Handling Null during Import
00:00 -
[Notes] Handling Null during Import
00:00 -
Sqoop export
00:00 -
Notes : Sqoop export
00:00 -
Sqoop Performance Tuning
00:00 -
[Notes] Sqoop Performance Tuning
00:00 -
Sqoop Interview Preparation
00:00
Hive
-
Hive-Data Preparation
00:00 -
[Notes] Hive-Data Preparation
00:00 -
What is Hive
00:00 -
[Notes] What is Hive
00:00 -
Create and load a table in Hive
00:00 -
[Notes] Create and load a table in Hive
00:00 -
Hive Table Types
00:00 -
[Notes] Hive Table Types
00:00 -
Hive Partitions
00:00 -
[Notes] Hive Partitions
00:00 -
Hive Use Case
00:00 -
[Notes] Hive Use Case
00:00 -
Hive Buckets
00:00 -
[Notes] Hive Buckets
00:00 -
Schema Evolution in Hive
00:00 -
[Notes] Schema Evolution in Hive
00:00 -
Execute hive queries using a script
00:00 -
[Notes] Execute hive queries using a script
00:00 -
Working with Dates in Hive
00:00 -
[Notes] Working with Dates in Hive
00:00 -
[Notes] Working with Dates in Hive
00:00 -
Joins in Hive
00:00 -
[Notes] Joins in Hive
00:00 -
MSCK Repair
00:00 -
[Notes] MSCK Repair
00:00 -
Performance Tuning in Hive
00:00 -
[Notes] Performance Tuning in Hive
00:00 -
Hive vs SQL
00:00 -
[Notes] Hive vs SQL
00:00 -
Hive Additional Resources
00:00 -
Hive Interview Preparation
00:00
Installation for Spark and Scala
-
Installing and setting up Spark and Scala
00:00 -
[Notes] Installing and setting up Spark and Scala – Download links
00:00
Learn Scala
-
Introduction to Scala
00:00 -
Executing our First Scala Program
00:00 -
Scala Basics
00:00 -
Conditional Statements
00:00 -
Loops in Scala
00:00 -
Functions in Scala
00:00 -
Scala Class
00:00 -
Constructors in Scala
00:00 -
Scala Inheritance Introduction
00:00 -
Single Inheritance
00:00 -
Multilevel Inheritance
00:00 -
Hierarchical Inheritance
00:00 -
Scala Traits – for Multiple Inheritance
00:00 -
Hybrid Inheritance
00:00 -
Method overriding and Method Overloading
00:00 -
Singleton and Companion Object
00:00 -
Case Class
00:00 -
Abstraction and Final – Preview
00:00 -
Higher Order Functions and Lambda Expressions
00:00 -
What is Partially Applied Function
00:00 -
What is Currying
00:00 -
What is Option Type
00:00 -
Pattern Matching in Scala
00:00 -
Exception Handling in Scala
00:00 -
Scala Collections
00:00 -
[Notes] Scala Collections
00:00 -
Collection Methods
00:00 -
[Notes] Collection Methods
00:00 -
Group By vs Grouped
00:00 -
Variable Arguments – What is it and how is it useful?
00:00 -
Working with files – a Preview
00:00 -
Scala Interview Questions and Answers
00:00
Introduction to Spark
-
What is Spark
00:00 -
Why is Spark Faster than MapReduce
00:00
Spark RDDs
-
RDD Basics – Reading and Writing a File
00:00 -
[Notes] RDD Basics – Reading and Writing a File
00:00 -
Deploying code to Cluster
00:00 -
Use Case – Analyze the Log Data
00:00 -
[Notes] Use Case – Analyze the Log Data
00:00 -
Common RDD Transformations and Actions
00:00 -
What is Pair RDD
00:00 -
Use Case – The word count example
00:00 -
Using Schema RDD
00:00 -
Using Row RDD
00:00
Spark Data Frames
-
What is Spark DataFrame
00:00 -
Creating DataFrames from RDD
00:00 -
Spark Seamless Dataframe- Reading and Writing
00:00 -
[Notes] Spark Seamless Dataframe- Reading and Writing
00:00 -
Reading and Writing AVRO Data
00:00 -
Reading and Writing XML Data
00:00 -
[Notes] Reading and Writing XML Data
00:00 -
Reading Multi Lines Json
00:00 -
[Notes] Reading Multi Lines Json
00:00 -
Write Modes in Spark
00:00 -
Passing schema to a file
00:00 -
Applying Transformations using tempView and DSL
00:00