1. What is Hadoop? Explain its main components
Hadoop is a distributed computing framework for storing and processing large datasets, consisting of the Hadoop Distributed File System (HDFS) for storage and MapReduce for processing
2. What is the Hadoop Distributed File System (HDFS)? How does it differ from traditional file systems?
The Hadoop Distributed File System (HDFS) is a distributed file system designed to store large datasets across multiple machines, optimized for big data processing, and differs from traditional file systems by its ability to handle massive amounts of data with fault tolerance and scalability.
3. What is MapReduce? How does it work in the context of Hadoop?
MapReduce is a programming model and processing engine used for parallel processing of large datasets in Hadoop, where data is divided into smaller chunks, processed independently on different nodes, and results are combined to generate final output.
4. What are the advantages of using Hadoop for processing large datasets?
: The advantages of using Hadoop for processing large datasets include its ability to handle massive volumes of data, its scalability to grow with data needs, fault tolerance for data recovery, and cost-effectiveness compared to traditional solutions.
5. What is the role of NameNode and DataNode in HDFS?
The NameNode manages metadata and coordinates access to files in HDFS, while DataNodes store actual data blocks and handle read/write requests from clients in Hadoop Distributed File System (HDFS).
6. How does Hadoop ensure fault tolerance?
: Hadoop ensures fault tolerance through data replication, where it stores copies of data blocks across multiple DataNodes, allowing for automatic recovery in case of node failures
7. Explain the key differences between Hadoop 1 and Hadoop 2 (YARN).
Hadoop 1 had a single JobTracker for both resource management and job scheduling, while Hadoop 2 introduced YARN (Yet Another Resource Negotiator), separating resource management (ResourceManager) from job scheduling (ApplicationMaster), thus enabling more efficient resource utilization and supporting diverse processing frameworks.
8. What is the significance of YARN in Hadoop 2?
YARN (Yet Another Resource Negotiator) in Hadoop 2 is significant because it separates resource management from job scheduling, allowing for better resource utilization, scalability, and support for various processing frameworks beyond MapReduce
9. . What is Apache Spark? How does it compare to MapReduce?
: Apache Spark is a fast and general-purpose cluster computing system designed for big data processing. It outperforms MapReduce in terms of speed due to its in-memory computing capabilities and offers a more flexible programming model, supporting multiple types of workloads beyond batch processing.
10. What are the various deployment modes of Hadoop?
The various deployment modes of Hadoop are standalone mode, pseudo-distributed mode, and fully-distributed mode.
