Check Hadoop Eco information from CLI
The following explains how to check Hadoop Eco information in a CLI environment.
Use webhdfs
curl http://$(hostname -f):50070/webhdfs/v1/user/?op=LISTSTATUS
Check HDFS usage
You can check the usage of each data node in HDFS.
Check hdfs usage
hdfs dfsadmin -report
Check HDFS usage
| Code | Description |
|---|---|
| Present Capacity | Total available HDFS capacity |
| DFS Remaining | Remaining HDFS capacity |
| DFS Used | Used HDFS capacity |
| Live datanodes (N) | N active data nodes - You can check detailed usage for each data node |
Use WebHDFS REST API
You can check HDFS using the REST API via the curl command. For all commands, refer to WebHDFS.
Use webhdfs rest api
curl -s http://$(hostname -f):50070/webhdfs/v1/user/?op=LISTSTATUS
Using webhdfs rest api
Yarn
| Command | Description |
|---|---|
| yarn node -list | View node manager list |
| yarn node -list -showDetails | View detailed node manager information |
| yarn top | Check job status |
| yarn application -list | View application list |
| yarn application -list -appStates RUNNING | View application list based on application state |
| yarn application -status < Application ID > | Check status of < Application ID > |
| yarn application -kill < Application ID > | Terminate the < Application ID > job |
| yarn logs -applicationId < Application ID > | View logs for the < Application ID > job |
| mapred queue -list | View job queue list |
| yarn queue -status < Queue Name > | Check the current status of the < Queue Name > queue |
Check node manager
You can check the status of the node manager.
Check node manager
yarn node -list -showDetails
Check node manager
Check Job status
You can check the current operational status (vCore, memory usage) of the resource manager.
Check job status
yarn top
Check job status
View application list
You can view the list of currently running applications.
View application list
yarn application -list
View application list
| Code | Description |
|---|---|
| Application-Id | Application ID |
| Application-Name | Application Name |
| Application-Type | Application Type |
| User | User |
| Queue | Queue Name |
| State | State |
| Final-State | Final Execution State |
| Progress | Progress |
| Tracking-URL | Application Master URL |
View application logs
You can view the logs of the currently running application.
View application logs
yarn logs -applicationId <Application Id>
View application logs
View all queue list
You can view the list of all queues.
View all queue list
mapred queue -list
View all queue list
| Code | Description |
|---|---|
| Capacity | Configured available capacity for the queue |
| MaximumCapacity | Maximum capacity |
| CurrentCapacity | Current capacity |
Check queue status
You can check the status of the queue.
Check queue status
yarn queue -status <Queue Name>
| Code | Description |
|---|---|
| Capacity | Configured available capacity for the queue |
| CurrentCapacity | Current capacity |
| MaximumCapacity | Maximum capacity |
Check queue status