Miss Zhu 2022-02-13 08:39:14 阅读数:123
This article aims to complete a command-line version of HDFS Control procedures , Can be in Windows or Linux Up operation , As shown in the figure below :
1. Create a new project , stay src Under the new service package 、view package , There are two classes under the package as shown in the following figure .
2. newly build lib Folder , Will need jar Copy the package in , And added to the project java bulid path in
Be careful : The above two steps , You can also copy TestHDFS And come
3. Write code , According to the previous article TestHDFS Class code , reform HDFSService class :
HDFSService.java:
package hdfs;
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
public class HDFSService {
FileSystem fs;
public HDFSService() {
try {
URI uri = new URI("hdfs://192.168.150.21:9000");
Configuration conf = new Configuration();
// conf.set("dfs.replication", "2");
// conf.set("dfs.block.size", "64m");
conf.set("fs.hdfs.impl", "org.apache.hadoop.hdfs.DistributedFileSystem");
fs = FileSystem.get(uri, conf, "root");
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (URISyntaxException e) {
e.printStackTrace();
}
}
public boolean put(String local, String hdfs) {
boolean flag = false;
try {
fs.copyFromLocalFile(new Path(local), new Path(hdfs));
flag = true;
} catch (IllegalArgumentException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return flag;
}
// Configuration required for download hadoop_home
public boolean get(String local, String hdfs) {
boolean flag = false;
try {
fs.copyToLocalFile(new Path(hdfs), new Path(local));
flag = true;
} catch (IllegalArgumentException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return flag;
}
// close resource
public void close() {
try {
fs.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
Use JDK Provided Scanner Class implements console input , collocation System Output , Realize a simple command-line interactive interface .
HDFSView.java:
package view;
import java.util.Scanner;
import service.HDFSService;
public class HDFSView {
public static void main(String[] args) {
HDFSService service = new HDFSService();
service.init();
Scanner sc = new Scanner(System.in);
System.out.println(" Welcome to HDFS Command line operating system ");
while (true) {
System.out.println("1. Upload 2. download 0. sign out ");
int type = sc.nextInt();
if (type == 1) {
System.out.println(" Please enter the local path :");
String local=sc.next();
System.out.println(" Please enter HDFS route :");
String hdfs=sc.next();
boolean flag=service.put(local, hdfs);
if (flag) {
System.out.println(" success !");
} else {
System.out.println(" Failure !");
}
} else if (type == 2) {
System.out.println(" Please enter the local path :");
String local=sc.next();
System.out.println(" Please enter HDFS route :");
String hdfs=sc.next();
boolean flag=service.get(local, hdfs);
if (flag) {
System.out.println(" success !");
} else {
System.out.println(" Failure !");
}
} else if (type == 0) {
break;
}
}
service.close();
}
}
4. function :
windows effect :
become involved jar Packages uploaded to linux It also runs normally on .
5. restructure
Function test successful , But the code quality is not good , As shown in the figure below , about View Upload and download operations in , Only the code in the red box is different , The rest get the path 、 The prompt is the same . If there is a lot of repetitive code in the project , It will have an adverse impact on future project maintenance , We need to refactor the code .
The so-called refactoring is to ensure that the code has realized the function , Rearrange , Make the code quality meet better requirements .
After the reconstruction HDFSView.java The code is as follows , In fact, it is a common method to extract the input path and prompt information :
package view;
import java.util.Scanner;
import hdfs.HDFSService;
public class HDFSView {
static Scanner sc = new Scanner(System.in);
static String local;
static String hdfs;
public static void input() {
System.out.println(" Please enter the local directory :");
local = sc.next();
System.out.println(" Please enter HDFS Catalog :");
hdfs = sc.next();
}
public static void showMes(boolean flag) {
if (flag) {
System.out.println(" success !");
} else {
System.out.println(" Failure !");
}
}
public static void main(String[] args) {
HDFSService service = new HDFSService();
System.out.println(" Welcome to HDFS Control procedures !");
while (true) {
System.out.println(" Please select the menu :1. Upload 2. download 3. sign out ");
int menu = sc.nextInt();
if (menu == 1) {
input();
boolean flag = service.put(local, hdfs);
showMes(flag);
} else if (menu == 2) {
input();
boolean flag = service.get(local, hdfs);
showMes(flag);
} else if (menu == 3) {
break;
} else {
System.out.println(" Please enter a valid menu item !");
}
}
System.out.println(" End of program running !");
}
}
copyright:author[Miss Zhu],Please bring the original link to reprint, thank you. https://en.javamana.com/2022/02/202202130839120533.html