本地文件上传至hdfs文件系统

来源:转载

将本地文件存储到Hadoop中的hdfs中去,uploadPath问本地文件的路径,hdfsFileName为存入hdfs中的文件名,一般路径的格式为:
String hdfsPath="hdfs://192.168.248.129:9000/"+hfdsFileName具体代码:
public static void save(String uploadPath,String hfdsFileName)throws IOException{long startTime = System.currentTimeMillis();//String localPath="/home/localfile/sing.mp3";String localPath = uploadPath; String hdfsPath="hdfs://192.168.248.129:9000/"+hfdsFileName;Configuration conf = new Configuration();//设定输入目录和输出目录Path inputDir = new Path(localPath);Path hdfsFile = new Path(hdfsPath);FileSystem local = FileSystem.get(conf);FileSystem hdfs = hdfsFile.getFileSystem(conf);try{FileStatus[] inputFiles = local.listStatus(inputDir);//得到本地文件列表FSDataOutputStream out = hdfs.create(hdfsFile);//生成HDFS输出流for(int i = 0;i<inputFiles.length;i++){System.out.println(inputFiles[i].getPath().getName());FSDataInputStream in = local.open(inputFiles[i].getPath());//打开本地输入流byte buffer[] = new byte[256];int bytesRead = 0;while((bytesRead = in.read(buffer))>0){out.write(buffer,0,bytesRead);}in.close();}out.close();long endTime = System.currentTimeMillis();System.out.println("time:"+(endTime-startTime));}catch(Exception e){e.printStackTrace();}}
 


分享给朋友:
您可能感兴趣的文章:
随机阅读: