我无法使用
Java从HDFS读取文件：
String hdfsUrl = "hdfs://<ip>:<port>";
Configuration configuration = new Configuration();
configuration.set("fs.defaultFS",hdfsUrl);
FileSystem fs = FileSystem.get(configuration);
Path filePath = new Path(hdfsUrl + "/projects/harmonizome/data/achilles/attribute_list_entries.txt.gz");
FSDataInputStream fsDataInputStream = fs.open(filePath);
SEVERE: Servlet.service() for servlet [edu.mssm.pharm.maayanlab.Harmonizome.api.DownloadAPI] in context with path [/Harmonizome] threw exception
java.lang.IllegalArgumentException: Wrong FS: hdfs://146.203.54.165:8020/projects/harmonizome/data/achilles/attribute_list_entries.txt.gz,expected: file:///
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:310)
at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:47)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:357)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:356)
at edu.mssm.pharm.maayanlab.Harmonizome.api.DownloadAPI.readLines(DownloadAPI.java:37)
at edu.mssm.pharm.maayanlab.Harmonizome.api.DownloadAPI.doGet(DownloadAPI.java:27)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
...
我没有设置我们的HDFS,所以我不知道我不知道什么.任何帮助表示赞赏.
解决方法
试试这个：
Configuration configuration = new Configuration(); FileSystem fs = FileSystem.get(new URI(<url:port>),configuration); Path filePath = new Path(<path/to/file>); FSDataInputStream fsDataInputStream = fs.open(filePath); BufferedReader br = new BufferedReader(new InputStreamReader(fsDataInputStream));
请参考http://techidiocy.com/java-lang-illegalargumentexception-wrong-fs-expected-file/
解决了类似的问题.
