我试图从公共URL下载一个大文件.起初似乎工作正常,但是1/10的电脑似乎超时了.我最初的尝试是使用WebClient.DownloadFileAsync,但因为它永远不会完成,我回到使用WebRequest.Create并直接读取响应流.
我的第一个使用WebRequest.Create的版本发现与WebClient.DownloadFileAsync相同的问题.操作超时,文件未完成.
如果下载超时,我的下一个版本添加了重试.这是奇怪的吗下载确实最终完成了1次重试以完成最后的7092个字节.所以文件下载的大小相同,但文件已损坏,与源文件不同.现在我会期望这个腐败是在最后的7092个字节,但情况并非如此.
使用BeyondCompare我发现从损坏的文件中丢失了2个字节,总共丢失了7092个字节!这个丢失的字节在1CA49FF0和1E31F380之间,在下载超时并重新启动之前.
这可能会发生什么?有关如何进一步追踪这个问题的提示?
以下是有关代码.
public void DownloadFile(string sourceUri,string destinationPath) { //roughly based on: https://stackoverflow.com/questions/2269607/how-to-programmatically-download-a-large-file-in-c-sharp //not using WebClient.DownloadFileAsync as it seems to stall out on large files rarely for unknown reasons. using (var fileStream = File.Open(destinationPath,FileMode.Create,FileAccess.Write,FileShare.Read)) { long totalBytesToReceive = 0; long totalBytesReceived = 0; int attemptCount = 0; bool isFinished = false; while (!isFinished) { attemptCount += 1; if (attemptCount > 10) { throw new InvalidOperationException("Too many attempts to download. Aborting."); } try { var request = (HttpWebRequest)WebRequest.Create(sourceUri); request.Proxy = null;//https://stackoverflow.com/questions/754333/why-is-this-webrequest-code-slow/935728#935728 _log.AddInformation("Request #{0}.",attemptCount); //continue downloading from last attempt. if (totalBytesReceived != 0) { _log.AddInformation("Request resuming with range: {0},{1}",totalBytesReceived,totalBytesToReceive); request.AddRange(totalBytesReceived,totalBytesToReceive); } using (var response = request.GetResponse()) { _log.AddInformation("Received response. ContentLength={0},ContentType={1}",response.ContentLength,response.ContentType); if (totalBytesToReceive == 0) { totalBytesToReceive = response.ContentLength; } using (var responseStream = response.GetResponseStream()) { _log.AddInformation("Beginning read of response stream."); var buffer = new byte[4096]; int bytesRead = responseStream.Read(buffer,buffer.Length); while (bytesRead > 0) { fileStream.Write(buffer,bytesRead); totalBytesReceived += bytesRead; bytesRead = responseStream.Read(buffer,buffer.Length); } _log.AddInformation("Finished read of response stream."); } } _log.AddInformation("Finished downloading file."); isFinished = true; } catch (Exception ex) { _log.AddInformation("Response raised exception ({0}). {1}",ex.GetType(),ex.Message); } } } }
以下是损坏的下载的日志输出:
Request #1. Received response. ContentLength=939302925,ContentType=application/zip Beginning read of response stream. Response raised exception (System.Net.WebException). The operation has timed out. Request #2. Request resuming with range: 939295833,939302925 Received response. ContentLength=7092,ContentType=application/zip Beginning read of response stream. Finished read of response stream. Finished downloading file.
解决方法
这是我通常使用的方法,迄今为止,对于您需要的相同类型的加载,它还没有失败.尝试使用我的代码来改变你的一些,看看是否有帮助.
if (!Directory.Exists(localFolder)) { Directory.CreateDirectory(localFolder); } try { HttpWebRequest httpRequest = (HttpWebRequest)WebRequest.Create(Path.Combine(uri,filename)); httpRequest.Method = "GET"; // if the URI doesn't exist,exception gets thrown here... using (HttpWebResponse httpResponse = (HttpWebResponse)httpRequest.GetResponse()) { using (Stream responseStream = httpResponse.GetResponseStream()) { using (FileStream localFileStream = new FileStream(Path.Combine(localFolder,filename),FileMode.Create)) { var buffer = new byte[4096]; long totalBytesRead = 0; int bytesRead; while ((bytesRead = responseStream.Read(buffer,buffer.Length)) > 0) { totalBytesRead += bytesRead; localFileStream.Write(buffer,bytesRead); } } } } } catch (Exception ex) { throw; }