当前位置: 动力学知识库 > 问答 > 编程问答 >

multithreading - Multi Thread java downloader downloading corrupted file?

问题描述:

I am attempting to make a multiThread downloader in java. It works but the file that I download is always missing some bytes.

I have searched and found many example of multiThreaded web crawlers but nothing simple, so can anyone tell me if my method could work?

I don't know if the problem is with ordering the bytes.

I have tried BlockingQueue but it did not work

URL url = new URL(mURL);

final HttpURLConnection conn;

conn = (HttpURLConnection) url.openConnection();

conn.setConnectTimeout(10000);

conn.connect();

final BufferedInputStream in = new BufferedInputStream(conn.getInputStream());

final File f = new File("tr.exe");

if (f.exists()) {

f.delete();

}

// open the output file and seek to the start location

Thread t1 = new Thread() {

public void run() {

try {

RandomAccessFile raf = new RandomAccessFile(f, "rw");

raf.seek(0);

int numRead;

int mStartByte = 0;

byte data[] = new byte[conn.getContentLength() / 2];

while (((numRead = in.read(data, 0, 1024)) != -1) && (mStartByte < conn.getContentLength() / 2)) {

// write to buffer

raf.write(data, 0, numRead);

// increase the startByte for resume later

mStartByte += numRead;

// increase the downloaded size

}

raf.close();

} catch (FileNotFoundException ex) {

Logger.getLogger(MyDownloader.class.getName()).log(Level.SEVERE, null, ex);

} catch (IOException ex) {

Logger.getLogger(MyDownloader.class.getName()).log(Level.SEVERE, null, ex);

}

}

};

Thread t2 = new Thread() {

public void run() {

try {

RandomAccessFile raf = new RandomAccessFile(f, "rw");

raf.seek(conn.getContentLengthLong() / 2);

int numRead;

int mStartByte = conn.getContentLength() / 2;

byte data[] = new byte[conn.getContentLength() / 2];

while (((numRead = in.read(data, 0, 1024)) != -1) && (mStartByte < conn.getContentLength())) {

// write to buffer

raf.write(data, 0, numRead);

// increase the startByte for resume later

mStartByte += numRead;

// increase the downloaded size

}

raf.close();

} catch (FileNotFoundException ex) {

Logger.getLogger(MyDownloader.class.getName()).log(Level.SEVERE, null, ex);

} catch (IOException ex) {

Logger.getLogger(MyDownloader.class.getName()).log(Level.SEVERE, null, ex);

}

}

};

t1.start();

t2.start();

网友答案:

Here's the first few problems I noticed...

  • First of all I would put your download/write code in a separate class that implements Runnable then execute it with an ExecutorService.

  • byte[] data should be the same size as the chunk you're reading which you've hardcoded as 1024.

  • int mStartByte = conn.getContentLength() / 2 what if the length is an odd number?

  • As you have 2 threads reading from same stream you are not guaranteeing that eg. 1st thread reads first half, 2nd thread reads second half. I suspect it won't work for files over 2048 bytes. Within each read iteration you need to know where in the stream you are starting so you can seek to the right place.

网友答案:

what is happening as i gues, is that you are writing to the same file at the same time, then yes the order change, and u may even lose some bytes in the way, now what you should do, is write the output of the first thread of a file called for example 'Thread1Output' and the second to 'Thread2Output' and at the end assemble the two files in one file, hope it works for u, and good luck, sorry, i cant provide code cause i have never create a downloader =D

分享给朋友:
您可能感兴趣的文章:
随机阅读: