web seeds wasting data

[showstoppre]

Hi

I ve noticed this behaviour in many torrents with web seeds where downloaded size(1.89 gb) is far more than tasksize(966 mb). And also there is no dropped data in the below example. Im using Bitcomet 1.68, but this issue existed in previous versions as well. This torrent has pretty much no seeds and only downloaded data from web seeds except for 6 MB.

eps downloaded - 8-13

 

There is also the issue of dropped data when d/lding from webseeds. Usually the large part of the dropped data is from the "first" file of the torrent. But I've also noticed some same pieces that keep dropping that are not from first file of the torrent. Please find below the sample torrent. First file size is 214 mb(1 mb per piece)

 

Any help would be appreciated. Please let me know if the torrent file/magnet link is needed

Update: FYI - even though consecutive pieces are dropped, I was not using  "Sequential Download" . 

Finished task screenshot

Thanks

 

Edited June 30, 2020 by showstoppre
Update (see edit history)

[showstoppre]

Is there any update on this. Is it just me having these issues?

I m happy to share the magnet if some one could help test it

 

[Rhubarb]

I avoid using BC for web transfers (just a metter of preference) but what is probably happening is that the dropped packets are being counted in the total download, i.e., if you are downloading a 10 MB file and you have 50% dropped packets, the total download witll be 15 MB (10 MB for the actual file and 5 MB 'rejected' data packets)

[showstoppre]

Thanks for trying. Just to be clear though, the bug is with "Torrents with web seeds" and not web transfers in general. There are two scenarios, one is that no dropped data is indicated, but the downloaded size is larger than task size.(Again, if no data is dropped why is the download size high). Another issue is that,  BC is dropping a lot of pieces when downloading from web seeds, especially from the first file of the torrent. The file never gets completed. 

You might have a different opinion or different result  if you could try downloading the torrent I posted above. This torrent has no peers with working web seeds. I dont think BC is designed to drop data from web seeds and it must be a bug.

I hope this gets reported to devs and they can take a look into it.

 

[Rhubarb]

Web seeds are downloads from the web - NOT dedicated BT files - hence the problem with dropped packets

[showstoppre]

I'm sorry but my questions are not really answered

So is it or is it not a bug? Is BC intentionally dropping data/pieces? Any idea why BC is dropping data? Or why is it not reporting dropped data as in the first case where there is no reported dropped data in log and the downloaded size is higher than the task size? Is it really dropping data in the first case?

The torrent creator has a active web server and configured web seeds in his torrent. It is a DEDICATED source for this torrent. 

This is a bug. 

- Scenario 1 - No dropped data( 0 B)  in first screenshot. downloaded size(1.89 gb) is far more than tasksize(966 mb). Question: If there is no dropped data, why did BC download more data. Where did the extra data go to.

-Scenario 2 - Dropped data(672.8 MB). Downloaded size (2.90 GB).Refer screenshot 4. Downloaded pieces failed in hash check in task log (Refer screenshot 3). Now this makes one think IF the web server is sending bad data. Nope it's  not. Other torrent clients are doing a better job and files are downloaded with no wastage of data. Question: Why did the pieces fail in hash check when the server is not sending bad data.

Point noted on your suggestion on avoiding BC. As a matter of preference, I intend to use BC for all torrents irrespective of whether they have web seeds or not. 

I'm happy to have helped identify a bug in my preferred client. It is upto the devs to fix the bug or leave it as is. But I hope the bug is highlighted to the dev team and hear what's their say on this.

 

[showstoppre]

Quoting @The UnUsual Suspect from thread https://www.cometforums.com/topic/12798907-lt-seeds-waste-too-much-data/

"if it is a bitcomet problem and it can be reproduced, we will forward it to development."

Steps to reproduce the bug - Please find the magnet below.

magnet:?xt=urn:btih:FCBPCM5PPCY6TNH3UELAWP36O7D3IRXO&dn=%5BHi10%5D_Hataraku_Maou-sama%21_%5BBD_1080p%5D_%5BDual_Audio%5D&tr=http%3A%2F%2Fanisaishuu.de%3A2710%2Fannounce&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.openbittorrent.com%2Fannounce&tr=udp%3A%2F%2Fopen.demonii.com%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.publicbt.com%2Fannounce&tr=udp%3A%2F%2Ftracker.istole.it%3A80%2Fannounce&xl=2357080564
 

1 - Uncheck all files, download only eps 8-13. Observation - by the time of completion, task size is 966 mb while the downloaded size is far higher than the task size with relatively low or no "dropped data" when you hover on "Download Size" in summary tab. Task log doesn't help either as there is no dropped data

2 - Remove torrent, checked all files and download the torrent - During the download, you can notice multiple error messages as "Downloaded pieces failed in hash check". Downloaded size is higher than task size. With stats showing dropped data.

 

Please note that this is not just one torrent I m having issue with. This happens in all torrents with web seeds. This is merely an example.

 

 

[Rhubarb]

TUS referred to a 'magnet' problem - you have a web problem.  It's not the same thing (that's why apples don't taste like bananas - they're different). The server doesn't 'send bad data' - it gets corrupted as it passes through various nodes for a lot of different reasons

AsI've said before, what is happening is that the web downloads is counting the number of bytes RECEIVED - NOT the number that can be used. Consider a 256 byte data packet - due to corruption, lag or whatever, you only receive 250 bytes. The program will reject that and request a resend but it will add the bytes from the faulty packet to its total. When AX25 was the main standard for use, you could SEE the dropped packets - with the present interface, they're hidden, but they are still there - the basic principles haven't changed.

It's not a bug in BC- it's something inherent in web based data transfer

[showstoppre]

If it is not a bug in BC, then why are other clients downloading the data without wastage. 

The is not a "web problem"

@The UnUsual Suspect Requesting your support on this to route it to dev. Please find the magnet link and steps to reproduce.

 

 

 

[Rhubarb]

Who says they do it? BC REPORTS the rejected packets - other apps may or may not do that.

It is plainly and simply a web issue (that has been known since the very early days even before the Web that packets can (and do) get corrupted en route. What you are complaining about is something that happens all the time - and BC is simply telling you that it happened.

[showstoppre]

Why are you so fixated on it being a "web problem". Please see for yourself before coming to a conclusion. 

I'm giving you as much as details I can and you don't even bother checking.

Try for yourself with other clients and then with BC.

I'm not looking for an explanation

Be it as it may, route it to the Dev team.

 

 

 

[Rhubarb]

I'm not fixated on anything - I called it a 'web problem' because that's exactly what it is. BitComet reports the number of 'lost' bytes due to packet loss/corruption.

I've been running packet software for years - both on Ham Radio and ARPNET (from before the WWW came into being) and what you are describing is simply that - CORRUPT PACKETS

It is NOT a Bit Comet error - if there is any error in the app it's because it is showing the number of discarded bytes  and that is NOT a problem or a bug. If you look at your screen dump from earlier you will see that it says that the hash check failed and a megabyte was dropped - that is a corrupt packet. If half your packets fail then you can expect it to show double the size of the actual file

Now whether or not any other app reports or doesn't report 'lost' bytes is immaterial - Bit Comet DOES and that is by design and not a 'bug'

[showstoppre]

I'm really sorry if my previous comments were rude.

All I'm saying is you may be right. But you may also be wrong.

If possible, can you just try the magnet link I shared on other bittorrent clients and then on BC. 

In bitcomet, the torrent never gets completed as it keeps dropping pieces.

Other client I tried was Qbittorent which don't have issue with downloading

Look its simple math.

Avg Download Speed x Task Run Time = Downloaded Size

Now with other clients, there is no loss of data. 

You wouldn't know the difference without trying.

Now please don't tell me, other clients might be reporting their Avg.Download Speed and Task Run time incorrectly.

😞

[Rhubarb]

Actually it IS simple math - 50 MB file with 25 MB rejected packets = 75 MB downloaded. It's nothing to do with the speed or run time but all to do with the accuracy of the data

[showstoppre]

Dude. Really. Other clients don't drop data from web seeds  ffs.

Are you really the BC tech supp?

Why do you think you are SO RIGHTEOUS

Why do you not bother verifying with BC and other client to actually see if what I am saying might be true

 

 

 

Edited July 14, 2020 by showstoppre (see edit history)

[Rhubarb]

Obviously you are going to believe what you want to believe. The fact that your own screen dump is showing rejected packets 'has' to be a bug in the software beggars belief.

If you aren't happy with the app, please feel free to use another one (that doesn't tell you when packets are being rejected)

I don't have problems with using magnet files (I do prefer to avoid them as I can't be sure of the trackers, but that's a different matter

)

[showstoppre]

It is not what I believe. I'm presenting you the fact which you turned a blind eye and chose to ignore

I am content with app and I like it and want to use it. That is the main reason I'm here trying to get across a BUG that I identified to the developers.

Unless we have a third man here, this won't go anywhere.

6 minutes ago, Rhubarb said:

don't have problems with using magnet files (I do prefer to avoid them as I can't be sure of the trackers, but that's a different matter

Now what does it have to do with magnet files huh?. I just wanted to share the torrent info one way or the other. So I shared the magnet link.

It could have been anything. magnet link, hash info,.torrent link or a link to the .torrent

 

 

 

 

Edited July 14, 2020 by showstoppre (see edit history)

[Rhubarb]

You brought up magnet links - but, for hopefully the very last time: IT IS NOT A BUG

You are getting corrupt packets - that's nothing to DO with Bit Comet and all to do with the transfer from the other stations to you. I had a similar occurence earlier this week - we had a momentary power glitch and, when the computer restarted, an incoming packet was incomplete, so it restarted it - I then showed a download of 25.6 GB for a 25.4 GB file - the break in reception was the cause of the damaged packets.

The point is that I knew why it happened only because BCshowed it

[showstoppre]

 

Why not try downloading the torrent shared on BC and other client and then see the difference for yourself.

Please don't tell me you don't have to because you know so.

In case you missed the link

magnet:?xt=urn:btih:FCBPCM5PPCY6TNH3UELAWP36O7D3IRXO&dn=%5BHi10%5D_Hataraku_Maou-sama%21_%5BBD_1080p%5D_%5BDual_Audio%5D&tr=http%3A%2F%2Fanisaishuu.de%3A2710%2Fannounce&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.openbittorrent.com%2Fannounce&tr=udp%3A%2F%2Fopen.demonii.com%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.publicbt.com%2Fannounce&tr=udp%3A%2F%2Ftracker.istole.it%3A80%2Fannounce&xl=2357080564

 

 

 

 

 

Edited July 14, 2020 by showstoppre (see edit history)

[Rhubarb]

What you are complaining about is that the hash check is actually doing what it's supposed to do - checking the incoming packets against the checksum and askingfor a re-transmission of the data.

Would you prefer that it didn't work and you end up with a corrupt file?

[showstoppre]

I really find it funny that you don't understand my point.

The point is other clients are better at handling torrents with webseeds while BC has some issues which I highlighted in the first post.

Now how would you know the difference anyway. It is not like you tested the link I shared in BC and any other torrent client of your choice.

Really. I'm laughing so hard right now.

[showstoppre]

I've given some thoughts and trying to keep it simple and gonna explain like you are five 🙂

See there is a torrent file.

You put that file in some torrent client other than BC(let's say qbittorrent). 

You start the torrent. After sometime the download gets finished.

Now you put the same file in BitComet.

You start the torrent.

And you wait

.

.

.

.

.

.

The torrent downloads gets completed after a while

Lets compare the stats

------------------------------------------------------------------------

Other torrent client

task size - 60 mb

download size - 60 mb

task duration - 1 min

avg download speed - 1mbps

--------------------------------------------------------------------------

BitComet

task size - 60 mb

download size - 120 mb

task duration - 2 min

avg download speed - 1mbps

-------------------------------------------------------------------------

if you still don't see the difference, that just tells you your comprehension

i start to wonder. "Oh is all my torrent downloads affected the same way". Then I realize, "It is just the ones with webseeds"

[Rhubarb]

What I can't understand is why you assume that data isn't being corrupted and the hash check is doing exactly what it is SUPPOSED to do

Rejected packets are counted in the overall download count  and that is NOT the problem with the app. It's working exactly as it should

What is your MTU set at and are you running wifi or cable connection?

You can check your optimal MTU here

[showstoppre]

I cannot explain any simpler than my previous post.

Look why do wanna make things complicated.

Other torrent client

task size - 60 mb

download size - 60 mb

task duration - 1 min

avg download speed - 1mbps

--------------------------------------------------------------------------

BitComet

task size - 60 mb

download size - 120 mb

task duration - 2 min

avg download speed - 1mbps

How do you explain this???????

[Rhubarb]

If anyone is complicating a simple procedure - it's not me, LOOK at what your screen dump says "Downloaded piece #100 failed in hash check 1024 KB dropped"

For hopefully the last time:

Client downloads a packet

Client hash checks the packet

Hash check doesn't match the checksum

Client rejects packet

Client requests a resend of data

That's what is SUPPOSED to happen.

Now whether or not another client will tell you yhe overheadd of rejected packets, I don't know - but that's what you are seeing in Bit Comet - a count of packets rejected.