Bandwidth usage far exceeds files downloaded

[davridorian]

My apologies if this is somewhere in the FAQs, but I couldn't find it. I am an occasional user of BitComet, and never had this problem before.

Using BitComet 64-bit v1.31 on a fast Windows 7 machine, Ethernet cable, Netgear DG834Gv4 modem router with DGTeam Firmware, Ports properly forwarded, ADSL2+ with DL connection 16.5Mbps, UL 1020Kbps, (actual tested DL throughput approx 14.5Mbps, UL approx 800Kbps, Windows Firewall, Norton 2012 Antivirus:

I found that according to my Router Stats, I downloaded 82MB of files from a 2.45GB torrent (all the unwanted files unticked). Started with last good BC settings which used to work fine - DL 60Kbps, UL 10Kbps - but found that download was very slow so increased DL to 200Kbps. So increased again DL to 500Kbps (!) and UL to 30Kbps (on the basis DL:UL ca 15:1). Still very slow. Seems to be because was downloading lots of bytes.

Checked my monitoring software: it had recorded over 3 GB downloaded for 85MB of files!!!! 35x! (In this case it was at the Router, all LAN traffic excluded). Nothing else was running at the time. My monitoring software correlates well with bandwidth used for other stuff (e.g. TV on demand etc)

Files virus checked after downloads, nothing found.

So did some more downloads on my old PC, XP 32-bit with BitComet v0.99. Connection, settings, modem, forwarding, A-V etc etc as above. This time monitored bandwidth at the local machine. For 11MB of files, the monitor (different software to the other moniitor) showed that about 86MB had been downloaded. So, this time 8x!

The only thing that's changed recently is that I changed from ADSLMax (up to 8Mbps connection) where 60Kbps Down/10Kbps Up worked fine; to an unbundled connection at 16.5Mbps. There was (allegedly) no P2P restriction at the time I was on BitComet.

Can anyone explain this, and how to remedy it? (My montlhy FUP cap is 100GB, but if these download stats are really correct, this is not sustainable....)

[The UnUsual Suspect]

multiply all readings in bitcomet by a factor of 8 to convert kB into kb. Your ISP uses kb, as does your your monitoring software and since you reported bitcomet's settings improperly as being kb, you don't seem to realize the difference.

1kB (as measured in bitcomet) equals 8kb

You can do a google search on "bits vs Bytes" if you want to read more, or look in the forum, it's been discussed hundreds of times.

[davridorian]

Thanks. I wrote Kbps for BitComet download/upload speeds when I should have written kB/s. My apologies. I am aware that there are 8 biits in a byte, also that there are differences between decimal (1000) and binary (1024) multipliers. Bu that is a bit of a red herring.

My problem was that the actual downloaded file size was 81.8MB but the bandwidth usage was over 3GB (3000MB) to achieve that download. And the download speed was very slow, especially as my connection speed is now 4 times higher (ADSL2+) than when I previously used BitComet (ADSLMax) when it worked without that huge transfer. It's almost as though there was unrequested rubbish which was being pulled down at the same time.

Of course it could be that the monitoring software is misreporting (but unlikely - LAN traffiic is excluded, it is collecting ppp0 downtream/upstream traffic reported via SNMP at the modem router, and other tests e.g. with BBC iPlayer indicate that the monitoring software correlates with what actually comes down).

I did see some similar issues on another thread http://www.cometforu...h__1#entry47008 which similarly was blamed on bits/bytes without clear resolution,

and a more enlightened one here http://www.cometforu...h__1#entry63458 But again no clear resolution, although some ideas which may be relevant.

In the absence of any other ideas, in the first instance I'll (reluctantly) do some testing with another Client to see if that makes a difference. It's more than just bits v bytes. I will however report back in due course.

[kluelos]

Nobody's denying that it's a potential issue, it's just that it always seems to be clouded with misunderstanding -- and that when the misunderstanding is cleared away, nobody comes back to say there's still an issue.

There is a certain amount of overhead, which is usually understood and undeniably reasonable. Exactly how much overhead, nobody's really prepared to say.

Just empirically, it does seem to be a LOT higher when you're downloading a partial torrent rather than the whole torrent, but I can't say that for sure.

[davridorian]

OK, thanks. I will post back here when I have resolved this (or not).

[The UnUsual Suspect]

It's also possible you're getting some repeated corrupt data from non-bittorrent sources, Other clients cannot download from other sources so won't connect to the bad sources and not appear to have this issue, but if this is the reason for your issue, it's really not a bug with bitcomet, it only connects you to peers, it cannot control what those peers send you.

If you want bitcomet to behave like a bittorrent only client, you can disable all the "services", as well as LTseeding and emule plugin, and don't sign into comet ID. It will then behave exactly like any other bittorrent client. However in most cases this could drastically reduce your speeds, but in some rare cases it will avoid some corrupt data.

Additionally, if you look at your task specific data and hover over the downloaded amount, it should show you how much came from each type of source, if you can identify which is sending you huge amounts of bad data, you can disable that type of source on that task alone, but I know of no way to easily identify which peers are sending it and block only them. It is possible to do, but not something we could explain in a forum post as it would involve some third party tools such as wireshark, and an ip filtering program either installed on your system or in your routers firmware.

[davridorian]

Ok, thanks UnUsual Suspect, some useful ideas there. It should just be a systematic approach to exclude and include variables one by one. It all just takes so much time!

[The UnUsual Suspect]

One more thought regarding selecting certain files from within a torrent and non-bittorrent sources.

In some rare cases you may select a specific file that is only 1mb in size (for example), and it could require two or more bittorrent pieces to construct that file and if these pieces are rare, bitcomet will look outside the bittorrent swarm to find them and in some rare cases, it could be required to download one or more (unwanted) and possibly much larger files in order to get the pieces you need. For example, lets assume your file named "audio.mp3" had two bittorrent pieces, one of which also contained a small part of video.avi (50mb). It would be necessary to download the entire 50mb to verify it is intact, then discard most of it.

This is a very rare circumstance, and it is completely unavoidable, except for one very simple method...

Don't disable files in a torrent download!!!

Doing so is highly inefficient and can trigger such conditions, and although rare, even in optimal conditions when you disable a file, parts of that file still need to be downloaded and discarded.

File selection is a tool that should be used rarely, and used only when you don't intend to ever download the file you're disabling.

[The UnUsual Suspect]

Ok, thanks UnUsual Suspect, some useful ideas there. It should just be a systematic approach to exclude and include variables one by one. It all just takes so much time!

In most cases it's not necessary, but some problematic tasks may require fine-tuning to get to run efficiently. Things like LTseed, emule and others can increase your speed, or even complete an unseeded task, which no other client can, but this is a highly complex issue and bitcomet has spent a couple years perfecting the system. It runs pretty much flawless now, but does require good sources (peers) for the data to work, throw in some bad sources and you have problems.

[davridorian]

Well, there I was, just commencing a systematic exercise to eliminate variables....and on the very first effort I downloaded 78MB-worth of files in a bandwidth usage of 109MB. Close enough, given that there is always some overhead. They also came down at warp speed, unlike anything I had ever seen before. In that first effort I didn't disable anything at all. Although I did select only those only 78MB of files from within a 802MB torrent, because I didn't want the rest.

Therefore I conclude that the excessive bandwidth usage experienced 2 days ago (over 3000MB to download 82MB) was probably something like in UnUsual Suspect's post #8 above, i.e. there were bits of file all over the place in that particular torrent.

The lesson for me is that download rates need monitoring closely at the start of a session, and if the bandwidth usage is excessive, abandon and find an alternative torrent if possible.

Thanks for the various ideas/poss explanations, though. Much appreciated.

Edited February 24, 2012 by davridorian (see edit history)