Large files on non-NTFS partition

Malgoth · January 8, 2010, 3:28pm

Got an issue when trying to download large files with Bitcomet 1.17. When opening a torrent file and choosing destination directory is says: “File to be downloaded is larger than 4GB, please modify your destination location to NTFS…”. Okey-dokey! I know that FAT32 doesn’t support files larger than 4GB, but the thing is I’m using an EXT3 (for network media player) formatted HDD partition which is connected to Windows XP OS via Ext2Fsd utility. Windows recognizes it correctly (like an ordinary FAT32 or NTFS local drive in “My Computer”) and says it is EXT3 partition in its properties.

SO, it would be really nice if in such cases (file(s) >4GB in torrent) Bitcomet could check whether the destination file system really is FAT, FAT32 or any other FS that doesn’t support large files and only then give an advice to change FS.

kluelos · January 8, 2010, 4:13pm

Filesystems do not generally report their own parameters and limitations. That is, Win98 does not say, “I don’t support files larger than 4GB.”.

Rather, this is extrinsic knowledge – you (as a programmer) just have to know that. The way Windows applications do this is to ask what the filesystem is mounted as. It can only handle a few answers. If the answer is not NTFS, then it must be one of the other two filesystems extrinsically known about, which are extrinsically known not to support files this large.

Likewise, since filesystems do not report this, applications do not query about it. One cannot ask, through an OS call, what the maximum filesize supported is. There is no common mechanism to ask, and no common mechanism to answer, this query. Not much need, since most OS’s can’t even communicate with most filesystems to ask any questions in the first place.

It follows that unless BitComet and other applications are charged with supporting every new and fringe OS in existence (they aren’t, they won’t, kthxbye), then it is up to the operating system in question to supply answers to the questions applications CAN ask (like, what OS are you?), that allow operations to continue.

Very, very few applications are going to be able to handle an answer like, “I’m Ext2Fsd, thanks for asking!” in a usable way. Unless Ext2Fsd becomes so extremely mainstream that it actually gets Windows support (Mac can’t, OS/2 can’t unix can’t, even Netware can’t) for some reason, it’s unlikely that most applications ever will support it or other fringe OS filesystems. If Ext2Fsd cannot handle this environment, it cannot handle Windows, nice try, next contestant, please.

It’s going to be up to Ext2Fsd to answer, “Yes, I am NTFS”, to that question in order for the vast majority of applications not to choke on the answer.

To put it another way, Windows doesn’t natively support any of these other file systems, all of which are in far wider use than Ext2Fsd. So hold not thy breath.

BitComet does not work on the requisite level to bypass filesystem commands and do raw disk access for other filesystems. It uses Windows to do that, and the developers are unlikely to rewrite enough of the basic OS to intervene at those levels. If they wanted to work on filesystem internals they probably wouldn’t be working for BitComet. BC will be able to do it when Windows is able to do it. Windows still hasn’t natively gotten around to NWFS yet.

Malgoth · January 8, 2010, 6:19pm

If the answer is not NTFS, then it must be one of the other two filesystems extrinsically known about, which are extrinsically known not to support files this large.

That’s the way Bitcomet works now and it can go wrong sometimes (well, at least in my case it does ). I suggest not to check whether the destination file system for the large file is NTFS and then decide it must be FAT16 or FAT32 only, but instead of this check whether it is FAT16 or FAT32 and if not - allow torrent task to start. In this way nothing is changed for these 3 native Windows file systems but allows in cases like mine to get the things done.

Or at least make some advanced option to ignore this ‘Non-NTFS’-warning…

BitComet does not work on the requisite level to bypass filesystem commands and do raw disk access for other filesystems. It uses Windows to do that, and the developers are unlikely to rewrite enough of the basic OS to intervene at those levels. If they wanted to work on filesystem internals they probably wouldn’t be working for BitComet. BC will be able to do it when Windows is able to do it. Windows still hasn’t natively gotten around to NWFS yet.

Thats what I’m trying to say! No need for Bitcomet to get raw disk access or something - with Ext2Fsd Windows handles EXT2/EXT3 partitions as easy as its native FAT/NTFS.

/monthly_01_2010/post-55932-126297370268.png" rel=“”>

No need for Bitcomet event to know about the Ext2Fsd running - just start the torrent task and Windows will store the files on this partition just perfectly!

kluelos · January 8, 2010, 9:01pm

That’s the way Bitcomet works now and

No, that’s the way Windows works now. BC just makes calls into the windows file system at a much higher level. It does NOT make low-level OS calls or attempt to bypass same.

No need for Bitcomet event to know about the Ext2Fsd running - just start the torrent task and Windows will store the files on this partition just perfectly!

BitComet does not know.

BitComet only knows what it’s told by Windows, namely that this is not an OS which supports large files.

BitComet doesn’t make that judgment call. Windows does. BC proposes to write a file larger than 4GB to the disk, and the operating system returns an error, which BC interprets for you as the message you saw.

BitComet does not get involved in filesystem operations on that level.

It is going to be up to Ext2Fed to work out an accomodation with windows, to present itself as NTFS in order to work around this. BitComet cannot adapt to it short of bypassing the Windows disk handling and taking care of that itself. This isn’t going to happen.

Ext2Fed must change, must adapt to Windows. Windows isn’t going to adapt to it. You are wasting your time trying to pursue this as a change to BitComet.

Malgoth · January 9, 2010, 3:36pm

No, that’s the way Windows works now. BC just makes calls into the windows file system at a much higher level. It does NOT make low-level OS calls or attempt to bypass same.

To me the conversation between BC & Windows is like this:

BC: Large file in the torrent. Hey, Win, what’s the file system of the destination partition and is there enough free space?

Win: FS is EXT3, enough free space.

BC: <_< uh… what’s that? Oh, never mind, you Win can only handle large files with NTFS and this must be some kind of a FAT16 or FAT32

Win: Well, actually it’s not, and I can…

BC: No, you can not, Win B) Hey, user, give me NTFS for this torrent!

And some prove is on the way…

For example, Total Commander (and any other Windows file manager) works with Ext2Fsd-mounted EXT3 partition and large files on it just perfectly (I:\ is an EXT3 partition):

/monthly_01_2010/post-55932-126304895005.png" rel=“”>

But let’s try something more relevant to BitComet like µTorrent :rolleyes:

Adding a torrent with a large file to a EXT3 as a destination:

/monthly_01_2010/post-55932-126304924837.png" rel=“”>

Starting the task:

/monthly_01_2010/post-55932-126304928773.png" rel=“”>

Works OK… Strange, isn’t it? :rolleyes:

Checking the file in the destination dir:

/monthly_01_2010/post-55932-126304943239.png" rel=“”>/monthly_01_2010/post-55932-126304949541.png" rel=“”>

No errors, no “give-me-NTFS” demands, works just as it should.

The very same torrent in BC - and you already know what the result will be!

/monthly_01_2010/post-55932-126305023544.png" rel=“”>

BitComet doesn’t make that judgment call. Windows does. BC proposes to write a file larger than 4GB to the disk, and the operating system returns an error, which BC interprets for you as the message you saw.

But why doesn’t OS return any errors to Total Commander, µTorrent or any other application? Or would you say they all perform that low-level disk access bypassing Windows HDD managment? :lol: There would be no need for Ext2Fsd then.

BitComet cannot adapt to it short of bypassing the Windows disk handling and taking care of that itself.

There’s no need for this! All the troubles come just of BC’s logic: “Large file on non-NTFS? Won’t even try and ask Windows if it can handle this!”

Malgoth · January 16, 2010, 2:19pm

Maybe I’m wrong with my inferences

So, any suggestions why µTorrent works OK in this situation and BitComet doesn’t? :wacko:

kluelos · January 16, 2010, 4:46pm

µtorrent simply doesn’t ask. (You don’t have to.) And when it doesn’t work, µtorrent cheerfully runs off the rails.

I hung on to Win98 for a long time past its sell-by date, on a dual-boot system. (I still dual-boot Win2k).My largest partition then was FAT-32, and I started µtorrent to downloading the several DVD iso’s of Myst: Uru. The disk images were > 4GB, but I’d forgotten about that limit. µtorrent cheerfully started the download, then just abruptly choked with a disk error when the file tried to grow larger than 4GB.

µtorrent left it to me to figure out why it had just stopped and couldn’t write to the disk. (It was this incident that finally got me to embrace XP, though 2K was still my primary OS. Funny, but I didn’t even like Uru as it turned out.)

They’re both valid (for some value of valid) ways of handling the issue. It’s your choice, but I druther have BitComet’s way, “doesn’t look like this will work”, than µtorrent’s stoner-like, “it broke, d00d, dunno what’s wrong…”.

Malgoth · January 17, 2010, 10:20pm

µtorrent simply doesn’t ask. (You don’t have to.) And when it doesn’t work, µtorrent cheerfully runs off the rails.

That’s not true. For the latest stable release version 1.8.5 if you try to start a >4Gb-file task on a FAT32 partition, you’ll get this:

/monthly_01_2010/post-55932-126376227567.png" rel=“”>

I hung on to Win98 for a long time past its sell-by date, on a dual-boot system. (I still dual-boot Win2k).My largest partition then was FAT-32, and I started µtorrent to downloading the several DVD iso’s of Myst: Uru. The disk images were > 4GB, but I’d forgotten about that limit. µtorrent cheerfully started the download, then just abruptly choked with a disk error when the file tried to grow larger than 4GB.
µtorrent left it to me to figure out why it had just stopped and couldn’t write to the disk. (It was this incident that finally got me to embrace XP, though 2K was still my primary OS. Funny, but I didn’t even like Uru as it turned out.)

That could happen with the early versions of µt, but from that days they have worked on this issue for sure

They’re both valid (for some value of valid) ways of handling the issue. It’s your choice, but I druther have BitComet’s way, “doesn’t look like this will work”, than µtorrent’s stoner-like, “it broke, d00d, dunno what’s wrong…”.

This is where we come again to your previous statement:

BitComet doesn’t make that judgment call. Windows does. BC proposes to write a file larger than 4GB to the disk, and the operating system returns an error, which BC interprets for you as the message you saw.

I just tried to copy a 5Gb image file in Windows Explorer (I mean this native file manager) to a FAT32 partition with enough free space and, as expected, got this error

That’s the error you are talking about. But there aren’t any errors if you try to copy it to NTFS or EXT3 partition in the same Windows Explorer!

And all I’m asking is to really make BitComet work the way you described above: “BC proposes to write a file larger than 4GB to the disk, and the operating system…”. Let BC offer the OS to write a large file! If the OS returns an error like it does in case of FAT32, then stop, but if it doesn’t - let the OS do the job!

gavin.wang · January 19, 2010, 8:35am

Thank you. It will be fixed in v 1.18 or v1.19 .

Malgoth · January 20, 2010, 11:19pm

Thank you. It will be fixed in v 1.18 or v1.19 .

Oh, thanks! That would be great.

kluelos

kluelos · January 21, 2010, 5:11pm

You’re welcome, Malgoth.

meanie · December 28, 2019, 11:13pm

Is this fix? I still have the issue with BitComet not able to save to a network drive with the latest version. It’s requiring NTFS even though the network drive is NTFS formatted and other torrent programs can write large files to the drive.

Rhubarb · December 30, 2019, 12:18am

You do realise you are asking a question to which the last posting was 9 YEARS ago?

eefaan · February 17, 2020, 2:59am

	On 12/30/2019 at 8:18 AM, Rhubarb said:



	

		You do realise you are asking a question to which the last posting was **9 YEARS** ago?
	



However I still meet the error on version 1.63.0 (macOS). When I modify the properties and click the 'OK' button, BitComet give me a 'tip' says 'The file to be downloaded is larger than 4GB, please modify your destination location to NTFS disk drive!'. My disk is exFAT format, btw.

Otherwise, sometimes I didn't meet the tip even if the file is almost 6GB, and the file can be downloaded as expect, which is strange.

Rhubarb · February 17, 2020, 5:10pm

While theoretically Ex(tended)FAT canhandle files up to 128 Petabytes, it’s possible that the Mac version of FreeBSD is limiting the amount that can be transferred. Noit being a Mac person (well not since they switched to Intel CPUs and the Marklaar engine under the desktop) I can’t give a specific answer

By the way, necro-posting is a very bad idea -  start a new thread of your own if you have issues rather than a 'me too' on a thread that has been dead for years

wxhere · April 9, 2020, 7:47am

	On 2/17/2020 at 10:59 AM, eefaan said:



	

		However I still meet the error on version 1.63.0 (macOS). When I modify the properties and click the 'OK' button, BitComet give me a 'tip' says 'The file to be downloaded is larger than 4GB, please modify your destination location to NTFS disk drive!'. My disk is exFAT format, btw.
	



Thanks for feedback. This bug will be fixed in next release of BitComet macOS edition.

meanie · May 8, 2020, 7:21pm

	On 4/9/2020 at 12:47 AM, wxhere said:



	

		Thanks for feedback. This bug will be fixed in next release of BitComet macOS edition.
	



Can you make a general comment regarding saving large files to any non-NTFS locations (such as network drives/shares or other FS)?

I have used BitComet for a long time and looks like I need to switch to another torrent program as more and more files are larger than 4GB, and I'm saving to non-NTFS locations that supports large files, but BitComet refuse to use those locations; while other programs work fine.

Rhubarb · May 9, 2020, 6:14am

As stated, the devs are working on that

meanie · August 16, 2020, 1:02am

	On 4/9/2020 at 12:47 AM, wxhere said:



	

		Thanks for feedback. This bug will be fixed in next release of BitComet macOS edition.
	



Will you fix the issue of saving 4TB files to any non-NTFS locations (such as network drives/shares or other FS)?  Not specifically related to issues with macOS or exFAT?  This issue has existed for 10 years, though it says it will be fixed in 1.18 or 1.19, it remains an issue.

Rhubarb · August 16, 2020, 10:06pm

The question has been answered by one of the devs.

**THEY ARE WORKING ON IT**