amiga.org
     
iconAll times are GMT -6. The time now is 05:45 AM. | Welcome to Forum, please register to access all of our features.

Amiga.org Amiga computer related discussion Amiga Software Issues and Discussion PFS3 block size

Amiga Software Issues and Discussion This forum exists for the discussion of the use, issues with, and fun brought about by classic and next generation Amiga software.

Reply
 
Thread Tools Display Modes
Old 09-23-2010, 05:20 AM   #21
brownb2
Technoid
Points: 3,856, Level: 39 Points: 3,856, Level: 39 Points: 3,856, Level: 39
Activity: 5% Activity: 5% Activity: 5%
 
brownb2's Avatar
 
Join Date: Aug 2010
Location: UK
Posts: 194
Default Re: PFS3 block size

Quote:
Originally Posted by Piru View Post
The absolute answer you questioned was the only correct answer to the question presented.
You missed the boat then re. second edit added specifically because I read the entirety of the thread after submitting the post.

Quote:
Originally Posted by Piru View Post
Regarding blocksize: Using large blocksize to gain performance only makes sense with poor filesystems that don't keep the data sequental (and thus don't perform mostly sequental access), or with hardware that actually uses larger hardware block size.
I disagree. Since block size is a minimum unit size you are also discussing the impact on the CPU and the amount of usable data transferred in a given time. An extreme example - if the average file size being read is 1K and a cluster size is 8K (thus transfer block size) it'll be transferring more slack space than say with a block of 4K, plus the CPU will be doing more redundant IO operations (CRC, reads etc). Sequential access only lowers seek latency as far as I'm aware...
Quote:
Originally Posted by Piru View Post
FFS is notoriously slow so with that you get a performance boost.
Now that I agree with

Quote:
Originally Posted by Piru View Post
PS. While some amiga file systems allow 4k block sizes (and thus seem to be ready for the new 4k block size hard drives) there still is the issue of proper alignment. In order to obtain best performance the partition beginning must be aligned by the block size, too. I doubt the Amiga partitioning programs account for this.

This probably doesn't matter for compact flash which has a standardised sector of 512B so I guess that part answers my initial question as to the best size for CF, either 512B or 1K Some hard disks have been using 4K since 2008 btw.
__________________
A600 (3.84 MIPS) - 2MB Chip, 4MB PCMCIA, 11MB Fast, ACA-620 680EC20@16.67MHz, RTC, 512MB CF HD, WB 2.1
A1200 (7.36 MIPS) - 2MB Chip 8MB Fast, MTEC Viper 68030@42MHz MMU, 68882 FPU, RTC, 1GB CF HD, T.Turbo 2, 2 Ext. Floppies, WB 3.0.
Atari 520 STFM - 1MB, Multiface ST.
Commodore 64C - 64K, SD2IEC "Floppy" Drive
ZX Spectrum +2A - 128KB, Custom 3'5 External Floppy.
ZX Spectrum +3 - 128KB, DivIDE+ 64MB CF HD, Multiface 3.
brownb2 is offline   Reply With Quote
Old 09-23-2010, 05:34 AM   #22
brownb2
Technoid
Points: 3,856, Level: 39 Points: 3,856, Level: 39 Points: 3,856, Level: 39
Activity: 5% Activity: 5% Activity: 5%
 
brownb2's Avatar
 
Join Date: Aug 2010
Location: UK
Posts: 194
Default Re: PFS3 block size

Quote:
Originally Posted by Golem!dk View Post
In general? And of course not everyone uses the builtin ide controllers... so that's another thing for you to consider. But do post your results, would be more interesting than speculation.
"In general..." something to recommend using certain controllers/CPU combinations/media/fs/blue moons when someone asks what size should I use on a forum there'd be some guidelines.

I *think* based on reading around for this thread on hardware standards: for the internal controller 512B would be about the best for low end hardware 000,020 (CF or hard disk). 512B-1K would be better for 030,040,060 and probably 4K for the newer HD standard discussed (040 upwards though). Again this all assumes the alignment is correct as Piru mentions... and that I'm not talking out of my bottom
__________________
A600 (3.84 MIPS) - 2MB Chip, 4MB PCMCIA, 11MB Fast, ACA-620 680EC20@16.67MHz, RTC, 512MB CF HD, WB 2.1
A1200 (7.36 MIPS) - 2MB Chip 8MB Fast, MTEC Viper 68030@42MHz MMU, 68882 FPU, RTC, 1GB CF HD, T.Turbo 2, 2 Ext. Floppies, WB 3.0.
Atari 520 STFM - 1MB, Multiface ST.
Commodore 64C - 64K, SD2IEC "Floppy" Drive
ZX Spectrum +2A - 128KB, Custom 3'5 External Floppy.
ZX Spectrum +3 - 128KB, DivIDE+ 64MB CF HD, Multiface 3.
brownb2 is offline   Reply With Quote
Old 09-23-2010, 05:45 AM   #23
Piru
' union select name,pwd--
Points: 30,457, Level: 100 Points: 30,457, Level: 100 Points: 30,457, Level: 100
Activity: 69% Activity: 69% Activity: 69%
 
Piru's Avatar
 
Join Date: Aug 2002
Location: Helsinki, Finland
Posts: 6,946
Default Re: PFS3 block size

Quote:
Since block size is a minimum unit size you are also discussing the impact on the CPU and the amount of usable data transferred in a given time.
How? How is it different to transter 64 blocks of 512 bytes vs 32 blocks of 1024 bytes?

Quote:
An extreme example - if the average file size being read is 1K and a cluster size is 8K (thus transfer block size) it'll be transferring more slack space than say with a block of 4K, plus the CPU will be doing more redundant IO operations (CRC, reads etc).
Why would the filesystem read the whole 8K in this case? Even if it would read the whole 8k, modern drives are so fast in sequental access that it makes little difference if you read 1k or 8k.

What CRC are you talking about? If it's something in the filesystem (no amiga filesystem CRCs the data btw), why would the filesystem calculate CRC for the whole 8k if only 1k is of is valid data? Lets not forget that most time is spent reading and writing the data. Metadata is insignificant in comparison.

Quote:
This probably doesn't matter for compact flash which has a standardised sector of 512B so I guess that part answers my initial question as to the best size for CF, either 512B or 1K
It was about the new 4k hard disk drives which report 512k block size (for compatibility reasons) while actually using 4k block sizes internally. They do work, but writing gets really slow of the block is not aligned properly. Until recently both Windows and Linux created such bad partitioning.

As for flash, they have a totally different internal structure which the typical filesystem has no way of knowing. The firmware tries to do some magic tricks to accomodate for the silly things typical filesystems do when they assume classical HDD. There's little that can be assumed about flash, except that small writes are typically really slow (due to flash internal arrangement).

http://en.wikipedia.org/wiki/UBIFS is an interesting approach to the problem. Rather than going thru the firmware, access the flash directly. Obviously this requires special HW (that is: it's not usable with your off-the-shelf SSD drives). Here's some more about UBIFS: http://www.linux-mtd.infradead.org/d...whitepaper.pdf

Last edited by Piru; 09-23-2010 at 05:54 AM..
Piru is offline   Reply With Quote
Old 09-23-2010, 06:16 AM   #24
Piru
' union select name,pwd--
Points: 30,457, Level: 100 Points: 30,457, Level: 100 Points: 30,457, Level: 100
Activity: 69% Activity: 69% Activity: 69%
 
Piru's Avatar
 
Join Date: Aug 2002
Location: Helsinki, Finland
Posts: 6,946
Default Re: PFS3 block size

As reminded by a friend: AmigaOS is one of the few OSes that don't do proper block level caching by itself. This is another reason by large block size gives benefit with FFS.

It has gone as far as some filesystems implementing such caching, read ahead etc in itself (SFS).
Piru is offline   Reply With Quote
Old 09-23-2010, 07:18 AM   #25
brownb2
Technoid
Points: 3,856, Level: 39 Points: 3,856, Level: 39 Points: 3,856, Level: 39
Activity: 5% Activity: 5% Activity: 5%
 
brownb2's Avatar
 
Join Date: Aug 2010
Location: UK
Posts: 194
Default Re: PFS3 block size

Quote:
Originally Posted by Piru View Post
How? How is it different to transter 64 blocks of 512 bytes vs 32 blocks of 1024 bytes?
A CRC-enabled device calculates a short, fixed-length binary sequence, known as the CRC code or just CRC, for each block of data and sends or stores them both together.

I submit that less blocks (transferred disk sectors) => less CRC checking, unless I am misunderstanding something here.

[EDIT]Also (again might be misunderstanding) as it is PIO the CPU is dealing with the IO and requesting the file at the lowest level, i.e. manipulating/copying blocks (say in contrast to DMA where a single request will be made whilst the CPU does something else).[/EDIT]
__________________
A600 (3.84 MIPS) - 2MB Chip, 4MB PCMCIA, 11MB Fast, ACA-620 680EC20@16.67MHz, RTC, 512MB CF HD, WB 2.1
A1200 (7.36 MIPS) - 2MB Chip 8MB Fast, MTEC Viper 68030@42MHz MMU, 68882 FPU, RTC, 1GB CF HD, T.Turbo 2, 2 Ext. Floppies, WB 3.0.
Atari 520 STFM - 1MB, Multiface ST.
Commodore 64C - 64K, SD2IEC "Floppy" Drive
ZX Spectrum +2A - 128KB, Custom 3'5 External Floppy.
ZX Spectrum +3 - 128KB, DivIDE+ 64MB CF HD, Multiface 3.

Last edited by brownb2; 09-23-2010 at 07:35 AM..
brownb2 is offline   Reply With Quote
Old 09-23-2010, 07:30 AM   #26
Thomas
Desperately needs a life
Points: 14,991, Level: 79 Points: 14,991, Level: 79 Points: 14,991, Level: 79
Activity: 20% Activity: 20% Activity: 20%
 
Thomas's Avatar
 
Join Date: Jun 2002
Posts: 3,016
Default Re: PFS3 block size

Quote:
Originally Posted by brownb2 View Post
A CRC-enabled device calculates a short, fixed-length binary sequence, known as the CRC code or just CRC, for each block of data and sends or stores them both together.

I submit that less blocks (transferred disk sectors) => less CRC checking, unless I am misunderstanding something here.

A CRC-enabled device always calculates a checksum for its physical blocks. It does not know about logical clusters and therefore it does not make a difference if larger clusters are used. Unfortunately the word "block" is used with many different meanings. Physical device block size (e.g. 512 bytes or 4k) is always fixed for SCSI or ATA devices. File system block size (a.k.a. cluster size) can be varied by software, but it depends on the file system implementation whether larger clusters improve performance or not.
__________________
Email: thomas-rapp@web.de
Home: thomas-rapp.homepage.t-online.de/
Thomas is offline   Reply With Quote
Old 09-23-2010, 08:30 AM   #27
Franko
Banned
Points: 22,799, Level: 93 Points: 22,799, Level: 93 Points: 22,799, Level: 93
Activity: 8% Activity: 8% Activity: 8%
 
Join Date: Jun 2010
Posts: 5,707
Blog Entries: 33
Default Re: PFS3 block size

I've tried PFS in the past but I much prefer to use SmartFileSystem myself. I'ts very easy to use and set up, no problems with block sizes or such like. Been using it for about 4 to 5 years now and have never lossed a single bit of data.

Best of all using SFS2 you can have up to 1 Terabyte sized partitions and no limit on file lengths, which is needed if you want to store backup ISO images of your DVDs which can be up to 8Gb in size.
Franko is offline   Reply With Quote
Old 09-23-2010, 08:43 AM   #28
Piru
' union select name,pwd--
Points: 30,457, Level: 100 Points: 30,457, Level: 100 Points: 30,457, Level: 100
Activity: 69% Activity: 69% Activity: 69%
 
Piru's Avatar
 
Join Date: Aug 2002
Location: Helsinki, Finland
Posts: 6,946
Default Re: PFS3 block size

Quote:
Originally Posted by Franko View Post
I've tried PFS in the past but I much prefer to use SmartFileSystem myself. I'ts very easy to use and set up, no problems with block sizes or such like. Been using it for about 4 to 5 years now and have never lossed a single bit of data.
The sad thing about SFS is that it's inferior compared to PFS3, regardless of SFS being much later development.
  • PFS3 is faster than SFS.
  • PFS3 doesn't generate massively fragmented files when two or more applications write files to disk. SFS does.
  • PFS3 performance doesn't deteriorate over time. SFS does.
  • PFS3 has a repair tool. Often with SFS the only option is to copy data over, reformat and copy data back (MorphOS does have a SFSDoctor tool, however).

Last edited by Piru; 09-23-2010 at 08:48 AM..
Piru is offline   Reply With Quote
Old 09-23-2010, 08:57 AM   #29
Franko
Banned
Points: 22,799, Level: 93 Points: 22,799, Level: 93 Points: 22,799, Level: 93
Activity: 8% Activity: 8% Activity: 8%
 
Join Date: Jun 2010
Posts: 5,707
Blog Entries: 33
Default Re: PFS3 block size

Quote:
Originally Posted by Piru View Post
The sad thing about SFS is that it's inferior compared to PFS3, regardless of SFS being much later development.
  • PFS3 is faster than SFS.
  • PFS3 doesn't generate massively fragmented files when two or more applications write files to disk. SFS does.
  • PFS3 performance doesn't deteriorate over time. SFS does.
  • PFS3 has a repair tool. Often with SFS the only option is to copy data over, reformat and copy data back (MorphOS does have a SFSDoctor tool, however).
SFS being inferior is just your opinion, I've been using it for years, and both of my 500GB HDs are almost full. I've never noticed any performance deterioration or ever had the need to repair any files or partions.

Im perfectly happy with it, but that's just my opinion...
Franko is offline   Reply With Quote
Old 09-23-2010, 09:09 AM   #30
Piru
' union select name,pwd--
Points: 30,457, Level: 100 Points: 30,457, Level: 100 Points: 30,457, Level: 100
Activity: 69% Activity: 69% Activity: 69%
 
Piru's Avatar
 
Join Date: Aug 2002
Location: Helsinki, Finland
Posts: 6,946
Default Re: PFS3 block size

Quote:
Originally Posted by Franko View Post
SFS being inferior is just your opinion
Benchmarks are out there, PFS3 is a lot faster than SFS. It's easy to reproduce the fragmentation issue by having multiple apps writing a file on SFS volume at the same time, and to verify the excessive fragmentation that results. Performance deterioration over time is a bit trickier to test, but most likely it's a direct result from the fragmentation issue described, but other factors can be in play as well. The issue has been observed by many (more than just me). Lack of repair tool (for anything but MorphOS) is a fact, too.

It's more than just an opinion.

Quote:
I've been using it for years, and both of my 500GB HDs are almost full. I've never noticed any performance deterioration or ever had the need to repair any files or partions.
You're extremely lucky.
Piru is offline   Reply With Quote
Old 09-23-2010, 09:21 AM   #31
Franko
Banned
Points: 22,799, Level: 93 Points: 22,799, Level: 93 Points: 22,799, Level: 93
Activity: 8% Activity: 8% Activity: 8%
 
Join Date: Jun 2010
Posts: 5,707
Blog Entries: 33
Default Re: PFS3 block size

Benchmarks, smenschmarks, Piru I'm perfectly happy with SFS, never personally experienced any of the issues you described, I was simply just saying that there are other alternative file systems available.

Yup, maybe I am lucky...
Franko is offline   Reply With Quote
Old 09-23-2010, 09:25 AM   #32
Darrin
Lifetime Member
Points: 20,321, Level: 89 Points: 20,321, Level: 89 Points: 20,321, Level: 89
Activity: 76% Activity: 76% Activity: 76%
 
Darrin's Avatar
 
Join Date: May 2002
Location: Lake Charles, Louisiana
Posts: 4,372
Send a message via AIM to Darrin Send a message via Yahoo to Darrin
Default Re: PFS3 block size

Quote:
Originally Posted by Franko View Post
Benchmarks, smenschmarks, Piru I'm perfectly happy with SFS, never personally experienced any of the issues you described, I was simply just saying that there are other alternative file systems available.

Yup, maybe I am lucky...
I'm a satisfied SFS user too, however I am looking forward to testing PFS3 when the "free" version is openly released.
__________________
A2000, A3000, 2 x A1200T, A1200, A4000Tower & Mediator, CD32, VIC-20, C64, C128, C128D, PET 8032, Minimig & ARM, C-One, FPGA Arcade... and AmigaOne X1000.
Darrin is offline   Reply With Quote
Old 09-23-2010, 04:45 PM   #33
ChaosLord
Premium Member
Points: 15,941, Level: 81 Points: 15,941, Level: 81 Points: 15,941, Level: 81
Activity: 14% Activity: 14% Activity: 14%
 
ChaosLord's Avatar
 
Join Date: Nov 2003
Location: Houston, Texas
Posts: 2,528
Default Re: PFS3 block size

Pfs3 ftw!
__________________
Wanna try a wonderfull strategy game with lots of handdrawn anims,
Magic Spells and Monsters, Incredible playability and lastability,
English speech, etc. Total Chaos AGA
ChaosLord is offline   Reply With Quote
Old 09-23-2010, 05:56 PM   #34
wawrzon
Defender of the Faith
Points: 9,026, Level: 63 Points: 9,026, Level: 63 Points: 9,026, Level: 63
Activity: 99% Activity: 99% Activity: 99%
 
Join Date: Nov 2007
Posts: 1,660
Default Re: PFS3 block size

Quote:
Originally Posted by Franko View Post
Benchmarks, smenschmarks, Piru I'm perfectly happy with SFS, never personally experienced any of the issues you described, I was simply just saying that there are other alternative file systems available.

Yup, maybe I am lucky...
wait your hour.
wawrzon is online now   Reply With Quote
Old 09-23-2010, 06:22 PM   #35
matthey
Cult Member
Points: 7,155, Level: 56 Points: 7,155, Level: 56 Points: 7,155, Level: 56
Activity: 55% Activity: 55% Activity: 55%
 
Join Date: Aug 2007
Location: Kansas
Posts: 907
Default Re: PFS3 block size

Quote:
Originally Posted by Piru View Post
PFS3 is faster than SFS.
No comparison on my system. PFS destroyed SFS in my SysSpeed tests.
Quote:
PFS3 doesn't generate massively fragmented files when two or more applications write files to disk. SFS does.
I can't verify this. SFS is slower and thrashes the HD more with simultaneous writes.
Quote:
PFS3 performance doesn't deteriorate over time. SFS does.
This is my experience too. I wouldn't consider this is a major issue with SFS.
Quote:
PFS3 has a repair tool. Often with SFS the only option is to copy data over, reformat and copy data back (MorphOS does have a SFSDoctor tool, however).
SFS has a recovery tool that works under AmigaOS 3. It's very buggy and slow but I used it to recover a partition. I did have to reformat.
The biggest issue with SFS is bugs.
Can't the mask force the buffer alignment to whatever is needed? I use MASK=0x7FFFFFFC to force longword alignment as that is faster.

Last edited by matthey; 09-23-2010 at 06:26 PM..
matthey is offline   Reply With Quote
Old 09-23-2010, 11:23 PM   #36
ChaosLord
Premium Member
Points: 15,941, Level: 81 Points: 15,941, Level: 81 Points: 15,941, Level: 81
Activity: 14% Activity: 14% Activity: 14%
 
ChaosLord's Avatar
 
Join Date: Nov 2003
Location: Houston, Texas
Posts: 2,528
Default Re: PFS3 block size

I have received various reports from playtesters that SFS has severe problems trying to do 2 or more simultaneous reads. (even worse if there is a simultaneous read+write). Supposedly, SFS does not multitask internally. Maybe some knowledgeable person out there could shed some light on this aspect?
__________________
Wanna try a wonderfull strategy game with lots of handdrawn anims,
Magic Spells and Monsters, Incredible playability and lastability,
English speech, etc. Total Chaos AGA
ChaosLord is offline   Reply With Quote
Old 09-23-2010, 11:36 PM   #37
Franko
Banned
Points: 22,799, Level: 93 Points: 22,799, Level: 93 Points: 22,799, Level: 93
Activity: 8% Activity: 8% Activity: 8%
 
Join Date: Jun 2010
Posts: 5,707
Blog Entries: 33
Default Re: PFS3 block size

I can still only say, with SFS I have never experienced any of the above mentioned problems...

I must be really lucky...
Franko is offline   Reply With Quote
Old 09-23-2010, 11:51 PM   #38
runequester
It's Amiga time!
Points: 25,611, Level: 96 Points: 25,611, Level: 96 Points: 25,611, Level: 96
Activity: 96% Activity: 96% Activity: 96%
 
runequester's Avatar
 
Join Date: Oct 2009
Location: Portland OR
Posts: 3,695
Blog Entries: 7
Default Re: PFS3 block size

so this may be an incredibly retarded question but if I was to use one of these file systems, how does one go about it?

I am assuming a fresh install would be needed, correct?
__________________
Amiga AND Linux fan. Zealots are people that lack faith

I blog..a lot.
http://doctorwhoretrospec.blogspot.com/
http://redleftperspective.blogspot.com/
http://theamigablog.blogspot.com/
runequester is offline   Reply With Quote
Old 09-24-2010, 12:00 AM   #39
Franko
Banned
Points: 22,799, Level: 93 Points: 22,799, Level: 93 Points: 22,799, Level: 93
Activity: 8% Activity: 8% Activity: 8%
 
Join Date: Jun 2010
Posts: 5,707
Blog Entries: 33
Default Re: PFS3 block size

Correct !!!

Naw only kidding, You can easily change a partition to SFS, but of course you still need to do a quick format and reinstall. It's well worth the hassle if you use large single files of up to 8GB like I do...

Last edited by Franko; 09-24-2010 at 12:10 AM..
Franko is offline   Reply With Quote
Old 09-24-2010, 12:51 AM   #40
Thomas
Desperately needs a life
Points: 14,991, Level: 79 Points: 14,991, Level: 79 Points: 14,991, Level: 79
Activity: 20% Activity: 20% Activity: 20%
 
Thomas's Avatar
 
Join Date: Jun 2002
Posts: 3,016
Default Re: PFS3 block size

You don't need to reinstall. Just backup (for example copy all files to another partition), change file system, quick format and restore the backup.
__________________
Email: thomas-rapp@web.de
Home: thomas-rapp.homepage.t-online.de/
Thomas is offline   Reply With Quote
Reply

Bookmarks

Tags
pfs3 , block , size

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump