Differences between current version and predecessor to the previous major change of FileSystemsComparison.
Other diffs: Previous Revision, Previous Author, or view the Annotated Edit History
| Newer page: | version 5 | Last edited on Friday, July 29, 2005 1:54:38 pm | by JohnMcPherson | |
| Older page: | version 1 | Last edited on Saturday, August 16, 2003 11:51:59 am | by AristotlePagaltzis | Revert |
@@ -1,7 +1,5 @@
-''Who wrote this?''
-
-OnlineBackupSolution
with machines with 800G of [RAID] storage. This storage is partitioned into one large partition since we need to be able to use hardlinking. So the question became, which file systems will work and how well on a partition like this?
+At WaikatoUniversity, they are installing an OnlineBackup solution
with machines with 800G of [RAID] storage. This storage is partitioned into one large partition since we need to be able to use hardlinking. So the question became, which file systems will work and how well on a partition like this?
We first of all tried [Ext3], a reasonable basic choice. We discovered that [Ext3] took about 200 minutes to do a backup. This was clearly not very acceptable. We tried tweaking [Ext3] extending the journal size to the maximum. No percievable change. Looking at the machine it was using 100% System CPU, however no process was in the D state. Oh well.
We then thought that perhaps the journalling was slowing it down, so we tried [Ext2] (Which has no journal but is pretty much the same). This took about 153 minutes as well. Nope, wasn't the journalling.
@@ -9,7 +7,12 @@
So, next on our hitlist was [ReiserFS]. [ReiserFS] worked well and gave us 45 minutes. We were reasonably happy with this, but thought we should perhaps investigate other filesystems. When talking to people about [ReiserFS], they all said that they'd had problems with unspecified problems with it "a while ago", so, while we might use this on Desktops or even occasionally production machines that require its abilities, we didn't want to use it for a backup server.
[XFS] was next on our list, and since we had to run 2.4.20-pre4 to get the drivers for the on board [IDE] controller working correctly, I forward ported the [XFS] patches forward (pretty trivial job). Running XFS we got 43 minutes. Clearly XFS was superior in terms of speed, and has a long history with [SGI]'s [IRIX] OperatingSystem that most of the glitches have been worked out.
-However, looking at what we were backing up, we found one directory that had almost 500,000 files in it. Not just in the directory tree, just in the directory. Each file is tiny, less than 4k. This explained why [Ext2]/[Ext3] had such poor performance, they don't handle large directories very well, and why [Reiser
] and [XFS] performed so well, they both support using [BTree]s to store their directory layouts.
+However, looking at what we were backing up, we found one directory that had almost 500,000 files in it. Not just in the directory tree, just in the directory. Each file is tiny, less than 4k. This explained why [Ext2]/[Ext3] had such poor performance, they don't handle large directories very well, and why [ReiserFS
] and [XFS] performed so well, they both support using [BTree]s to store their directory layouts.
After deleting this directory we discovered that [Ext3]'s performance had dropped to 17 minutes. This was acceptable for time, and kept us very happy. And thusly ended my time spent swearing at FileSystems, at least for now.
+
+;: -- PerryLorier (August 2003)
+
+-----
+CategoryStorage
