Shared File Systems?

[Jacob]

How many of you our there are leveraging shared file systems with clusters for Sugar? I am curious. The easiest thing is to have one big file system and perform sync for backups. Shared file systems work really well. Larger driver arrays provide great management tools, expandability, ... into the mix.

I have just finished reading about Google's File System. This is a relatively old document, but it is an interesting case study in making great things out of unreliable components. I particularly like the ways that they have streamlined the process. The master server does very little and keeps its entire state in memory (using only megabytes). There are a high number of replicas on different systems, different racks, ... This allows for a high level of redundancy without expensive equipment or expensive RAID. When the cost of loosing a volume is really low, why have redundancy for the volume? There system also has high performance for multiple reads and producer/consumer based logic leveraging the file system.

What are you leveraging for Sugar? I know some of you are using the larger storage array vendors. I am interested in some of the following characteristics:

• I would prefer to have the file system local to the web server
• The file system should support large numbers of file stat requests (this is a requirement for PHP)
• The file system would ideally support large log files with high concurrent writes (these can be stored locally and aggregated later or shipped to a log server)
• The file system needs to support large numbers of files efficiently
• The cost of adding an additional server should be low
• Geographically distributed replication should be supported

I have been wanting a system that replicates changes at a cost proportional to the change. It should detect changes in real time. Servers should always be in sync. I am very interested in the systems and tools that you have used and the experiences that you have had with them.

Jacob