diff --git a/Filer-Stores.md b/Filer-Stores.md new file mode 100644 index 0000000..6acb531 --- /dev/null +++ b/Filer-Stores.md @@ -0,0 +1,43 @@ +### Comparing Storage Options + +The Filer Store persists all file metadata and directory information. + +| Filer Store Name | Lookup | number of entries in a folder | Scalability | Renaming | TTL | Note | +| ---------------- | -- | -- | -- | -- | -- | -- | +| memory | O(1) | limited by memory | Local, Fast | | | for testing only, no persistent storage | +| leveldb | O(logN)| unlimited | Local, Very Fast | | | Default, fairly scalable | +| leveldb2 | O(logN)| unlimited | Local, Very Fast, faster than leveldb | | | Similar to leveldb, part of the lookup key is 128bit MD5 instead of the long full file path | +| Redis | O(1) | limited | Local or Distributed, Fastest ||Yes| one directory's sub file names are stored in one key~value entry | +| Cassandra | O(logN)| unlimited | Local or Distributed, Very Fast||Yes| | +| MySql | O(logN)| unlimited | Local or Distributed, Fast |Atomic| | Easy to manage, export | +| Postgres | O(logN)| unlimited | Local or Distributed, Fast |Atomic| | Easy to manage, export | +| Etcd | O(logN)| ~10GB | Distributed, 10,000 writes/sec || | No SPOF. High Availability.| + +#### Switching between different Stores +It is easy to switch between different filer stores. + +For example: +```sh + +# first save current filer meta data + +$ weed shell +> fs.cd http://filerHost:filerPort/ +> fs.meta.save +... +total 65 directories, 292 files +meta data for http://localhost:8888/ is saved to localhost-8888-20190417-005421.meta +> exit + +# now switch to a new filer, and load the previously saved metadata +$ weed shell +> fs.meta.load localhost-8888-20190417-005421.meta +... +total 65 directories, 292 files +localhost-8888-20190417-005421.meta is loaded to http://localhost:8888/ + +``` + +### Extending Storage Options + +For any new storage option, please implement the FilerStore interface. It should be fairly straight forward to implement. Welcome to contribute back.