From 66a8db7a28907e95a90c81eb3bfe5c75199cd4ed Mon Sep 17 00:00:00 2001 From: Chris Lu Date: Tue, 9 Feb 2021 14:41:46 -0800 Subject: [PATCH] Created Tiered Storage (markdown) --- Tiered-Storage.md | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) create mode 100644 Tiered-Storage.md diff --git a/Tiered-Storage.md b/Tiered-Storage.md new file mode 100644 index 0000000..3cfb2b9 --- /dev/null +++ b/Tiered-Storage.md @@ -0,0 +1,31 @@ +All read and write operation for volume data is O(1) disk seek. However, there are fast, slow, and remote storages. +Since data that are hot, warm, and cold, it would be cost-efficient to move data accordingly. + +# Speed up volume index + +To write any data, the volume index needs to append one entry. To read any data, the volume index lookup is required. The volume index can be in memory mode or an LevelDB instance. The amount of index content is small while it is accessed frequently. + +You can run `weed volume -dir.idx=/fast/disk/dir` or `weed server -volume.dir.idx=/vast/disk/dir` to ensure the volume index is located on a fast disk, e.g., a fast SSD mount. + +# Volume Disk Type + +If the volume data also needs fast access, you can create them on fast disks. + +For example, one volume server can have two disks with two kinds of disk types: +``` +weed volume -max=10000 -disk=hdd,ssd -port=8080 -dir=/data,/ssd -compactionMBps=40 -minFreeSpacePercent=7 +``` + +Please notice the `-disk=hdd,ssd & -dir=/data,/ssd`. + +Then you can create with `fs.configure` a path specific setting and set buckets/collections that start with (example) "ssd_" to be allocated to the ssd, and all other collection will be created on the hdd. you can also create specific bucket name. + +Example (within the weed shell): +``` +fs.configure -locationPrefix=/buckets/ssd_ -disk=ssd -apply +``` + +https://github.com/chrislusf/seaweedfs/wiki/Path-Specific-Configuration + +For now, ssd volumes needs to be manually moved from ssd directories to hdd directories, if you want to change the disk type. +