From beb904cd930bbcdbfdc8ccf012a8fab89ddba2ee Mon Sep 17 00:00:00 2001 From: Ruth Fuchss Date: Fri, 1 Jul 2022 16:23:25 +0200 Subject: [PATCH] doc/storage: add introductions for storage drivers Add a short introduction for each storage driver describing the most important features and introducing any terminology that is needed. Signed-off-by: Ruth Fuchss --- doc/reference/storage_btrfs.md | 12 ++++++++++++ doc/reference/storage_ceph.md | 27 ++++++++++++++++++++++++++- doc/reference/storage_cephfs.md | 17 +++++++++++++++++ doc/reference/storage_dir.md | 4 ++++ doc/reference/storage_lvm.md | 13 +++++++++++++ doc/reference/storage_zfs.md | 20 ++++++++++++++++++++ 6 files changed, 92 insertions(+), 1 deletion(-) diff --git a/doc/reference/storage_btrfs.md b/doc/reference/storage_btrfs.md index 501100223c..71d09378b2 100644 --- a/doc/reference/storage_btrfs.md +++ b/doc/reference/storage_btrfs.md @@ -1,6 +1,18 @@ (storage-btrfs)= # Btrfs - `btrfs` +{abbr}`Btrfs (B-tree file system)` is a local file system based on the {abbr}`COW (copy-on-write)` principle. +COW means that data is stored to a different block after it has been modified instead of overwriting the existing data, reducing the risk of data corruption. +Unlike other file systems, Btrfs is extent-based, which means that it stores data in contiguous areas of memory. + +In addition to basic file system features, Btrfs offers RAID and volume management, pooling, snapshots, checksums, compression and other features. + +## Terminology + +A Btrfs file system can have *subvolumes*, which are named binary subtrees of the main tree of the file system with their own independent file and directory hierarchy. +A *Btrfs snapshot* is a special type of subvolume that captures a specific state of another subvolume. +Snapshots can be read-write or read-only. + ## `btrfs` driver in LXD - Uses a subvolume per instance, image and snapshot, creating Btrfs snapshots when creating a new object. diff --git a/doc/reference/storage_ceph.md b/doc/reference/storage_ceph.md index fc4f4f5749..7c9232bacd 100644 --- a/doc/reference/storage_ceph.md +++ b/doc/reference/storage_ceph.md @@ -1,5 +1,30 @@ (storage-ceph)= -# Ceph - `ceph` +# Ceph RBD - `ceph` + + + +```{youtube} https://youtube.com/watch?v=kVLGbvRU98A +``` + +[Ceph](https://ceph.io/) is an open-source storage platform that stores its data in a storage cluster based on {abbr}`RADOS (Reliable Autonomic Distributed Object Store)`. +It is highly scalable and, as a distributed system without a single point of failure, very reliable. + +Ceph provides different components for block storage and for file systems. + + +Ceph {abbr}`RBD (RADOS Block Device)` is Ceph's block storage component that distributes data and workload across the Ceph cluster. +It uses thin provisioning, which means that it is possible to over-commit resources. + +## Terminology + + +Ceph uses the term *object* for the data that it stores. +The daemon that is responsible for storing and managing data is the *Ceph {abbr}`OSD (Object Storage Daemon)`*. +Ceph's storage is divided into *pools*, which are logical partitions for storing objects. +They are also referred to as *data pools*, *storage pools* or *OSD pools*. + + +Ceph block devices are also called *RBD images*, and you can create *snapshots* and *clones* of these RBD images. ## `ceph` driver in LXD diff --git a/doc/reference/storage_cephfs.md b/doc/reference/storage_cephfs.md index 1e436ac052..bda51d600d 100644 --- a/doc/reference/storage_cephfs.md +++ b/doc/reference/storage_cephfs.md @@ -1,6 +1,23 @@ (storage-cephfs)= # CephFS - `cephfs` +% Include content from [storage_ceph.md](storage_ceph.md) +```{include} storage_ceph.md + :start-after: + :end-before: +``` + +{abbr}`CephFS (Ceph File System)` is Ceph's file system component that provides a robust, fully-featured POSIX-compliant distributed file system. +Internally, it maps files to Ceph objects and stores file metadata (for example, file ownership, directory paths, access permissions) in a separate data pool. + +## Terminology + +% Include content from [storage_ceph.md](storage_ceph.md) +```{include} storage_ceph.md + :start-after: + :end-before: +``` + ## `cephfs` driver in LXD - Can only be used for custom storage volumes diff --git a/doc/reference/storage_dir.md b/doc/reference/storage_dir.md index 17b196c7bc..5ef49b2137 100644 --- a/doc/reference/storage_dir.md +++ b/doc/reference/storage_dir.md @@ -1,6 +1,10 @@ (storage-dir)= # Directory - `dir` +The directory storage driver is a basic backend that stores its data in a standard file and directory structure. +This driver is quick to set up and allows inspecting the files directly on the disk, which can be convenient for testing. +However, LXD operations are {ref}`not optimized ` for this driver. + ## `dir` driver in LXD - While this backend is fully functional, it's also much slower than diff --git a/doc/reference/storage_lvm.md b/doc/reference/storage_lvm.md index 26844089ea..8b3a4656c1 100644 --- a/doc/reference/storage_lvm.md +++ b/doc/reference/storage_lvm.md @@ -1,6 +1,19 @@ (storage-lvm)= # LVM - `lvm` +{abbr}`LVM (Logical Volume Manager)` is a storage management framework rather than a file system. +It is used to manage physical storage devices, allowing you to create a number of logical storage volumes that use and virtualize the underlying physical storage devices. + +Note that it is possible to over-commit the physical storage in the process, to allow flexibility for scenarios where not all available storage is in use at the same time. + +## Terminology + +LVM can combine several physical storage devices into a *volume group*. +You can then allocate *logical volumes* of different types from this volume group. + +One supported volume type is a *thin pool*, which allows over-committing the resources by creating thinly provisioned volumes whose total allowed maximum size is larger than the available physical storage. +Another type is a *volume snapshot*, which captures a specific state of a logical volume. + ## `lvm` driver in LXD - Uses LVs for images, then LV snapshots for instances and instance snapshots. diff --git a/doc/reference/storage_zfs.md b/doc/reference/storage_zfs.md index 8ccc42674e..322899cfde 100644 --- a/doc/reference/storage_zfs.md +++ b/doc/reference/storage_zfs.md @@ -5,6 +5,26 @@ discourse: 1333 (storage-zfs)= # ZFS - `zfs` +{abbr}`ZFS (Zettabyte file system)` combines both physical volume management and a file system. +A ZFS installation can span across a series of storage devices and is very scalable, allowing you to add disks to expand the available space in the storage pool immediately. + +ZFS is a block-based file system that protects against data corruption by using checksums to verify, confirm and correct every operation. +To run at a sufficient speed, this mechanism requires a powerful environment with a lot of RAM. + +In addition, ZFS offers snapshots and replication, RAID management, copy-on-write clones, compression and other features. + +## Terminology + +ZFS creates logical units based on physical storage devices. +These logical units are called *ZFS pools* or *zpools*. +Each zpool is then divided into a number of *datasets*. +These datasets can be of different types: +- A *ZFS filesystem* can be seen as a partition or a mounted file system. +- A *ZFS volume* represents a block device. +- A *ZFS snapshot* captures a specific state of either a ZFS filesystem or a ZFS volume. + ZFS snapshots are read-only. +- A *ZFS clone* is a writable copy of a ZFS snapshot. + ## `zfs` driver in LXD - When LXD creates a ZFS pool, compression is enabled by default.