Allow to trigger/control removing clean lines

[jvinolas]

Description

Would be useful to allow triggering clean lines removal when load is not high, and even better if we could set up how aggressive this removal thread is.

Reason

When hitting 100% cache occupancy the replacement thread for clean lines increases server cpu load, and that's quite a problem if the server is at that time handling a lot of I/O. So, having the option to run removal of clean lines when low I/O demand would be nice.

Your Environment

• OpenCAS version (commit hash or tag): c142610
• Operating System: 24.04.1 LTS (Noble Numbat)
• Kernel version: 6.9.12-060912-generic #202408281052 SMP PREEMPT_DYNAMIC
• Cache device type (NAND/Optane/other): ram disk (wa) over nvme (wb)
• Core device type (HDD/SSD/other): dm_vdo over an mdadm raid0 over iscii disks
• Cache configuration:

# casadm -P -i 1
Cache Id                  1
Cache Size                877915904 [4KiB Blocks] / 3348.98 [GiB]
Cache Device              /dev/nvme0n1p1
Exported Object           -
Core Devices              1
Inactive Core Devices     0
Write Policy              wb
Cleaning Policy           alru
Promotion Policy          always
Cache line size           64 [KiB]
Metadata Memory Footprint 4.0 [GiB]
Dirty for                 16090 [s] / 4 [h] 28 [m] 10 [s]
Status                    Running

# casadm -P -i 2
Cache Id                  2
Cache Size                65453856 [4KiB Blocks] / 249.69 [GiB]
Cache Device              /dev/ram0
Exported Object           -
Core Devices              1
Inactive Core Devices     0
Write Policy              wa
Cleaning Policy           alru
Promotion Policy          always
Cache line size           64 [KiB]
Metadata Memory Footprint 319.3 [MiB]
Dirty for                 0 [s] / Cache clean
Status                    Running

# casadm -L
type    id   disk             status    write policy   device
cache   1    /dev/nvme0n1p1   Running   wb             -
└core   1    /dev/dm-4        Active    -              /dev/cas1-1
cache   2    /dev/ram0        Running   wa             -
└core   1    /dev/cas1-1      Active    -              /dev/cas2-1
# casadm --get-param --name cleaning-alru -i 1
╔═════════════════════════╤═══════╗
║ Parameter name          │ Value ║
╠═════════════════════════╪═══════╣
║ Wake up time [s]        │     5 ║
║ Stale buffer time [s]   │     1 ║
║ Flush max buffers       │  9750 ║
║ Activity threshold [ms] │     0 ║
╚═════════════════════════╧═══════╝

# casadm --get-param --name cleaning-alru -i 2
╔═════════════════════════╤═══════╗
║ Parameter name          │ Value ║
╠═════════════════════════╪═══════╣
║ Wake up time [s]        │    20 ║
║ Stale buffer time [s]   │   120 ║
║ Flush max buffers       │   100 ║
║ Activity threshold [ms] │ 10000 ║
╚═════════════════════════╧═══════╝

• • Cache mode: wa rmdisk over wb nvme
• • Cache line size: 64
• • Promotion policy: always
• • Cleaning policy: alru
• • Sequential cutoff policy: full

# casadm --get-param --name seq-cutoff -i 1 -j 1
╔═════════════════════════════════════════════════════╤═══════╗
║ Parameter name                                      │ Value ║
╠═════════════════════════════════════════════════════╪═══════╣
║ Sequential cutoff threshold [KiB]                   │  1024 ║
║ Sequential cutoff policy                            │ full  ║
║ Sequential cutoff promotion request count threshold │     8 ║
╚═════════════════════════════════════════════════════╧═══════╝
# casadm --get-param --name seq-cutoff -i 2 -j 1
╔═════════════════════════════════════════════════════╤═══════╗
║ Parameter name                                      │ Value ║
╠═════════════════════════════════════════════════════╪═══════╣
║ Sequential cutoff threshold [KiB]                   │  1024 ║
║ Sequential cutoff policy                            │ full  ║
║ Sequential cutoff promotion request count threshold │     8 ║
╚═════════════════════════════════════════════════════╧═══════╝

• Other (e.g. lsblk, casadm -P, casadm -L)

[crabique]

The fact that there is no way to control the removal of clean lines causes some modes (e.g. WT) to actually induce very significant additional write latency on top of the already slow HDD cores once the occupancy hits 100%.

There absolutely needs to be a way to control how much clean cache is maintained, also ideally for some modes (e.g. WO) it would be great to completely opt out of having any clean cache, just remove the lines completely once they've been flushed..

[jvinolas]

I'ts a useless cache without this.

[crabique]

I'ts a useless cache without this.

It's still extremely useful in a lot of use-cases, but adding some tuning knobs to clean cache lines management would make it even better 🙂