Provided metrics
LXD provides a number of instance metrics and internal metrics. See How to monitor metrics for instructions on how to work with these metrics.
Instance metrics
The following instance metrics are provided:
| Metric | Description |
|---|---|
lxd_cpu_effective_total | Total number of effective CPUs |
lxd_cpu_seconds_total{cpu="<cpu>", mode="<mode>"} | Total number of CPU time used (in seconds) |
lxd_disk_read_bytes_total{device="<dev>"} | Total number of bytes read |
lxd_disk_reads_completed_total{device="<dev>"} | Total number of completed reads |
lxd_disk_written_bytes_total{device="<dev>"} | Total number of bytes written |
lxd_disk_writes_completed_total{device="<dev>"} | Total number of completed writes |
lxd_filesystem_avail_bytes{device="<dev>",fstype="<type>"} | Available space (in bytes) |
lxd_filesystem_free_bytes{device="<dev>",fstype="<type>"} | Free space (in bytes) |
lxd_filesystem_size_bytes{device="<dev>",fstype="<type>"} | Size of the file system (in bytes) |
lxd_memory_Active_anon_bytes | Amount of anonymous memory on active LRU list |
lxd_memory_Active_bytes | Amount of memory on active LRU list |
lxd_memory_Active_file_bytes | Amount of file-backed memory on active LRU list |
lxd_memory_Cached_bytes | Amount of cached memory |
lxd_memory_Dirty_bytes | Amount of memory waiting to be written back to the disk |
lxd_memory_HugepagesFree_bytes | Amount of free memory for hugetlb |
lxd_memory_HugepagesTotal_bytes | Amount of used memory for hugetlb |
lxd_memory_Inactive_anon_bytes | Amount of anonymous memory on inactive LRU list |
lxd_memory_Inactive_bytes | Amount of memory on inactive LRU list |
lxd_memory_Inactive_file_bytes | Amount of file-backed memory on inactive LRU list |
lxd_memory_Mapped_bytes | Amount of mapped memory |
lxd_memory_MemAvailable_bytes | Amount of available memory |
lxd_memory_MemFree_bytes | Amount of free memory |
lxd_memory_MemTotal_bytes | Amount of used memory |
lxd_memory_OOM_kills_total | The number of out-of-memory kills |
lxd_memory_RSS_bytes | Amount of anonymous and swap cache memory |
lxd_memory_Shmem_bytes | Amount of cached file system data that is swap-backed |
lxd_memory_Swap_bytes | Amount of used swap memory |
lxd_memory_Unevictable_bytes | Amount of unevictable memory |
lxd_memory_Writeback_bytes | Amount of memory queued for syncing to disk |
lxd_network_receive_bytes_total{device="<dev>"} | Amount of received bytes on a given interface |
lxd_network_receive_drop_total{device="<dev>"} | Amount of received dropped bytes on a given interface |
lxd_network_receive_errs_total{device="<dev>"} | Amount of received errors on a given interface |
lxd_network_receive_packets_total{device="<dev>"} | Amount of received packets on a given interface |
lxd_network_transmit_bytes_total{device="<dev>"} | Amount of transmitted bytes on a given interface |
lxd_network_transmit_drop_total{device="<dev>"} | Amount of transmitted dropped bytes on a given interface |
lxd_network_transmit_errs_total{device="<dev>"} | Amount of transmitted errors on a given interface |
lxd_network_transmit_packets_total{device="<dev>"} | Amount of transmitted packets on a given interface |
lxd_procs_total | Number of running processes |
Internal metrics
The following internal metrics are provided:
| Metric | Description |
|---|---|
lxd_api_requests_completed_total | Total number of completed requests. See API rates metrics. |
lxd_api_requests_ongoing | Number of requests currently being handled. See API rates metrics. |
lxd_go_alloc_bytes_total | Total number of bytes allocated (even if freed) |
lxd_go_alloc_bytes | Number of bytes allocated and still in use |
lxd_go_buck_hash_sys_bytes | Number of bytes used by the profiling bucket hash table |
lxd_go_frees_total | Total number of frees |
lxd_go_gc_sys_bytes | Number of bytes used for garbage collection system metadata |
lxd_go_goroutines | Number of goroutines that currently exist |
lxd_go_heap_alloc_bytes | Number of heap bytes allocated and still in use |
lxd_go_heap_idle_bytes | Number of heap bytes waiting to be used |
lxd_go_heap_inuse_bytes | Number of heap bytes that are in use |
lxd_go_heap_objects | Number of allocated objects |
lxd_go_heap_released_bytes | Number of heap bytes released to OS |
lxd_go_heap_sys_bytes | Number of heap bytes obtained from system |
lxd_go_lookups_total | Total number of pointer lookups |
lxd_go_mallocs_total | Total number of mallocs |
lxd_go_mcache_inuse_bytes | Number of bytes in use by mcache structures |
lxd_go_mcache_sys_bytes | Number of bytes used for mcache structures obtained from system |
lxd_go_mspan_inuse_bytes | Number of bytes in use by mspan structures |
lxd_go_mspan_sys_bytes | Number of bytes used for mspan structures obtained from system |
lxd_go_next_gc_bytes | Number of heap bytes when next garbage collection will take place |
lxd_go_other_sys_bytes | Number of bytes used for other system allocations |
lxd_go_stack_inuse_bytes | Number of bytes in use by the stack allocator |
lxd_go_stack_sys_bytes | Number of bytes obtained from system for stack allocator |
lxd_go_sys_bytes | Number of bytes obtained from system |
lxd_operations_total | Number of running operations |
lxd_uptime_seconds | Daemon uptime (in seconds) |
lxd_warnings_total | Number of active warnings |
API rates metrics
The API rates metrics include lxd_api_requests_completed_total and lxd_api_requests_ongoing. These metrics can be consumed by an observability tool deployed externally (for example, the Canonical Observability Stack (opens in a new tab) or another third-party tool) to help identify failures or overload on a LXD server. You can set thresholds on the observability tools for these metrics' values to trigger alarms and take programmatic actions.
These metrics consider all endpoints in the LXD REST API, with the exception of the / endpoint. Requests using an invalid URL are also counted. Requests against the metrics server are also counted. Both introduced metrics include a label entity_type based on the main entity type that the endpoint is operating on.
lxd_api_requests_ongoing contains the number of requests that are not yet completed by the time the metrics are queried. A request is considered completed when the response is returned to the client and any asynchronous operations spawned by that request are done. lxd_api_requests_completed_total contains the number of completed requests. This metric includes an additional label named result based on the outcome of the request. The label can have one of the following values:
error_server, for errors on the server side, this includes responses with HTTP status codes from 500 to 599. Any failed asynchronous operations also fall into this category.error_client, for responses with HTTP status codes from 400 to 499, indicating an error on the client side.succeeded, for endpoints that executed successfully.
Related topics
How-to guides:
Explanation: