Skip to content

Folder

Checks the contents of a folder for size, age and count. Folder based checks are useful in a number of scenarios:

  • Verifying that backups have been uploaded and are the appropiate size
  • Checking that logs or other temporary files are being cleaned up
  • For batch processes:
  • Checking if files are being processed (and/or produced)
  • Checking the size of queue processing backlog
  • Checking if any error (.err or .log) files have been produced.
folder-check.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: folder-check
spec:
  interval: 30
  folder:
    - path: /etc/
      name: folder-check-min
      description: Checks if there are at least 10 files in the folder
      minCount: 10
Field Description Scheme Required
path A local folder path or a remote folder (SMB, SFTP, S3, GCS) string Yes
filter Filter objects out before checking if they are valid FolderFilter
minCount The minimum number of files inside the path int
maxCount The maximum number of files inside the path, can be used in conjunction with filter.regex to detect error files int
minAge The youngest age a file can be Duration
maxAge The oldest age a file can be, often used to check for unprocessed files or files that have not been cleaned up Duration
minSize The minimum file size, can be used to detect backups that did not upload successfully Size
maxSize The maximim file size Size
* All other commons field Common

Duration

Durations are strings with an optional fraction and unit e.g. 300ms, 1.5h or 2h45m. Valid time units are ms, s, m, h.

Size

Sizes are string with a unit suffix e.g. 100 / 100b, 10mb, Valid size units are kb, mb, gb, tb

FolderFilter

Field Description Scheme Required
maxAge MaxAge the latest object should be younger than defined age Duration
maxSize MaxSize of the files inside the searchPath Size
minAge MinAge the latest object should be older than defined age Duration
minSize MinSize of the files inside the searchPath Size
regex Filter files based on regular expression regex

e.g. to verify that database backups are being performed

postgres-backup-check.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: folder-check
spec:
  interval: 30
  folder:
    - path: /data/backups
      filter:
        regex: "pg-backups-.*.zip"
      maxAge: 1d # require a daily backup
      minSize: 10mb # the backup should be at least 10mb

Result Variables

The following fields are available in test, display and transform scripts.

Field Scheme
Oldest os.FileInfo
Newest os.FileInfo
MinSize os.FileInfo
MaxSize os.FileInfo
SupportsTotalSize (Only true for SMB folders) bool
SupportsAvailableSize (Only true for SMB folders) bool
TotalSize int64
AvailableSize int64
Files []os.FileInfo

FolderFilter

Field Description Scheme Required
minAge Duration
maxAge Duration
minSize Size
maxSize Size
regex string