Module: OpenObserve::DirectorySource

Included in:
Alerts, Dashboards, Destinations, Templates
Defined in:
lib/openobserve/directory_source.rb

Overview

Reads a directory of JSON documents into a hash keyed by one of their fields.

Shared by the sync of every resource that is versioned as files, because the failure modes are the same whatever the resource: a directory that moved, a document mangled by a bad merge, two files that ended up claiming the same object. Each is refused with the offending filename, since a synchronisation that half-applies is worse than one that refuses to start.

The key doubles as the idempotence key — it is what a later run matches against what the server already holds — so a document without one is refused rather than created afresh on every push.

Instance Method Summary collapse

Instance Method Details

#read_definitions(directory, pattern, key, noun: 'document') ⇒ Hash{String=>Hash}

Returns key => document, in filename order.

Parameters:

  • directory (String) —

    directory holding the documents

  • pattern (String) —

    glob applied inside directory

  • key (String) —

    field whose value identifies the document

  • noun (String) (defaults to: 'document') —

    what the documents are, so a refusal names it

Returns:

  • (Hash{String=>Hash}) —

    key => document, in filename order

Raises:

  • (OpenObserve::Client::Error) —

    unreadable directory, invalid JSON, non-object document, missing key, or two files claiming the same key



23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
 
# File 'lib/openobserve/directory_source.rb', line 23

def read_definitions(directory, pattern, key, noun: 'document')
  raise Client::Error, "no such #{noun} directory: #{directory}" unless Dir.exist?(directory)

  seen = {}
  # Dir.glob has sorted its result since Ruby 3.0, which is what makes the run order —
  # and therefore the created/updated report — stable across machines.
  Dir.glob(File.join(directory, pattern)).each_with_object({}) do |file, out|
    definition = parse_definition(file, noun)
    name       = key_of(definition, file, key, noun)

    raise Client::Error, "two files claim the #{key} #{name.inspect}: #{seen[name]} and #{file}" if seen.key?(name)

    seen[name] = file
    out[name]  = definition
  end
end