Sinks

Use this page to understand how sinks are created, shared between loggers, and extended with custom implementations.

Sinks are objects responsible for writing logs to their respective targets.

A Sink object serves as the base class for various sink-derived classes.

Each sink handles outputting logs, metrics, or both to a single target, such as a file, console, database, or metrics backend.

Upon creation, a sink object is registered and owned by a central manager object, the SinkManager.

For files, one sink is created per normalized file path, and the file is opened once. If a sink is requested that refers to an already opened file, the existing Sink object is returned and any additional constructor arguments are ignored.

When creating a logger, one or more sinks for that logger can be specified. Sinks can only be registered during the logger creation.

Sinks can be obtained using FrontendImpl::get_sink(), FrontendImpl::create_or_get_sink(), or FrontendImpl::create_sink().

  • create_sink creates a new sink and throws QuillError if a sink with the same name already exists.

  • create_or_get_sink creates a new sink if one does not already exist; otherwise, it returns the existing sink with the specified name. Note that when a sink with the given name already exists, the provided constructor arguments are ignored — the existing sink is returned as-is.

  • get_sink retrieves an existing sink by name and throws QuillError if it does not exist.

Configuring Sinks

Some sinks accept a configuration object (e.g. FileSinkConfig, SyslogSinkConfig). These are passed as additional constructor arguments to the create_or_get_sink or create_sink call. A common pattern is to use a lambda to construct the config inline:

auto sink = quill::Frontend::create_or_get_sink<quill::SyslogSink>(
  "my_syslog",
  []()
  {
    quill::SyslogSinkConfig cfg;
    cfg.set_identifier("my_app");
    return cfg;
  }());

Sharing Sinks Between Loggers

It is possible to share the same Sink object between multiple Logger objects. For example, when all logger objects are writing to the same file. The following code is also thread-safe.

auto file_sink = Frontend::create_or_get_sink<FileSink>(
  filename,
  []()
  {
    FileSinkConfig cfg;
    cfg.set_open_mode('w');
    return cfg;
  }(),
  FileEventNotifier{});

quill::Logger* logger_a = Frontend::create_or_get_logger("logger_a", file_sink);
quill::Logger* logger_b = Frontend::create_or_get_logger("logger_b", file_sink);

Customizing the Library with User-Defined Sinks

You can extend the library by creating and integrating your own Sink types. The code within the Sink class is executed by a single backend worker thread.

This can be useful if you want to direct log output to alternative destinations, such as a database, a network service, or even to write Parquet files.

A custom sink can override write_log(), write_metric(), or both. The default Sink::write_metric() implementation is a no-op, so existing log-only sinks do not need to change. If you want to export metrics, implement write_metric() and bind a logger to that sink. See Metrics for the metric publishing model and the Prometheus/custom-sink examples.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
#include "quill/Backend.h"
#include "quill/Frontend.h"
#include "quill/LogMacros.h"
#include "quill/Logger.h"
#include "quill/sinks/Sink.h"

#include <cstdint>
#include <iostream>
#include <string>
#include <string_view>
#include <utility>
#include <vector>

/**
 * This example shows how to implement a custom sink by subclassing
 * `quill::Sink`.
 */

class CustomSink final : public quill::Sink
{
public:
  CustomSink() = default;

  /***/
  void write_log(quill::MacroMetadata const* /** log_metadata **/, uint64_t /** log_timestamp **/,
                 std::string_view /** thread_id **/, std::string_view /** thread_name **/,
                 std::string const& /** process_id **/, std::string_view /** logger_name **/,
                 quill::LogLevel /** log_level **/, std::string_view /** log_level_description **/,
                 std::string_view /** log_level_short_code **/,
                 std::vector<std::pair<std::string, std::string>> const* /** named_args - only populated when named args in the format placeholder are used **/,
                 std::string_view /** log_message **/, std::string_view log_statement) override
  {
    // This function is called by the logger backend worker thread for each LOG_* macro.

    // Typically, this is where you would write the message to a file, send it over the network, etc.

    // In this example, instead of immediately writing the log statement, we cache it.
    // This can be useful for batching log messages to a database for example.

    // The last character of log_statement is '\n', which we exclude by using size() - 1.
    _cached_log_statements.push_back(std::string{log_statement.data(), log_statement.size() - 1});
  }

  /***/
  void flush_sink() noexcept override
  {
    // This function is not called for each LOG_* invocation like the write function.

    // Instead, it is called periodically, when there are no more LOG_* writes left to process,
    // or when logger->flush() is invoked.

    // In this example, we output all our cached log statements at this point.

    for (auto const& message : _cached_log_statements)
    {
      std::cout << message << std::endl;
    }

    _cached_log_statements.clear();
  }

  /***/
  void run_periodic_tasks() noexcept override
  {
    // Executes periodic user-defined tasks. This function is frequently invoked by the backend thread's main loop.
    // Avoid including heavy tasks here to prevent slowing down the backend thread.

    // For example, this could be another place to submit a batch commit to a database, as this
    // function is called more frequently than `flush_sink`.
  }

private:
  std::vector<std::string> _cached_log_statements;
};

int main()
{
  // Start the backend thread
  quill::BackendOptions backend_options;
  quill::Backend::start(backend_options);

  auto file_sink = quill::Frontend::create_or_get_sink<CustomSink>("sink_id_1");
  quill::Logger* logger = quill::Frontend::create_or_get_logger("root", std::move(file_sink));

  LOG_INFO(logger, "Hello from {}", "sink example");
  LOG_INFO(logger, "Invoking user sink flush");

  logger->flush_log();

  LOG_INFO(logger, "Log more {}", 123);
}

Routing Log Messages to Multiple Sinks

The library provides multiple approaches for routing log messages to different sinks. The following examples demonstrate two common patterns using FileSink, but the same principles apply when using custom Sinks and Filters.

  1. Using multiple Logger instances, each bound to a specific Sink:

    This approach is straightforward and provides clear separation of log streams.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
#include "quill/Backend.h"
#include "quill/Frontend.h"
#include "quill/LogMacros.h"
#include "quill/Logger.h"
#include "quill/sinks/FileSink.h"

#include <utility>

int main()
{
  quill::BackendOptions backend_options;
  quill::Backend::start(backend_options);

  auto file_sink_a = quill::Frontend::create_or_get_sink<quill::FileSink>("sink_a.log");
  quill::Logger* logger_a = quill::Frontend::create_or_get_logger("logger_a", std::move(file_sink_a));

  auto file_sink_b = quill::Frontend::create_or_get_sink<quill::FileSink>("sink_b.log");
  quill::Logger* logger_b = quill::Frontend::create_or_get_logger("logger_b", std::move(file_sink_b));

  LOG_INFO(logger_a, "Hello from {}", "sink example");
  LOG_INFO(logger_a, "Using logger_a");

  LOG_INFO(logger_b, "Different data for sink B");
  LOG_INFO(logger_b, "Using logger_b");
}

  1. Using a single Logger instance with multiple Sinks and tag-based filtering:

    This approach centralizes logging through a single logger while still directing messages to different destinations.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
#include "quill/Backend.h"
#include "quill/Frontend.h"
#include "quill/LogMacros.h"
#include "quill/Logger.h"
#include "quill/sinks/FileSink.h"

#include <utility>

#define SINK_A_TAG "sink_a"
#define SINK_B_TAG "sink_b"

class TagsFilter : public quill::Filter
{
public:
  explicit TagsFilter(std::string tag) : quill::Filter("tags_filter"), _tag(std::move(tag))
  {
    // logging library adds by default an extra char and space to each tag, so we also add it manually for the correct comparison
    _tag.insert(0, "#");
    _tag.push_back(' ');
  };

  bool filter(quill::MacroMetadata const* log_metadata, uint64_t /** log_timestamp **/,
              std::string_view /** thread_id **/, std::string_view /** thread_name **/,
              std::string_view /** logger_name **/, quill::LogLevel /** log_level **/,
              std::string_view /** log_message **/, std::string_view /** log_statement **/) noexcept override
  {
    return log_metadata->tags() && (strcmp(log_metadata->tags(), _tag.data()) == 0);
  }

private:
  std::string _tag;
};

int main()
{
  quill::BackendOptions backend_options;
  quill::Backend::start(backend_options);

  auto file_sink_a = quill::Frontend::create_or_get_sink<quill::FileSink>("sink_a.log");
  file_sink_a->add_filter(std::make_unique<TagsFilter>(SINK_A_TAG));

  auto file_sink_b = quill::Frontend::create_or_get_sink<quill::FileSink>("sink_b.log");
  file_sink_b->add_filter(std::make_unique<TagsFilter>(SINK_B_TAG));

  quill::Logger* logger =
    quill::Frontend::create_or_get_logger("root", {std::move(file_sink_a), std::move(file_sink_b)});

  LOG_INFO_TAGS(logger, TAGS(SINK_A_TAG), "Hello from {}", "sink example");
  LOG_INFO_TAGS(logger, TAGS(SINK_A_TAG), "Using sink_a");

  LOG_INFO_TAGS(logger, TAGS(SINK_B_TAG), "Different data for sink B");
  LOG_INFO_TAGS(logger, TAGS(SINK_B_TAG), "Using sink_b");
}