CSV Writing

Use this page to write structured CSV data asynchronously using Quill’s backend thread.

The library provides functionality for asynchronously writing CSV files. Formatting and I/O operations are managed by the backend thread, allowing for efficient and minimal-overhead CSV file writing on the hot path. This feature can be used alongside regular logging.

The CsvWriter class is a utility designed to facilitate asynchronous CSV file writing.

Call CsvWriter::close() before stopping the backend worker if you need deterministic logger removal and file closure. The destructor performs best-effort asynchronous cleanup and does not block.

CSV Writing To File

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
#include "quill/Backend.h"
#include "quill/CsvWriter.h"
#include "quill/LogMacros.h"
#include "quill/core/FrontendOptions.h"
#include "quill/sinks/ConsoleSink.h"

/**
 * This example shows how to define a CSV schema, instantiate a CsvWriter object, and append rows to
 * the CSV file. The logging operations are handled asynchronously by the backend worker thread.
 */

struct OrderCsvSchema
{
  static constexpr char const* header = "order_id,symbol,quantity,price,side";
  static constexpr char const* format = "{},{},{},{:.2f},{}";
};

int main()
{
  quill::BackendOptions backend_options;
  quill::Backend::start(backend_options);

  quill::Logger* logger = quill::Frontend::create_or_get_logger(
    "root", quill::Frontend::create_or_get_sink<quill::ConsoleSink>("sink_id_1"));

  LOG_INFO(logger, "CSV writing example");

  quill::CsvWriter<OrderCsvSchema, quill::FrontendOptions> csv_writer{"orders.csv"};
  csv_writer.append_row(13212123, "AAPL", 100, 210.32321, "BUY");
  csv_writer.append_row(132121123, "META", 300, 478.32321, "SELL");
  csv_writer.append_row(13212123, "AAPL", 120, 210.42321, "BUY");
  csv_writer.close();
}

Csv output (orders.csv):

order_id,symbol,quantity,price,side
13212123,AAPL,100,210.32,BUY
132121123,META,300,478.32,SELL
13212123,AAPL,120,210.42,BUY

CSV Writing To Existing Sink

It is possible to pass an existing Sink, or a custom user-created Sink, to the CSV file for output. The following example shows how to use the console sink.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
#include "quill/Backend.h"
#include "quill/CsvWriter.h"
#include "quill/LogMacros.h"
#include "quill/core/FrontendOptions.h"
#include "quill/sinks/ConsoleSink.h"

struct OrderCsvSchema
{
  static constexpr char const* header = "order_id,symbol,quantity,price,side";
  static constexpr char const* format = "{},{},{},{:.2f},{}";
};

int main()
{
  quill::Backend::start();

  quill::Logger* logger = quill::Frontend::create_or_get_logger(
    "root", quill::Frontend::create_or_get_sink<quill::ConsoleSink>("sink_id_1"));

  LOG_INFO(logger, "CSV writing example");

  // Pass the existing ConsoleSink to the CsvWriter to also emit CSV rows there
  quill::CsvWriter<OrderCsvSchema, quill::FrontendOptions> csv_writer{
    "orders.csv", quill::Frontend::get_sink("sink_id_1")};

  csv_writer.append_row(13212123, "AAPL", 100, 210.32321, "BUY");
  csv_writer.append_row(132121123, "META", 300, 478.32321, "SELL");
  csv_writer.append_row(13212123, "AAPL", 120, 210.42321, "BUY");
  csv_writer.close();
}