Carbonaut Data Schema

Carbonaut Data Schema

This page describes how the Carbonaut Data Schema is structured and build. The schema is defined in protobuf (opens in a new tab) format as a programming language agnostic data schema which also allows to generate code for various programming languages plus gRPC (opens in a new tab) APIs for Carbonaut plugins. The protobuf files are located in the api (opens in a new tab) folder inside the Carbonaut project repository.

To integrate various vendor specific data sources, Carbonaut uses plugins to translate it into Carbonaut Data Transfer objects (DTOs). These DTOs are then used to generate the Carbonaut Metrics Schema which in turn are exposed a metrics. The data flow is described in the Carbonaut architecture data flow section.

The following diagram illustrates how the Carbonaut data schema is structured.

Carbonaut Data Schema Overview

  1. Carbonaut Data Models: define generic data models that are used by the Carbonaut Data Transfer Objects (DTOs) and the Carbonaut Metrics Schema.
  2. Carbonaut Data Transfer Objects (DTOs): define the intermediate representation of the data that is used to generate the Carbonaut Data Schema.
  3. Carbonaut Metrics Schema: define the final representation of the data that is exposed as metrics in opentelemetry format.

Carbonaut Data Transfer Objects

The Carbonaut Data Transfer Objects (DTOs) are the intermediate representation of the data that is used to generate the Carbonaut Data Schema. The DTOs are generated by the Carbonaut Data Plugins. The api/plugin/dto.proto (opens in a new tab) file contains the protobuf definition of the DTOs.

  • Electricity DTO Record: Contains the data of a single electricity meter reading.
  • ITResource DTO Record: Contains the data of a single IT resource reading. This can be a server, a virtual machine, a container, etc. This also includes hardware information.
  • Emission DTO Record: Contains the data of a single emission reading.

Since this is an intermediate representation, the DTOs are not exposed as metrics in opentelemetry format directly. Instead, they are used to generate the Carbonaut Data Schema.

Carbonaut Metrics Schema

The Carbonaut Metrics Schema is the final representation of the data that is exposed as metrics in opentelemetry format. The api/metrics (opens in a new tab) folder contains the protobuf files that define the Carbonaut Metrics Schema.

  • Meta Metrics: The meta metrics format expose metrics about the metrics collection itself and gives cross cutting information. For example, plugin data fetching times, records by plugin, cpu and memory utilization by plugins, plugin data transfomation errors, etc. For more detail, see source file api/models/meta (opens in a new tab).
  • Electricity Metrics: The electricity metrics format expose metrics about electricity consumption.
  • IT Resource Metrics: The IT resource metrics format expose metrics about IT resource consumption.
  • Emission Metrics: The emission metrics format expose metrics about emissions.
  • Mixed Metrics: The mixed metrics combine electricity, it-resource and emission metrics by configuration into one schema. It's used to map for example energy consumption to a specific it infrastructure stack. For more detail, see the data mixer documentation.

In the future more metrics with additional analytics might be added to the Carbonaut Metrics Schema (like forcasts, etc.).