Setup a Logging Pipeline Instance

To create a logging pipeline instance, you must meet the following terms:

1. You must have a valid and billable contract at IONOS.

2. You must have enough permission to manage the Data Center.

3. You must be the contract owner to be able to interact with the REST API.

Request

The following request creates a logging pipeline with 2 log streams.

curl -X "POST" "https://logging.de-txl.ionos.com/pipelines" \
     -H 'Content-Type: application/json' \
     -H 'Authorization: Bearer $TOKEN' \
     -d $'{
  "properties": {
    "name": "demo",
    "logs": [
      {
        "source": "docker",
        "tag": "dock",
        "protocol": "tcp"
      },
      {
        "source": "kubernetes",
        "tag": "k8s",
        "protocol": "tcp"
      }
    ]
  }
}'

This request will eventually create an instance of a logging pipeline with 2 log streams: Docker and Kubernetes. As you can see, each source must use a unique tag of your choice.

To see the full list of available sources, please refer to our log sources section.

Response

The following is just a sample of the response; your values will be different.

{
  "id": "6e9390e3-5e72-4c48-b28e-33767b8eb610",
  "type": "Pipeline",
  "metadata": {
    "createdDate": "2023-05-15T08:24:20Z",
    "createdBy": "[e-mail]",
    "createdByUserId": "2895",
    "createdByUserUuid": "f23564a8-2ad6-4826-b12f-9fec79866314",
    "lastModifiedDate": "",
    "lastModifiedBy": "",
    "lastModifiedByUserId": "",
    "lastModifiedByUserUuid": "",
    "status": "NotReady"
  },
  "properties": {
    "name": "demo",
    "logs": [
      {
        "public": true,
        "source": "docker",
        "tag": "dock",
        "destinations": [
          {
            "type": "loki"
          }
        ],
        "protocol": "tcp",
        "status": "NotReady"
      },
      {
        "public": true,
        "source": "kubernetes",
        "tag": "k8s",
        "destinations": [
          {
            "type": "loki"
          }
        ],
        "protocol": "tcp",
        "status": "NotReady"
      }
    ],
    "tcpAddress": "",
    "httpAddress": "",
    "grafanaAddress": ""
  }
}

The pipeline will remain in the NotReady status for a short amount of time till its provisioning is finished. You can get the pipeline information and its status by the following request.

Create a pipeline with custom labels

log sources like kubernetes, docker, and systemd collect and provide the relevant labels, which can be used in report analysis and dashboard queries.

However, you might need to label a few more fields from the log sources additionally. You can define the additional labels as follows when you create a pipeline:

curl -X "POST" "https://logging.de-txl.ionos.com/pipelines" \
     -H 'Content-Type: application/json' \
     -H 'Authorization: Bearer $TOKEN' \
     -d $'{
  "properties": {
    "name": "demo",
    "logs": [
      {
        "source": "docker",
        "tag": "dock",
        "protocol": "tcp",
        "labels": [
          "label1",
          "label2"
        ]
      }
    ]
  }
}'

---

Get a Logging Pipeline

To get your pipeline information, you can use the following request.

Request

curl "https://logging.de-txl.ionos.com/pipelines/6e9390e3-5e72-4c48-b28e-33767b8eb610" \
     -H 'Content-Type: application/json' \
     -H 'Authorization: Bearer $TOKEN'

Response

On a successful request, you will receive important information on a Ready pipeline.

{
  "id": "6e9390e3-5e72-4c48-b28e-33767b8eb610",
  "type": "Pipeline",
  "metadata": {
    "createdDate": "2023-05-15T08:24:20Z",
    "createdBy": "[e-mail]",
    "createdByUserId": "2895",
    "createdByUserUuid": "f23564a8-2ad6-4826-b12f-9fec79866314",
    "lastModifiedDate": "",
    "lastModifiedBy": "",
    "lastModifiedByUserId": "",
    "lastModifiedByUserUuid": "",
    "status": "Ready"
  },
  "properties": {
    "name": "demo",
    "logs": [
      {
        "public": true,
        "source": "docker",
        "tag": "dock",
        "destinations": [
          {
            "type": "loki"
          }
        ],
        "protocol": "tcp",
        "status": "Ready"
      },
      {
        "public": true,
        "source": "kubernetes",
        "tag": "k8s",
        "destinations": [
          {
            "type": "loki"
          }
        ],
        "protocol": "tcp",
        "status": "Ready"
      }
    ],
    "tcpAddress": "tcp-d3bc487b965f-logs.28f76fb7d505.logging.de-txl.ionos.com:9000",
    "httpAddress": "",
    "grafanaAddress": "grafana.28f76fb7d505.logging.de-txl.ionos.com"
  }
}

You need the following pieces of information to configure the FluentBit and access the logs:

Field	Usage
tcpAddress	This is the address you need to set in the FluentBit configuration for the TCP log server.
httpAddress	This is the address you need to set in the FluentBit configuration for the HTTP log server.
grafanaAddress	This is the address where you can login with your IONOS credentials to access the logs.
tag	This is the same tag you defined while creating the pipeline, which needs to be set in the FluentBit configuration.

You need to obtain one last piece of information to send logs, the Key. Follow the instruction on how to obtain a key.