Set Up a Logging Pipeline Instance

It is necessary to create an instance of the logging pipeline before sending log data to the Logging Service platform. For more information, see Log Pipelines.

When sending a request to create a logging pipeline, you can specify a unique tag of your choice for each log source. For more information about the complete list of available sources, see Log Sources.

This topic contains the following sections:

• Request to create a logging pipeline

• Sample response

• Create a pipeline with custom labels

Request to create a logging pipeline

The following request creates an instance of a logging pipeline with two log streams: docker and kubernetes.

curl --location \ 
--request POST 'https://logging.de-txl.ionos.com/pipelines' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $TOKEN' \
--data '{
  "properties": {
    "name": "demo",
    "logs": [
      {
        "source": "docker",
        "tag": "dock",
        "protocol": "tcp",
        "labels": [
          "label1d"
        ],
        "destinations": [
          {
            "type": "loki",
            "retentionInDays": 7
          }
        ]
      }, 
            {
             "source": "kubernetes",
             "tag": "k8s",
             "protocol": "tcp",
            "destinations":  
            [
              {
              "type": "loki",
              "retentionInDays": 14
              }
            ]
           }
    ]
  }
}'

Response

The following is a sample response. The values returned by each response differ based on the request.

{
    "id": "aaaaa-bbbb-1234-cccc-dddd",
    "type": "Pipeline",
    "metadata": {
        "createdDate": "2023-10-19T11:48:31Z",
        "createdBy": "[e-mail]",
        "createdByUserId": "ID",
        "createdByUserUuid": "UUID",
        "lastModifiedDate": "2023-10-19T11:48:31Z",
        "lastModifiedBy": "[e-mail]",
        "lastModifiedByUserId": "ID",
        "lastModifiedByUserUuid": "UUID",
        "status": "PROVISIONING"
    },
    "properties": {
        "name": "demo",
        "logs": [
            {
                "public": true,
                "source": "docker",
                "tag": "dock",
                "destinations": [
                    {
                        "type": "loki",
                        "retentionInDays": 7
                    }
                ],
                "labels": [
                    "label1d"
                ],
                "protocol": "tcp"
            },
            {
                "public": true,
                "source": "kubernetes",
                "tag": "k8s",
                "destinations": [
                    {
                        "type": "loki",
                        "retentionInDays": 14
                    }
                ],
                "protocol": "tcp"
            }
        ],
        "tcpAddress": "",
        "httpAddress": "",
        "grafanaAddress": "",
        "key": "key"
    }
}

You may notice that the pipeline's status is temporarily set to the PROVISIONING state while provisioning is in process. A GET request retrieves information about the pipeline and its status. For more information, see Retrieve logging pipeline information.

Create a pipeline with custom labels

Log sources like Kubernetes, Docker, and Linux Systemd collect and offer relevant labels. You can use these labels to analyze reports and query the dashboard. However, if you want to label additional fields from the log sources, you can define custom labels as follows when you create a pipeline:

curl --location \ 
--request POST 'https://logging.de-txl.ionos.com/pipelines' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $TOKEN' \
--data '{
    "properties": {
        "name": "demo",
        "logs": [
            {
                "source": "docker",
                "tag": "dock",
                "protocol": "tcp",
                "labels": [
                    "label1",
                    "label2"
                ]
            }
        ]
    }
}'