The IONOS Logging Service provides a centralized and scalable solution for logging, monitoring, and analyzing application and infrastructure logs. It offers a wide range of features to help you collect and analyze logs from different log sources, monitor performance effectively, and gain insights into your system's behavior.

Product Overview

Quick Links

Quick Start

Frequently Asked Questions (FAQs)

To get answers to the most commonly encountered questions about the Logging Service platform, see .

Logging Service is a cloud-based service that allows you to store, manage, and analyze logs generated by your applications or systems. It is a scalable and cost-effective solution for organizations that need to manage and analyze large volumes of logs.

Logging Service is also an efficient log management solution, which contains several important features and benefits that can help organizations improve their log management practices and troubleshoot issues more quickly.

Log data is the key to an early detection of security incidents; thus, using the Logging Service can improve your operational efficiency, enhance your security, and increase your visibility into your system's performance and errors. Log data is available 24/7, secure, high-performance, and scalable. Your data is encrypted before, during, and after it is transmitted. The Fluent Bit agent gathers log data locally, while the Logging Service forms the basis for analysis and better visualization of your logs with increased security.

The managed Grafana in the Logging Service comes with a pre-configured datasource for the Telemetry API called IONOS Telemetry. You can use this datasource to query metrics from the IONOS Cloud Telemetry API. For more information, see Integration with IONOS Telemetry API.

Logging Service also enables you to configure an unlimited log retention period for your logs. For more information, see Modify the Log Retention Policy.

Components of Logging Service

The architecture of the Logging Service includes the following three main components that can be used to aggregate logs from various sources, analyze the gathered logs, and create visualizations for monitoring and report generation.

• Data Collection: Data is collected from various log sources, such as applications, servers, and network devices, which are sent to a centralized Logging Service platform. Currently, the following log sources are supported: Kubernetes, Docker, Linux Systemd, HTTP (JSON REST API), and Generic.

• Logging Service Platform: The Logging Service platform stores the logs gathered from various sources in a centralized location for easy access. This data can be accessed for analysis and troubleshooting. The platform includes log search, routing, storage, analytics, and visualization features. For more information, see .

• Analytics and Visualization: Grafana, an analytics and visualization tool allows you to analyze the log data to identify patterns and trends and visualize the log data or generate reports. You can also use these reports to secure your log data from threats or troubleshoot underlying issues.

The illustration shows the default components of the Logging Service platform and the following is a brief description of the components:

• Systems (1, 2, and n): These are the various log sources with Fluent Bit installed to gather, parse, and redirect logs to the Logging Service platform.

• Logging Service Platform:

  • It contains an in-built log aggregator, Fluentd, easily compatible with various sources and targets. Hence, shipping logs from the source to the destination is hassle-free. It aggregates, processes, and ships data to the stipulated target.

Limitations

The following are the key limitations of Logging Service:

You can set user privileges for accessing and managing Logging Service via the DCD.

A centralized Logging Service platform consists of two major components: Log Collection and Log Aggregation. The responsibilities of the platform provider and the user differ in the context of Logging Service.

Log Collection: The responsibility for log collection and its configuration lies with the user. This involves setting up mechanisms to gather log data from various sources within the infrastructure and applications. These mechanisms can include agents, log shippers, or APIs that send log data to a central location for storage and analysis.

Log Aggregation: The Logging Service platform provider provides and manages the log aggregation component. This component involves the centralization of log data from multiple sources, making it accessible for analysis and visualization. The platform handles log storage, indexing, and search functionalities.

Scenario 1: Streamlined log centralization and analysis

Precondition

Maintaining log visibility is challenging in a distributed system with numerous services and environments. Logs are scattered across different nodes, making troubleshooting, performance monitoring, and security analysis complex tasks.

Users need appropriate privileges to access, create, and modify Logging Service. Without necessary privileges, users have a read-only access and cannot provision changes. You can grant appropriate privileges via the User Manager.

To allow users to access, create, and modify the Logging Service, follow these steps:

1. In the DCD, go to Menu > Management > Users & Groups. 2. Select the Groups tab in the User Manager window. 3. Select the appropriate group to assign relevant privileges. 4. In the

A log pipeline refers to an instance or configuration of the Logging Service you can create using the or the . To create an instance of the Logging Service, you can request the designated regional endpoint based on your desired location:

• Berlin: https://logging.de-txl.ionos.com/pipelines

• Frankfurt: https://logging.de-fra.ionos.com/pipelines

You can modify your logging pipeline by sending a PATCH request with a specific pipeline ID.

The following is a sample request. Remember to replace the {pipelineID} with a valid ID of the respective logging pipeline.

The Logging Service is a versatile and accessible platform that allows you to conveniently store and manage logs from various sources. This platform offers a seamless log aggregation solution for logs generated within the IONOS infrastructure, by your bare metal system, or another cloud environment. With its flexibility, you can effortlessly push logs from anywhere, ensuring comprehensive log monitoring and analysis.

The following two encryption mechanisms safeguard all HTTP or TCP communications that push logs:

• Transport Layer Security (TLS)

• KEY

  • If using HTTP, then the APIKEY must be specified in the header.

  • If using TCP, specify the Shared_Key.

The key brings an extra layer of security with which you can revoke or regenerate the existing key.

TCP (Fluent Bit)

When using TCP or TLS, you must enable tls and provide a Shared_Key in the Fluent Bit configuration. The key can be obtained through the REST API. For more information, see .

HTTP (cURL)

If using HTTP (JSON), provide the key in the header as shown in the following example:

HTTP (Fluent Bit)

This is an equivalent example of configuring Fluent Bit with HTTP outbound:

The Telemetry API is an API that allows you to interact with the IONOS Cloud Telemetry service, and it is compatible with Prometheus specifications.

The Telemetry API allows retrieval of instance metrics; it is a read-only API and does not support any write operations. Although the Prometheus specification contains many more API resources and operations, the Telemetry API selectively supports the following GET operations at the moment:

/api/v1/label
/api/v1/query
/api/v1/query_range
/api/v1/metadata
/api/v1/series

Integration with IONOS Telemetry API

The managed Grafana in the Logging Service comes with a pre-configured datasource for the Telemetry API called IONOS Telemetry. You can use this datasource to query metrics from the IONOS Cloud Telemetry API.

Authentication

The Telemetry API uses the same authentication as the IONOS Cloud API. You can use the same API token to authenticate with the Telemetry API. This means you need to update the IONOS Telemetry datasource with your API token:

Follow the instructions in to generate a token.

Once the header is configured, select Save & test.

To delete your logging pipeline, you need the ID of the respective pipeline.

Request

The following is a sample request. Remember to replace the {pipelineID} with a valid ID of the specific pipeline that must be deleted.

This section lists the key features and benefits of using the Logging Service.

Features

• Scalability: It is designed to handle extensive volumes of logs and scale up or down according to needs.

A logging pipeline refers to an instance of the logging service set up to receive, process, and store log data from various sources.

To create a logging pipeline, you need to follow these steps:

2. Click Create logging pipeline from the Logging Service overview page.

3. Configure the following details for a logging pipeline:

4. Click Create to create a logging pipeline.

Define logging pipeline properties

To define the logging pipeline properties, enter the following details:

1. Name: Enter a name for the logging pipeline. The name is only for your reference and does not have to be unique. Example, my-logging-pipeline.

2. Region: Select a region from the drop-down list to create the logging pipeline in that specific region. The region determines where your logs will be processed and stored.

Define log sources

The log sources refer to the origins from which the log agent will collect logs. You can define multiple log sources for a single logging pipeline, and each log source can be configured with specific settings.

To define the log sources for a logging service, enter the following details:

1. Source Type: Select the log source type from the drop-down list. It defines the source from which you want to send logs. The available options are:

• Kubernetes: For sending logs from Kubernetes cluster.

• Docker: For sending logs from Docker containers.

• Systemd: For sending logs from systemd journal.

• Generic: For sending logs from any other source that is not covered by the above options.

2. Tag: Enter a unique tag for the log source. The tag is used to identify the log source and can be used for filtering and searching logs later. Example, myk8s.

3. Protocol: Select one of the protocols from the drop-down list for sending logs. The available options are:

• TCP: For sending logs over TCP.

• HTTP: For sending logs over HTTP.

4. Retention: Choose the retention period for the logs collected from this source. The retention period determines how long the logs will be stored before they are automatically deleted. The available options are:

• 7 days

• 14 days

• 30 days

• Unlimited

5. Click Add source to define another log source. Repeat the steps from step 1 to step 4 to add the log source and click Create. You can add up to 5 log sources per logging pipeline.

Once a logging pipeline is successfully created, the instance is listed on the Logging Service overview page.

To view the logging pipeline details, follow these steps:

View details of a selected Logging Pipeline

For the selected logging pipeline, you can view the following details:

• Properties: The following system information related to the logging pipeline is displayed:

  • Name: Displays the name of the logging pipeline.

  • Region: Displays the region of the logging pipeline.

  • Status: Displays the state of the logging pipeline.

• Log sources: Refers to the log sources defined for the logging pipeline. For more information, see .

What is a Logging Service?

The Logging Service platform provides a centralized and scalable solution for collecting, monitoring, and analyzing logs from your application and infrastructure. It utilizes specialized components to aggregate, parse, and store the logs seamlessly.

The Logging Service offers regional APIs that enable programmatic interaction with the platform. These APIs serve various purposes: task automation, system integration, and platform functionality extension. Additionally, the APIs allow you to filter logs based on different criteria, such as the date range, log level, and source.

Sub-user access control

A sub-user is a user who has access to the Logging Service but is not an administrator or an owner. IONOS's crucial access control restriction does not allow sub-users to view or modify pipelines belonging to other sub-user accounts, except the primary administrator, who retains full cross-pipeline privileges. Ensure that the sub-user pipeline ownership and access permissions align with your organizational needs.

If a sub-user account creates a pipeline, access is restricted only to that sub-user and the primary administrator. Other sub-users cannot access or perform CRUD operations on the respective pipeline. For example, if sub-user A creates Pipeline 1, only sub-user A and the primary administrator account can view, edit, delete, or manage Pipeline 1. No other sub-user accounts will have access to it.

Once a logging pipeline is successfully created, the instance is listed on the Logging Service overview page.

To update the logging pipeline details, follow these steps:

2. In the Logging Service overview page, select the logging pipeline to update.

3. In View & Edit, update the logging pipeline details needed:

• Properties: Refers to the generic properties of the logging pipeline. To update these details, see .

To delete a logging pipeline, follow these steps:

2. From the Logging Service overview page, select the logging pipeline you want to delete. This page lists all the created logging pipelines.

3. In the Options column for the selected logging pipeline, click and select Delete Pipeline.

To retrieve your logging pipeline information, you need the ID of the respective pipeline.

Request

The following is a sample request. Remember to replace the {pipelineID} with a valid ID of the specific pipeline whose information you want to access.

curl --location \
--request GET 'https://logging.de-txl.ionos.com/pipelines/{pipelineID}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $TOKEN'

Response

If your request is successful, you will receive the relevant information on a Ready pipeline.

To access logs using Fluent Bit, you need the following information:

It is necessary to create an instance of the logging pipeline before sending log data to the Logging Service platform. For more information, see .

For more information about the complete list of available sources, see .

This topic contains the following sections:

It is essential to identify the origin of the logs before choosing the right approach to installing and configuring the agent. However, to provide convenient parsing and data labeling, IONOS accepts logs from the following four log sources: Kubernetes, Docker, Linux Systemd, and HTTP. The configuration of these log sources varies accordingly.

A key is necessary to send logs over the logging pipeline. For more information about the purpose of a pipeline key, see Log Security.

Request

To get a new key for a pipeline, you can use the following request. Remember to replace the {pipelineID} with a valid ID of a pipeline to access its key.

Response

The response contains the key necessary to configure the Shared_Key in Fluent Bit.