Grafana Alloy is an OpenTelemetry Collector distribution with programmable pipelines.


Wildfly 10 with Otel Java Agent to Grafana Cloud

Collect telemetry from Wildfly 10 with the OpenTelemetry Instrumentation for Java agent, and then push the telemetry, via Grafana Alloy, to Grafana Cloud:

  1. Configure and install Grafana Alloy (on Fedora/RHEL):

    curl -OL
    sudo dnf install alloy-1.0.0-1.amd64.rpm
    rpm --query --list --provides alloy-1.0.0-1.amd64.rpm
    sudo rpm -i alloy-1.0.0-1.amd64.rpm
  2. Pass OTLP/Grafana endpoint details to Alloy by setting environment variables in the systemd unit directly:

    export OTLP_ENDPOINT=
    export BASIC_AUTH_USER="___0___"
    export API_KEY="glc_eyJ......"
    sudo mkdir -p /etc/systemd/system/alloy.service.d
    sudo touch /etc/systemd/system/alloy.service.d/environment.conf
    sudo tee -a /etc/systemd/system/alloy.service.d/environment.conf << EOF
    sudo systemctl daemon-reload
  3. Add OTLP components to the default Alloy configuration file, and then restart.

    sudo tee -a /etc/alloy/config.alloy << EOF
    otelcol.exporter.otlphttp "default" {
      client {
        endpoint = env("OTLP_ENDPOINT")
        auth     = otelcol.auth.basic.credentials.handler
    otelcol.auth.basic "credentials" {
      // Retrieve credentials using environment variables.
      username = env("BASIC_AUTH_USER")
      password = env("API_KEY")
    otelcol.receiver.otlp "example" {
      grpc {
        endpoint = ""
      http {
        endpoint = ""
      output {
        metrics = [otelcol.exporter.otlphttp.default.input]
        logs    = [otelcol.exporter.otlphttp.default.input]
        traces  = [otelcol.exporter.otlphttp.default.input]
    sudo systemctl restart alloy
  4. Download and start Wildfly with the OpenTelemetry Java agent attached:

    curl -OL
    tar xvf wildfly-10.0.0.Final.tar.gz
    cd wildfly-10.0.0.Final/
    curl -OL
    export JAVA_OPTS="-javaagent:$PWD/opentelemetry-javaagent.jar"
    # Download Java 8 if you don't have it already
    curl -s "" | bash
    sdk install java 8.0.402-tem
  5. Build and deploy the “kitchensink” example app:

    git clone wildfly-quickstart
    cd wildfly-quickstart
    git checkout 10.x
    cd kitchensink
    mvn wildfly:deploy
  6. Visit http://localhost:8080/wildfly-kitchensink/ in a web browser and make some requests. Or, use the REST API:

    curl -v -H 'Content-type: application/json' -X POST http://localhost:8080/wildfly-kitchensink/rest/members -d '{ "name": "Benny", "email": "", "phoneNumber": "447999332451" }'

From this, you should get some useful out-of-the-box metrics, logs and traces in Grafana Cloud:

  • Metrics
    • http_server_request_duration_seconds_count (and _bucket and _sum) - e.g. http_server_request_duration_seconds_sum{http_route="/wildfly-kitchensink/rest/members/{id:[0-9][0-9]*}"}
    • jvm_cpu_count
    • jvm_memory_limit_bytes, jvm_memory_committed_bytes
    • target_info – an “info” style metric, with info like process PID, command line, OS version, etc
  • Logs: you’ll also get some logs with OpenTelemetry resource attributes
  • Traces: e.g. search for service name “JBoss Modules”

Example configs

Namespace-scoped Role and RoleBinding

apiVersion: v1
kind: ServiceAccount
  name: grafana-agent
  namespace: myapp
kind: Role
  name: grafana-agent
  namespace: myapp
- apiGroups:
  - ""
  - services
  - endpoints
  - pods
  - events
  - get
  - list
  - watch
kind: RoleBinding
  name: grafana-agent
  namespace: myapp
  kind: Role
  name: grafana-agent
- kind: ServiceAccount
  name: grafana-agent
  namespace: myapp


Metrics do not appear in Grafana Cloud

  • Check that the Agent is running and that the Service/Pod is healthy.
  • Check the list of endpoints that the Agent has discovered and is scraping. Port-forward to the Agent and use the HTTP API to list the discovered targets:
    • kubectl -n <namespace> port-forward grafana-agent-0 8001:80
    • curl localhost:8001/agent/api/v1/metrics/targets
  • If the targets list is empty, something is wrong:
    • Make sure that the Agent has permissions to view pods and services in the cluster (e.g. with a RoleBinding)
    • If scraping a single namespace (or just a few), make sure that the Agent is configured to do so:
        - role: pod
            - <NAMESPACE>