{"componentChunkName":"component---src-components-mark-down-block-js","path":"/docs/data-routing/","result":{"data":{"markdownRemark":{"html":"

Data Routing

\n

AppScope gives you multiple ways to route data. The basic operations are:

\n\n

For each of these operations, the CLI has command-line options, the config file has settings, and the AppScope library has environment variables.

\n

If you plan to use the config file, do take time to read it all the way through - then this page will make more sense!

\n

If you want to run Cribl Edge and AppScope in a container, and/or scope apps that are running in containers, see these instructions.

\n

Routing to Cribl Edge

\n

In a single operation, you can route both events and metrics to Cribl Edge.

\n

Routing to Edge with the CLI

\n

Use the -c or --cribldest option. You need this option because, by default, the CLI routes data to the local filesystem. For example:

\n
scope run -c edge -- curl https://wttr.in/94105\n
\n

This usage works with scope run, scope attach, scope watch, scope k8s, and scope service. It does not work with scope metrics, where the -c option stands for --cols, telling the CLI to output data in columns instead of rows.

\n

Routing to Edge with the AppScope Library

\n

Use the SCOPE_CRIBL environment variable to define a unix socket connection to Cribl Edge. For example:

\n
LD_PRELOAD=./libscope.so SCOPE_CRIBL=edge ls -al\n
\n

Routing to Edge with the Config File

\n

Complete these steps, paying particular attention to the sub-elements of cribl > transport, which is where you specify routing:

\n\n

Routing to Cribl Stream in the Cloud

\n

In a single operation, you can route both events and metrics to Cribl Stream in the Cloud, or Cribl.Cloud for short.

\n

Routing to Cribl.Cloud with the CLI

\n

Use the -c or --cribldest option. You need this option because, by default, the CLI routes data to the local filesystem. For example:

\n
scope run -c tls://127.0.0.1:10090 -- curl https://wttr.in/94105\n
\n

This usage works with scope run, scope attach, scope watch, scope k8s, and scope service. It does not work with scope metrics, where the -c option stands for --cols, telling the CLI to output data in columns instead of rows.

\n

Routing to Cribl.Cloud with the AppScope Library

\n

Use the SCOPE_CRIBL_CLOUD environment variable to define a TLS-encrypted connection to Cribl.Cloud. Specify a transport type, a host name or IPv4 address, and a port number, for example:

\n
LD_PRELOAD=./libscope.so SCOPE_CRIBL_CLOUD=tcp://in.main-default-<organization>.cribl.cloud:10090 ls -al\n
\n

As a convenience, when you set SCOPE_CRIBL_CLOUD, AppScope automatically overrides the defaults of three other environment variables, setting them to the values that Cribl.Cloud via TLS requires. That produces these (and a few other) settings:

\n\n

If you prefer an unencrypted connection to Cribl.Cloud, use SCOPE_CRIBL, as described here.

\n

Routing to Cribl.Cloud with the Config File

\n

Complete these steps, paying particular attention to the sub-elements of cribl > transport, which is where you specify routing:

\n\n

Routing Events

\n

You can route events independently of metrics.

\n

Routing Events with the CLI

\n

Use the -e or --eventdest option. For example:

\n
scope run -e tcp://localhost:9109 -- curl https://wttr.in/94105\n
\n

The above example sends events in ndjson, the CLI's default output format. To send events in StatsD, you would include --metricformat statsd in the command.

\n

Routing Events with the AppScope Library

\n

Use the SCOPE_EVENT_DEST environment variable, and set the SCOPE_CRIBL_ENABLE to false.

\n

For example:

\n
LD_PRELOAD=./libscope.so SCOPE_CRIBL_ENABLE=false SCOPE_EVENT_DEST=tcp://localhost:9109 curl https://wttr.in/94105\n
\n

The above example sends events in StatsD, the AppScope library's default output format.

\n

Routing Events with the Config File

\n

Complete these steps, paying particular attention to the sub-elements of event > transport, which is where you specify routing:

\n\n

Routing Metrics

\n

You can route metrics independently of events.

\n

Routing Metrics with the CLI

\n

Use the -m or --metricdest option. For example:

\n
scope run -m udp://localhost:8125 -- curl https://wttr.in/94105\n
\n

The above example sends events in ndjson, the CLI's default output format. To send events in StatsD, you would include --metricformat statsd in the command.

\n

Routing Events with the AppScope Library

\n

Use the SCOPE_METRIC_DEST environment variable, and set the SCOPE_CRIBL_ENABLE to false.

\n

For example:

\n
LD_PRELOAD=./libscope.so SCOPE_CRIBL_ENABLE=false SCOPE_METRIC_DEST=udp://localhost:8125 curl https://wttr.in/94105\n
\n

The above example sends events in StatsD, the AppScope library's default output format.

\n

This sends metrics in StatsD format. Adding SCOPE_METRIC_FORMAT=ndjson would change the format to ndjson.

\n

Routing Metrics with the Config File

\n

Complete these steps, paying particular attention to the sub-elements of metric > transport, which is where you specify routing:

\n\n

Running AppScope and Cribl Edge in a Container

\n

This section describes one of many possible scenarios involving AppScope, Cribl Edge, and containers. If you are interested in doing something different let us know via the #appscope channel of Cribl's Community Slack.

\n

You can start Cribl Edge and AppScope together in a container, then use Cribl Edge's AppScope Source to \"drive\" AppScope. You'll decide what apps to scope, and work with the resulting events and metrics in Cribl Edge.

\n

To do this, you can use the docker run command, choosing options based on considerations including whether to mount the host filesystem in read-only or read-write mode. By default, the -v or --volume mounts in read-write mode, for example -v /:/hostfs. For read-only mode, add :ro, for example -v /:/hostfs:ro.

\n

In the examples below, we use /hostfs to specify the root filesystem mount point; alternatively, you could use a path defined by the environment variable CRIBL_EDGE_FS_ROOT.

\n

The examples progress from most to least \"locked down.\"

\n

Example 1: Mount the Host Filesystem Read-only

\n

The command below mounts the overall host filesystem read-only. It then mounts the scope start command's three mount points in read-write mode, which is required for scope start to work, even when the overall filesystem is read-only.

\n
docker run -d -e CRIBL_EDGE=1 -p 9420:9420 -v /var/run/appscope:/var/run/appscope -v /var/run/docker.sock:/var/run/docker.sock -v /:/hostfs:ro -v /etc/cron.d/:/hostfs/etc/cron.d/ -v /tmp/:/hostfs/tmp/ -v /usr/lib/:/hostfs/usr/lib/ --restart unless-stopped --name cribl-edge cribl/cribl:4.0.4\n
\n

Example 2: Mount the Host Filesystem Read-Only With the --privileged flag

\n

The command below mounts the overall host filesystem read-only, but by adding the --privileged flag, gives the container access to processes running outside containers on the host. With this usage, there's no need to specify the scope start mount points.

\n
docker run -d -e CRIBL_EDGE=1 -p 9420:9420 -v /var/run/appscope:/var/run/appscope -v /var/run/docker.sock:/var/run/docker.sock -v /:/hostfs:ro  --privileged --restart unless-stopped --name cribl-edge cribl/cribl:4.0.4\n
\n

The --privileged flag should be used with care, because it bestows Linux capabilities – including ptrace, the ability to read the /proc filesystem, and more – on whatever apps run in the container.

\n

Example 3: Mount the Host Filesystem Read-Write

\n

The command below mounts the overall host filesystem read-write. With this usage, there's no need to specify the scope start mount points or to add the --privileged flag.

\n
docker run -d -e CRIBL_EDGE=1 -p 9420:9420 -v /var/run/appscope:/var/run/appscope -v /var/run/docker.sock:/var/run/docker.sock -v /:/hostfs  --restart unless-stopped --name cribl-edge cribl/cribl:4.0.4\n
","frontmatter":{"title":"Data Routing"}}},"pageContext":{"slug":"/data-routing/","id":"da6a6d78-653c-5e3e-b6d3-76de2ccc5875"}},"staticQueryHashes":["2787008656","306729481","625673991","640077088"]}