Splunk logs

samples/oci-logs-splunk-hec/README.md

@@ -0,0 +1,96 @@
+## Monitor Oracle Cloud Infrastructure Logs with Splunk
+
+Forward logs from Oracle Cloud Infrastructure Logging service to Splunk via
+HTTP Event Connector
+
+
+## Prerequisites
+
+Before you deploy this sample function, make sure you have run steps A, B
+and C of the [Oracle Functions Quick Start Guide for Cloud Shell](https://www.oracle.com/webfolder/technetwork/tutorials/infographics/oci_functions_cloudshell_quickview/functions_quickview_top/functions_quickview/index.html)
+* A - Set up your tenancy
+* B - Create application
+* C - Set up your Cloud Shell dev environment
+
+
+## List Applications
+
+Assuming you have successfully completed the prerequisites, you should see your
+application in the list of applications.
+
+```
+fn ls apps
+```
+
+
+## Configure your Function
+
+In order to send logs to Splunk you'll need to define two environment variables:
+* `SPLUNK_HEC_ENDPOINT` - the HTTP/HTTPS REST endpoint for the HEC service
+* `SPLUNK_HEC_TOKEN` - the Token used to authenticate
+
+
+### Splunk Enterprise / Splunk Cloud
+
+If you haven't already you'll need to set up the HTTP Event Collector service
+on your Splunk Instance. Instructions on configuring and using the HEC are
+available at https://docs.splunk.com/Documentation/Splunk/9.0.1/Data/UsetheHTTPEventCollector.
+
+
+## Deploy the function
+
+In Cloud Shell, run the `fn deploy` command to build *this* function and its dependencies as a Docker image,
+push the image to the specified Docker registry, and deploy *this* function to Oracle Functions
+in the application created earlier:
+
+![user input icon](./images/userinput.png)
+```
+fn -v deploy --app <app-name>
+```
+e.g.,
+```
+fn -v deploy --app myapp
+```
+
+
+## Configure the logs you want to capture
+
+1. From the [OCI Console](https://cloud.oracle.com) navigation menu, select **Logging**, and then select **Log Groups**.
+
+2. Click Create Log Group, select your compartment, add a Name and Description
+
+3. Select Logs in the left menu, click Enable Service Log, select your compartment, select Log Category on Service and fill the rest of the fields appropriately.
+
+
+## Create a Service Connector for reading logs from Logging and send to Functions
+
+1. From the navigation menu, select **Logging**, and then select **Service Connectors**.
+
+2. Click Create Connector, add a Name, Description, select the compartment, select the Source as **Logging** and Target as **Functions**.
+
+3. On Configure Source connection, select the compartment, select the Log Group created earlier.
+
+4. On Configure Target connection, select the compartment and select the Function. If prompted to create a policy for writing to functions, click Create.
+
+
+## Monitoring Functions and Service Connector
+
+Make sure you configure basic observability for your function and connector using metrics, alarms and email alerts:
+* [Basic Guidance for Monitoring your Functions](../basic-observability/functions.md)
+* [Basic Guidance for Monitoring your Service Connector](../basic-observability/service-connector-hub.md)
+
+---
+## Function Environment
+
+Here are the supported Function parameters:
+
+| Environment Variable        | Default           | Purpose  |
+| ------------- |:-------------:| :----- |
+| SPLUNK_HEC_ENDPOINT      | not-configured | REST API endpoint for reaching Splunk HEC ([see docs](https://docs.splunk.com/Documentation/Splunk/9.2.0/Data/UsetheHTTPEventCollector#Configure_HTTP_Event_Collector_on_Splunk_Cloud_Platform))|
+| SPLUNK_HEC_TOKEN      	| not-configured      |   HEC authentication token obtained from Splunk HEC configuration |
+| LOGGING_LEVEL | INFO     |    Controls function logging outputs.  Choices: INFO, WARN, CRITICAL, ERROR, DEBUG |
+
+---
+## License
+Copyright (c) 2024, Oracle and/or its affiliates. All rights reserved.
+Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl.

samples/oci-logs-splunk-hec/func.py

@@ -0,0 +1,55 @@
+#
+# oci-logs-splunk-hec version 0.1.
+#
+# Copyright (c) 2024 Splunk, Inc.  All rights reserved.
+# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl.
+#
+
+import io
+import os
+import json
+import requests
+import logging
+from fdk import response
+
+
+"""
+This Function receives the logging json and forwards to the Splunk HTTP Event
+Connector (HEC) for ingesting logs. Logging Format Overview
+https://docs.cloud.oracle.com/en-us/iaas/Content/Logging/Reference/top_level_logging_format.htm#top_level_logging_format
+If this Function is invoked with more than one log the function go over each log and invokes the HEC endpoint for ingesting one by one.
+"""
+
+def handler(ctx, data: io.BytesIO=None):
+    try:
+        logs = json.loads(data.getvalue())
+
+        # no need to have verbose logs from log forwarder
+        urllib3_logger = logging.getLogger('urllib3')
+        urllib3_logger.setLevel(logging.CRITICAL)
+
+        # Splunk HEC endpoint URL and token to call the REST interface. These values are defined in func.yaml
+        hec_endpoint = os.environ['SPLUNK_HEC_ENDPOINT']
+        hec_token = os.environ['SPLUNK_HEC_TOKEN']
+        headers = {'Content-type': 'application/json', 'Authorization': str("Splunk " + str(hec_token))}
+
+        # loop over each log and reformat for HEC.
+        concat_body = ""
+        for item in logs:
+            event = item['oracle']
+            event.update(item['data'])
+            body = {}
+            body['event'] = event
+            body['source'] = 'oci:' + item['source']
+            body['sourcetype'] = '_json'
+            concat_body = concat_body + str(json.dumps(body))
+
+        # Post the message to HEC payload.
+        if len(concat_body) > 0:
+            x = requests.post(hec_endpoint, data = concat_body, headers=headers)
+            if x.status_code != 200:
+                logging.getLogger().info(x.text)
+
+    except (Exception, ValueError) as ex:
+        logging.getLogger().info(str(ex))
+    return

samples/oci-logs-splunk-hec/func.yaml

@@ -0,0 +1,6 @@
+schema_version: 20180708
+name: oci-logs-to-splunk-hec
+version: 0.0.48
+runtime: python3.9
+entrypoint: /python/bin/fdk /function/func.py handler
+memory: 256

samples/oci-logs-splunk-hec/requirements.txt

@@ -0,0 +1,3 @@
+fdk
+requests
+oci

samples/oci-monitoring-metrics-to-splunk-observability-python/LICENSE.txt

@@ -0,0 +1,28 @@
+Copyright (c) 2024, Oracle and/or its affiliates.  All rights reserved.
+
+The Universal Permissive License (UPL), Version 1.0
+
+Subject to the condition set forth below, permission is hereby granted to any person obtaining a copy of this
+software, associated documentation and/or data (collectively the "Software"), free of charge and under any
+and all copyright rights in the Software, and any and all patent rights owned or freely licensable by each
+licensor hereunder covering either (i) the unmodified Software as contributed to or provided by such licensor,
+or (ii) the Larger Works (as defined below), to deal in both
+
+(a) the Software, and
+
+(b) any piece of software and/or hardware listed in the lrgrwrks.txt file if one is included with the
+Software (each a “Larger Work” to which the Software is contributed by such licensors), without restriction,
+including without limitation the rights to copy, create derivative works of, display, perform, and
+distribute the Software and make, use, sell, offer for sale, import, export, have made, and have sold
+the Software and the Larger Work(s), and to sublicense the foregoing rights on either these or other terms.
+
+This license is subject to the following condition:
+
+The above copyright notice and either this complete permission notice or at a minimum a reference to the
+UPL must be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT
+LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE
+OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.