Remote ad-hoc queries with OSQuery

OSQuery allows you to easily ask questions about your Linux, Windows, and macOS infrastructure.

OSQuery gives you the ability to query and log things like running processes, logged in users, password changes, USB devices, firewall exceptions, listening ports, and more. You can perform ad-hoc queries or schedule them

Ad-hoc queries

OSQueryi is the interactive query console of OSQuery. It gives you a SQL interface to try out new queries and explore your operating system using SQL language and dozens of useful tables built-in.

Working remotely

To work with OSQueryi in remote way a server that supports 3 REST APIs has to be built. Then you have to set up each OSQuery Remote Instance (OSQuery Nodes) to call these APIs.

This post includes the technical description of this APIs and what they should do.

Authentication

OSQuery remote supports 2 different kind of authentications.

• Simple shared secret enrollment
• TLS client-auth enrollment (not explain here)

Simple shared secret enrollment

To use this authentication method the following is needed:

OSQuery Nodes

• A CA's PEM-encoded certificate.
• The certificate secret password.

Server

• A certificate signed by the CA's certificate.
• the certificate private key
• The certificate secret password.

enroll API

This POST REST API is invoked by an OSQuery Node in order to announce to the server that it is running and being authenticated.

/enroll
 - Method: POST
 - Parameters: 
    - "enroll_secret": secret.
    - "host_identifier": Determined by the --host_identifier flag in the OSQuery remote instance. This is the HostUUID
 - Response:
    {
      "node_key": random generated ID
      "node_invalid": false
    }

Parameters example

{
  "enroll_secret": "password123",
  "host_identifier": "58f64939be000074588fb2a5" 
}

Response example

{
  "node_key": "00fba564958588fbe",
  "node_invalid": false
}

Procedure

When a POST /enroll is received the Server should do the follows:

1. Check that the enroll_secret matches the enroll secret in the server (This enroll secret can be store in a config file).
2. Register the hostUUID received in the host_identifier parameter in the Nodes list along with a new generated UUID as node key. If there is a row in the Node list for the hostUUID the current node key is replaced by a new one.
3. Respond the request using the the just generated node key as value for the "node_key" property.

distributed_read API

This POST REST API is invoked by an OSQuery Node to get a list of queries to executed.

/distributed_read
 - Method: POST
 - Parameters: 
    - "node_key": node key.
 - Response:
    {
      "queries": {
        "id1": "select * from osquery_info;",
        "id2": "select * from osquery_schedule;",
        "id3": "select * from does_not_exist;"
      },
      "node_invalid": false // Optional, return true to indicate re-enrollment. 
    }

Parameters example

{
  "node_key": "00fba564958588fbe"
}

Response example

{
  "queries": {
    "id1": "select * from osquery_info;",
    "id2": "select * from processes;",
    "id3": "select * from does_not_exist;"
  },
  "node_invalid": false
}

Procedure

When a POST /distributed_read is received the Server should do the follows:

1. Check if the 'node_key' is present in the Nodes List. If not return a response with 'node_invalid': true for indicating that new enrollment is needed.
2. Check the Queries List and take all the queries with the node key specified in the node_key parameter.
3. Generate the response with all the queries and their queryIDs associated.
4. Respond the request.

distributed_write API

This POST REST API is invoked by an OSQuery remote instance to return the queries' responses.

/distributed_write
 - Method: POST
 - Parameters: 
    {
      "node_key": node key,
      "queries": {
        "id1": [
          {"column1": "value1", "column2": "value2"},
          {"column1": "value1", "column2": "value2"}
        ],
        "id2": [
          {"column1": "value1", "column2": "value2"},
          {"column1": "value1", "column2": "value2"}
        ],
        "id3": []
      },
      "statuses": {
        "id1": 0,
        "id2": 0,
        "id3": 2,
      }
    }
 - Response:
    {
      "node_invalid": false // Optional, return true to indicate re-enrollment.
    }

Parameters example

{
  "node_key": "00fba564958588fbe",
  "queries": {
    "id1": [
      {"pid": "32421", "uuid": "32773f6b-9cfd-443b-9480-cbe46017fab8", "build_plattform": "Ubuntu"}
    ],
    "id2": [
      {"pid": "1254", "name": "systemd"},
      {"pid": "4575", "name": "watchdog"}
    ],
    "id3": []
  },
  "statuses": {
    "id1": 0,
    "id2": 0,
    "id3": 2,
  }
}

Response example

{
  "node_invalid": false
}

Procedure

When a POST /distributed_write is received the Server should do the follows:

1. Check if the 'node_key' is present in the Nodes List. If not return a response with 'node_invalid': true for indicating that new enrollment is needed.
2. Save the parameters data in the Responses List.
3. Response the request with an empty response.