Configuration

You can make any configuration changes you need for your deepstream setup in the config.yml file. Depending on the installation sources, the configuration can be found in

  • in the folder conf in the Node.js module directory if you installed with npm, or
  • in the folder conf after unpacking the standalone server binary.

Macros

Before we start diving into config, lets look at the current macros we have in order to make our lives a little bit easier:

  • ${ENVIRONMENT_VARIABLE}

This is the usual replace environment variable into my config before running, useful in around 100% of deployment usecases.

  • file(relative/path)

This allows the server to binary to point to the file relative to your config file. This is pretty useful for global and binary installs.

  • fileLoad(relative/path)

This is more of a utility for us plugin developers, it automatically loads up the data and parses it (if json or yml). This means node developers can now just deal with writing everything as parsed data and leave loading / parsing and errors to deepstream startup.

General Configuration

In this section you can change general settings for each server in a cluster.

# general configuration with default values
serverName: UUID
showLogo: true
dependencyInitializationTimeout: 2000
exitOnFatalError: false

serverName

Every server in a cluster of servers needs a unique name. You can add your own or set it to UUID to let deepstream auto-generate a unique ID.

Default: UUID

When starting, a server can show the deepstream logo. This setting is best left enabled.

Default: true

logLevel

The logLevel to use across the application

Default: INFO

dependencyInitializationTimeout

Sets how long deepstream will wait for dependencies to initialize.

Default:2000

libDir

The directory where all the plugins reside, this is used in standalone binaries

Default: None, it assumes all plugins are installed via npm

exitOnFatalError

Exit if a fatal error occurs

Default: false

RPC Configuration

ackTimeout

Sets how long deepstream will wait for a RPC provider to acknowledge receipt of a request.

Default:1000

responseTimeout

Sets how long deepstream will wait for RPCs to complete.

Default:10000

Record Configuration

cacheRetrievalTimeout

Sets how long deepstream will wait when retrieving values from the cache.

Default:1000

storageRetrievalTimeout

Sets how long deepstream will wait when retrieving values from the database.

Default:2000

storageExclusionPrefixes

A list of prefixes that, when a record starts with one of the prefixes the records data won’t be stored in the db

Default: []

storageHotPathPrefixes

A list of prefixes that designate a record for direct writes to storage. When a correctly permissioned matching record is updated via setData(), it will be written directly to the cache and storage without a record transition. This can be up to twice as fast as updating a normal record using setData()

Default: []

Listening

shuffleProviders

Try finding a provider randomly rather than by the order they subscribed to.

Default: true

responseTimeout

The amount of time to wait for a provider to acknowledge or reject a listen request

Default: 500

rematchInterval

The amount of time before trying to reattempt finding matches for subscriptions. This is not a cheap operation so it’s recommended to raise keep this at minutes rather then second intervals if you are experiencing heavy loads

Default: 60000

matchCooldown

The amount of time a server will refuse to retry finding a subscriber after a previously failed attempt. This is used to avoid servers constantly trying to find a match without a cooldown period

Default: 10000

Connection Endpoint Configuration

Deepstream (v5 and later) runs all websocket / HTTP services in one server to which they hook to provide their own functionality. This means you can run all the services at the same time on the same port if you want (for example text, binary and JSON). The idea is that EVERYTHING (minus MQTT) runs on port 6020, which makes deployments much easier. 

httpServer:
  type: default
  options:
    # url path for http health-checks, GET requests to this path will return 200 if deepstream is alive
    healthCheckPath: /health-check
    # -- CORS --
    # if disabled, only requests with an 'Origin' header matching one specified under 'origins'
    # below will be permitted and the 'Access-Control-Allow-Credentials' response header will be
    # enabled
    allowAllOrigins: true
    # a list of allowed origins
    origins:
      - 'https://example.com'
    # Headers to copy over from websocket
    headers:
      - user-agent
    ssl:
      key: fileLoad(/path/to/sslKey)
      cert: fileLoad(/path/to/sslCert)
      ca: fileLoad(/path/to/caAuth)

OR

httpServer:
  type: uws
  options:
    # url path for http health-checks, GET requests to this path will return 200 if deepstream is alive
    healthCheckPath: /health-check
    # -- CORS --
    # if disabled, only requests with an 'Origin' header matching one specified under 'origins'
    # below will be permitted and the 'Access-Control-Allow-Credentials' response header will be
    # enabled
    allowAllOrigins: true
    # a list of allowed origins
    origins:
      - 'https://example.com'
    # Headers to copy over from websocket
    headers:
      - user-agent
    # Options required to create an ssl app
    ssl:
      key: file(ssl/key.pem)
      cert: file(ssl/cert.pem)
    ##  dhParams: ...
    ##  passphrase: ...

The server can be configured with custom connection endpoints. You can supply as many as you want, each as any individual list entry below the connectionEndpoints key:

# Connection Endpoint Configuration
connectionEndpoints:
  - type: ws-binary
    options:
      # url path websocket connections connect to
      urlPath: /deepstream
      # the amount of milliseconds between each ping/heartbeat message
      heartbeatInterval: 30000
      # the amount of milliseconds that writes to sockets are buffered
      outgoingBufferTimeout: 10
      # the maximum amount of bytes to buffer before flushing, stops the client from large enough packages
      # to block its responsiveness
      maxBufferByteSize: 100000

      # Security
      # amount of time a connection can remain open while not being logged in
      unauthenticatedClientTimeout: 180000
      # invalid login attempts before the connection is cut
      maxAuthAttempts: 3
      # maximum allowed size of an individual message in bytes
      maxMessageSize: 1048576

  - type: ws-text
    options:
      # url path websocket connections connect to
      urlPath: /deepstream-v3
      # the amount of milliseconds between each ping/heartbeat message
      heartbeatInterval: 30000
      # the amount of milliseconds that writes to sockets are buffered
      outgoingBufferTimeout: 10
      # the maximum amount of bytes to buffer before flushing, stops the client from large enough packages
      # to block its responsiveness
      maxBufferByteSize: 100000

      # Security
      # amount of time a connection can remain open while not being logged in
      unauthenticatedClientTimeout: 180000
      # invalid login attempts before the connection is cut
      maxAuthAttempts: 3
      # maximum allowed size of an individual message in bytes
      maxMessageSize: 1048576

  - type: ws-json
    options:
      # url path websocket connections connect to
      urlPath: /deepstream-json
      # the amount of milliseconds between each ping/heartbeat message
      heartbeatInterval: 30000
      # the amount of milliseconds that writes to sockets are buffered
      outgoingBufferTimeout: 10
      # the maximum amount of bytes to buffer before flushing, stops the client from large enough packages
      # to block its responsiveness
      maxBufferByteSize: 100000

      # Security
      # amount of time a connection can remain open while not being logged in
      unauthenticatedClientTimeout: 180000
      # invalid login attempts before the connection is cut
      maxAuthAttempts: 3
      # maximum allowed size of an individual message in bytes
      maxMessageSize: 1048576

  - type: http
    options:
      # allow 'authData' parameter in POST requests, if disabled only token and OPEN auth is
      # possible
      allowAuthData: true
      # enable the authentication endpoint for requesting tokens/userData.
      # note: a custom authentication handler is required for token generation
      enableAuthEndpoint: false
      # path for authentication requests
      authPath: /api/auth
      # path for POST requests
      postPath: /api
      # path for GET requests
      getPath: /api
      # maximum allowed size of an individual message in bytes
      maxMessageSize: 1024

  - type: mqtt
    options:
        # port for the mqtt server
        port: 1883
        # host for the mqtt server
        host: 0.0.0.0
        # timeout for idle devices
        idleTimeout: 60000

Websockets

The websocket endpoints share the following options:

urlPath

Sets which URL path Websocket connections should connect to.

Default: /deepstream

heartbeatInterval

The number of milliseconds between each ping/heartbeat message.

Default: 30000

unauthenticatedClientTimeout

The amount of time a connection can remain open while not being logged in.

Default: 180000

maxAuthAttempts

Invalid login attempts before the connection is cut.

Default: 3

logInvalidAuthData

Controls whether logs should contain the cleartext usernames and passwords of invalid login attempts.

Default: false

maxMessageSize

Sets the maximum message size allowed to be sent to the server (in bytes).

Default: 1048576

outgoingBufferTimeout

The amount of milliseconds that secondary writes to sockets are buffered. This means writes that are not realtime critical, which currently are either ACKs or non critical ERROR messages.

Default: 0

Custom Plugin Configuration

You can extend deepstream with plugins for connectors to other services, these are for logging, storage engines, caching layers, and message systems. To enable a plugin, uncomment the relevant category key underneath the plugins key. Each plugin type has a path or name, and a set of options.

# Plugin example using redis
plugins:
  myCustomPlugin:
    path: ./my-custom-plugin
    options:
       host: localhost
       port: 5672

path

Set a path to a JavaScript file, node module or a folder with an index.js file which exports a constructor.

name

If you are using any of the official deepstream connectors, add the name of what the plugin connects to here, for example redis.

Note: You can set path or name, but not both.

options

Under this key, add sub key/value pairs to set the configuration options that are passed to the plugin. Each plugin should mention what configuration options you can set.

Logger

deepstream uses by default a logger which prints out to stdout (errors and warnings to stderr). You can set these options for the default logger by using the same configuration style for the plugins:

logger:
  name: default
  options:
    colors: true
    logLevel: INFO

colors

Sets whether the server’s logs should output in color. This will look great in a console, but will leave color markers in log files if you redirect the output into a file.

Default: true

logLevel

Sets at what level and above the server should log messages. Valid levels are DEBUG, INFO, WARN, ERROR, and OFF.

Default: INFO

Authentication

In this section you can configure the authentication types the server uses.

You set the authentication type as a subkey in the auth key. The available authentication options are none, file, storage and http, each of them having different options which are described in the tutorials on Auth None, file-based authentication, storage-based authentication, HTTP authentication, and JWT authentication respectively.

You can set multiple authentication types simultaneously and the incoming connection will be validated against each of them until one succeeds or all fail. Authentication strategies will be queried in the same order they are declared on the configuration file. 

#Authentication
auth:
  - type: none

Default: none

Permissioning

In this section you can configure the permissioning. The key used for this section is permission and you can create your own custom permission handler or use a configuration file. To use the former method, select the option type: none and override the permissionHandler with the aid of the NodeJS server API. To use the latter method, set type: config and modify the option values below.

# Permissioning example with default values for config-based permissioning
permission:
  type: config
  options:
    path: ./permissions.yml
    maxRuleIterations: 3
    cacheEvacuationInterval: 60000

path

Set the path to the file that declares permissions. This option is mandatory with type: config. The file can be in JSON, JavaScript, or YAML format. By default, deepstream ships with a permissions.yml permitting every action.

maxRuleIterations

The deepstream permissions model allows for some complex nested actions and queries. To prevent a performance hit you can limit the nesting level with this option.

Default: 3

cacheEvacuationInterval

The results of permission checks are cached to improve performance. Use this setting to change the time interval (in milliseconds) that the cache is regenerated.

Default: 60000

Storage and Cache

Storage and Cache plugins can be configured as follows, please look at current connectors for in depth tutorials how to configure them.

cache:
  name: redis
  options:
    host: localhost
    port: 6379

storage:
  name: postgres
  options:
    user: postgres
    database: postgres
    password: mysecretpassword
    host: 'localhost'
    port: 5432
- Configuration
- Command Line Interface
- Node API
- User File
- Valve Permissions