History

Running Locally

Summary

This doc explains how to invoke bots running locally that hit actual queues in a deployed RStreams Bus as well as how to mock out a queue and invoke a bot with that mock data. It also explains how to set breakpoints in your code.

Prerequisites

  • This assumes you are using Visual Studio Code as your IDE, however everything should be possible in your favorite IDE such as IntelliJ or whatever it may be
  • If you want to follow along, you should go through the Getting Started Guide

Things you need to know

Microservice Name / Service Name

Your project’s microservice name, also just called the Service Name, is used in many places. It’s located in <project-root>/serverless.yml near the top in the service attribute. Here, the project’s service name is rstreams-example.

service: rstreams-example

Bot

A bot is logical wrapper around code that executes to interact with an RStreams queue. Bots are found in the {project_dir}/bots/{bot_dir}. Each bot has its own serverless.yml file. Here’s an example from the weather-loader bot of the RStreams Flow Example project.

Example 1 code
 1weather-loader:
 2  name: ${self:service}-weather-loader
 3  handler: bots/weather-loader/index.handler
 4  description: Bot generated by serverless-leo
 5  memorySize: 256
 6  timeout: 300
 7  role: BotRole
 8  events:
 9    - leo:
10        destination: rstreams-example.weather
11        cron: 0 */1 * * * * 
12        extra_data: 123

Bot ID

Needed when invoking a bot.

Bots are referenced in config via their ID. The ID is the first attribute in the bot’s serverless.yml file that all other config is a child of. In the above example it’s on Line 1 and is named weather-loader.

Stage

Needed when invoking a bot.

Each RStreams Flow bot executes in the context of an environment, called a stage, such as dev/test/staging/prod. The RStreams Flow example template project expects that your stage, the environment, is dev and that a file named .env.dev exists that contains the name of the AWS secrets manager secret that contains the RStreams bus instance config to connect to.

RStreams Config

Each running RStreams SDK needs to know what resources to access that comprise the RStreams Bus instance you intend for it to read and write from/to. This is called the RStreams Config. The project has been setup such that the dev environment is meant for running locally. You should open up the .env.dev file and change the RSTREAMS_CONFIG_SECRET value to be the name of the AWS Secrets Manager secret that was created when your RStreams Bus instance was installed. It will be named like this: rstreams-{busStackName} where {busStackName} is the name of your bus stack. So, if you have a bus stack named named PlaygroundBus-Bus-1JX7JSIIUQRAO your secret would be named rstreams-PlaygroundBus-Bus-1JX7JSIIUQRAO.
Jump here for lots more on this if you care.

Workflow

A flag that may provided to run a chained set of bots locally with the serverless invoke-bot command (see below). The first bot in the chain is the one that is specified to be invoked on the command line. It can work in one of two modes. Either the first bot pulls actual data using the -actualSource flag and then writes to a queue that is really a file and then the next bot reads from the file instead of a real queue and so on. Or, without that flag the very first bot will read from a queue that is really a hard-coded mock file you created.

Mock

A flag that may be provided when running bots locally to have a bot read from a file of mock data instead of an actual queue and optionally write to a file instead of writing to an actual queue.

Mock data is located in the RStreams Flow project’s <project-root>/.mock-data directory. If you want to provide a mock file to be read by the SDK when running locally instead of hitting an actual queue, create a file named {queueName}.jsonl and put each JSON event object on a separate line, following the JSON Lines convention.

Be aware that files intended to represent queues need to be placed under the subdirectory queue, i.e. .mock-data/queue.

So, if your code reads from a queue named rstreams-example.weather then you would create this file and run the invoke command with the mock flag: <project-root>/.mock-data/queue/rstreams-example.weather.jsonl.

While mock queue files are useful as surrogate data sources, rsdk will always read ALL events from the beginning of the JSONL file irrespective of the desired starting checkpoint position as can be specified in the ReadOptions configuration object.

If you are running locally with the workflow flag on, then the SDK will write to an output file named for the queue instead of writing to an actual RStreams Bus queue. The output files will be in the <project-root>/.mock-data/output directory. They will be named queueName.jsonl. So, if your code writes to a queue named rstreams-example.weather-transformed then the SDK would create this file and use it as the queue of data to send to any subsequent bot that reads from that queue: <project-root>/.mock-data/output-data/rstreams-example.weather-transformed.jsonl.

Invoke Bots Locally

In all of the following commands, you may either use serverless invoke-bot <remaining args> or if more convenient npm test {functionName} -- {remainingArgs} to invoke bots locally.

RStreams Flow supports the following use cases when running locally.

  • Set a breakpoint in any bot invoked
    Use your IDE’s method for debugging a Node process and set a breakpoint in your bot and it will stop on your breakpoint. In Visual Studio Code, the easiest way to debug is to simply issue the command to invoke the bot in a Javascript debug terminal and then you don’t have to do anything else and your breakpoints will work.

VSCode JS Debug Terminal