Collect what you need, only when you need it
Sidekick is an open source live application debugger that comes with many built-in features to make data collection both easier and more efficient. Collecting snapshots and generating logs from running applications are powerful for learning how your code works as long as you collect the data you need.
Logging everything is not a smart move as you can find yourself searching in a log pile full of mostly unnecessary logs in a very short time. Similar case applies for the snapshots & traces. That is why we have included conditional data generation in Sidekick. Combined with hit and time limits, they allow you to collect only what you need, leaving a graveyard of useless data behind.
In addition to our out-of-the-box conditional limitation features, Sidekick’s API’s and clients let you to bring your own rules which leads to a relatively new approach to observability, a rule based one. With this approach we will be able to put tracepoints when rules met and start collecting data automatically based on rule engine’s output.
Rule Engines
There are plenty of different rule engines & workflow automation tools that we can use for such a scenario. A few can be listed as:
[The Universal Process Orchestrator | Camunda
Camunda's process automation platform allows developers to design, automate and improve processes. Start your free…camunda.com](https://camunda.com/ "camunda.com")
[Drools - Business Rules Management System (Java™, Open Source)
Drools introduces the Business Logic integration Platform which provides a unified and integrated platform for Rules…drools.org](https://www.drools.org/ "drools.org")
[json-rules-engine
A rules engine expressed in JSON json-rules-engine is a powerful, lightweight rules engine. Rules are composed of…npmjs.com](https://www.npmjs.com/package/json-rules-engine "npmjs.com/package/json-rules-engine")
[node-rules
Node-rules is a light weight forward chaining Rule Engine, written in JavaScript for both browser and node.js…npmjs.com](https://www.npmjs.com/package/node-rules "npmjs.com/package/node-rules")
We will be using Camunda for this example for broader use cases as Camunda is more than a rule engine it is a universal process orchestrator. For a usage example of Camunda you can check out:
[Use Camunda as an easy-to-use REST-based orchestration and workflow engine (without touching Java)
I talk a lot about microservice architectures with “non-Java-folks”, let it be a C# developer, a Node.JS/JavaScript…blog.bernd-ruecker.com](https://blog.bernd-ruecker.com/use-camunda-without-touching-java-and-get-an-easy-to-use-rest-based-orchestration-and-workflow-7bdf25ac198e "blog.bernd-ruecker.com/use-camunda-without-..")
Using Sidekick with Camunda
Camunda provides external task clients that allow developers to implement tasks to run when a workflow hits to the related step. We will be using Node.js client but cases below can also be achieved with different clients of Camunda and Sidekick’s upcoming Python & Java clients.
[camunda-external-task-client-js
Implement your BPMN Service Task in NodeJS. NodeJS >= v10 is required npm install -s camunda-external-task-client-js…npmjs.com](https://www.npmjs.com/package/camunda-external-task-client-js "npmjs.com/package/camunda-external-task-cli..")
Example task method:
// Subscribe to the topic: 'topicName'
client.subscribe("topicName", async function({ task, taskService }) {
// Put your business logic
// Complete the task
await taskService.complete(task);
});
[@runsidekick/sidekick-client
Sidekick is a production debugging and on-demand logging tool where you can debug your running applications while they…npmjs.com](https://www.npmjs.com/package/@runsidekick/sidekick-client "npmjs.com/package/@runsidekick/sidekick-cli..")
Task method with Sidekick Put tracepoint event:
// Subscribe to the topic: 'topicName'
client.subscribe("topicName", async function({ task, taskService }) {
const params= {
applicationFilters: [
{
name: "Demo application",
version: "v1.0",
stage: "prod"
}
],
fileName: "gitlab.com/repos...",
lineNo: 23,
expireSecs: -1,
expireCount: -1,
enableTracing: true,
persist: true
}
apiClient.putTracepoints(params);
await taskService.complete(task);
});
Example use cases
Case 1 — Activating Tracepoints based on error scenarios
Camunda’s DMN engine provides us ability to define rules in an excel like interface.
Sidekick DMN
As you can observe at the DMN above we can define where we put tracepoints when a certain type of error occurs.
Triggering workflows with error types makes a great use case as it enables us to automatically start collecting extra information based on different cases. Enabling different tracepoints at once is especially useful for spotting state changes.
Note: In addition to this use case Sidekick Node.js agent has ability to automatically capture error stacks, without any tracepoint. check out: https://www.npmjs.com/package/@runsidekick/sidekick-agent-nodejs
Example Sidekick BPMN
Case 2 —Putting Tracepoints on workflow errors
Camunda comes with an error handling mechanism that lets you run actions in cases of errors. In addition to Sidekick client, we can also attach Sidekick agents to task services to observe their state changes and debug them effectively.
Case 3 — Activating Tracepoints on certain events
Last but not least, Sidekick Actions bring more than debugging to the table. They are also great tools to understand how the system works. Especially for the cases like understanding why and how edge cases occur or understanding when does a certain step is reached.
We have briefly explained how Sidekick can be used with Camunda to achieve rule based observability, for better understanding systems, having a smoother debug experience and lowering monitoring costs with conditional data collection. The 3 cases we have explained are just the tip of the iceberg and there is a lot more to discover here.
Please let us know your invaluable ideas because they will guide us on our path to discovering the future of application observability. You can get in touch with us through Twitter, and join our Discord community.