By the end of this guide you should know how to manage your Watcher installed on VPS or dedicated server. The guide is useful for enterprise and individual clients who want to know how to manage their Watcher and receive notifications about its status, byzantine events, and alarms.
You should use this guide if you need to accomplish one of the following goals:
- Monitor the uptime of your Watcher's server.
- Automate regular Watcher status check.
- Receive important alarms from the Watcher.
- Know how to work with Watcher containers.
- Active Watcher running on the VPS.
- Basic knowledge of REST APIs.
- Basic knowledge of Docker tooling.
1. Working with Docker Containers
One of the ways to run a Watcher is using Docker containers. If you're new to Docker, these are the commands that should help you to solve the most common problems:
Start/Stop/Restart/Update Docker Containers
docker-compose -f docker-compose-watcher.yml start
Stop Only Watcher Containers
docker-compose -f docker-compose-watcher.yml stop
Stop All Docker Containers
docker stop $(docker ps -aq)
Restart Only Watcher Containers
docker-compose -f docker-compose-watcher.yml restart
Restart All Docker Containers
docker restart $(docker ps -aq)
docker pull omisego/watcher:latest && docker pull omisego/watcher_info:latest
2. Server Uptime Monitoring
Watcher monitoring is one of the most important processes you need to set up if you're planning to work with the OMG Network over the long term. Receiving alarms and status changes can help to provide more accurate data for the services and applications you're building using the OMG Network, understand how reliable is your VPS provider, respond faster and more effectively to potential byzantine events on the network.
2.1 Status Cake
To monitor the status of your server, you can use Pingdom, Status Cake or similar software that provides website/VPS monitoring functionality. If you're using Status Cake, go to the
MONITORING > Uptime Monitoring section, and select
New Uptime Test. Choose
TCP test type and fill the required values, including your server's IP address, port, test name, and contact group that contains details about users who will receive notifications.
Status Cake provides multiple ways to receive notifications, such as Datadog, Slack, Discord, Telegram, OpsGenie, etc. You can set your preferred method in the
ALERTING > Integrations section.
3. Status Monitoring
Running the OMG Network's Watcher implies that you're running both
Watcher Info services that will serve separate API endpoints and provide different access level to the network. Most of the time you'll be using
Watcher Info service but you should monitor both.
It is possible for
Watcher Info to become inactive, even if its server is up. This can happen due to sudden configuration changes, internet, or hardware issues with your VPS provider.
There are a few ways to monitor the status of these services. The simplest way is to send corresponding API requests on a certain schedule. This guide demonstrates Datadog as one of the tools you can use to monitor various parts of your infrastructure.
3.1 Watcher Status
To monitor Watcher status, use the following steps:
Create Datadog account.
Choose Ubuntu as OS for Datadog Agent.
Log in to the Watcher's server from your terminal:
ssh $USER@$REMOTE_SERVER -p $PORT
$USER- the name of the user with root privileges used to log into the remote server. Default: root.
$REMOTE_SERVER- an ip address of your remote server.
$PORT- a port used to connect to the server. Default: 22.
- Paste the command from
New installationto your server's terminal and press Enter:
The process may take a few minutes. After a successful installation on the server, your Datadog account will be redirected to the dashboard.
- Go to
UX Monitoring > Synthetic Tests. Click
Get Startedand select the
New API Test.
- Fill in the following values and press
- URL method:
- URL path:
- Name: some indication about this specific endpoint, e.g.
- Environment: environment name/identifier
- Locations: locations you want your API to be tested from
$REMOTE_SERVER- an IP address of your remote server.
- Specify the test frequency.
Test frequency defines how often the API should be called. A general suggestion is 5-15 minutes.
- Define assertions.
Assertions are conditions required for a test to pass. Our main condition is for a Watcher to return
success status. To do that, select the
success field in the
BODY result section. This will add the corresponding assertion as follows:
- Define alert conditions.
Alert conditions help to set the number of failed tests, locations, retries needed to notify the team.
- Set up notifications.
You can notify your team via email, Slack, Jira, Webhook, or dozens of other tools. Check Datadog Notifications for more info.
- Monitor the result.
You can monitor alerts, as well the status of your APIs in the
UX Monitoring > Synthetic Tests sections.
3.2 Watcher Info Status
Setting up the
Watcher Info status check has the same approach as described above. When creating a new synthetic test, replace the Define request as follows:
Then, define assertions with
success:true and follow the rest of the steps.