# Finex metrics

Metrics are exposed on port 4242 by default. To disable the metrics in you can set the following entry in config.yaml:

metrics:
  enabled: false

Visit localhost:9090/graph

# Some PromQuery examples

Average time of Persister SQL batch execution

rate(finex_engine_sql_batch_flush_time_sum[15m]) / rate(finex_engine_sql_batch_flush_time_count[15m])

Order processing rate (speed)

rate(finex_engine_orders_processed_total[15m])

Average size fo Persister SQL batched query size (bytes)

rate(finex_engine_sql_batch_size_sum[15m]) / rate(finex_engine_sql_batch_size_count[15m])

Order processing latency (time since order was processed in api and till it was flushed to db)

rate(finex_engine_order_processing_latency_sum[15m]) / rate(finex_engine_order_processing_latency_count[15m])

A bit more complex example: reject rate (percentage of orders rejected over time)

sum(rate(finex_engine_orders_processed_total{state="reject"}[15m])) / sum(rate(finex_engine_orders_processed_total[15m]))

Monitor upstream balance:

finex_engine_upstream_balance_available
finex_engine_upstream_balance_available{currency="usd"}

One option to detect low balance with AlertManager on Upstream: when available is less than 10% of funds you deposited:

finex_engine_upstream_balance_available/finex_engine_upstream_balance < 0.1

# List of available metrics

# HELP finex_engine_order_processing_latency Time spent by an order from the finex API server to the database
# TYPE finex_engine_order_processing_latency histogram
# HELP finex_engine_orderbook_size Current orderbook size
# TYPE finex_engine_orderbook_size gauge
# HELP finex_engine_orders_processed_total The total number of processed orders both accepted (wait) and rejected
# TYPE finex_engine_orders_processed_total counter
# HELP finex_engine_sql_batch_flush_time Time for sql buffer to flush
# TYPE finex_engine_sql_batch_flush_time histogram
# HELP finex_engine_sql_batch_size Size of SQL query in bytes
# TYPE finex_engine_sql_batch_size histogram
# HELP finex_engine_upstream_balance Total balance (free + locked)
# TYPE finex_engine_upstream_balance gauge
# HELP finex_engine_upstream_balance_available Availbale balance, funds you can trade with
# TYPE finex_engine_upstream_balance_available gauge
# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles.
# TYPE go_gc_duration_seconds summary
# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
# HELP go_info Information about the Go environment.
# TYPE go_info gauge
# HELP go_memstats_alloc_bytes Number of bytes allocated and still in use.
# TYPE go_memstats_alloc_bytes gauge
# HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed.
# TYPE go_memstats_alloc_bytes_total counter
# HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table.
# TYPE go_memstats_buck_hash_sys_bytes gauge
# HELP go_memstats_frees_total Total number of frees.
# TYPE go_memstats_frees_total counter
# HELP go_memstats_gc_cpu_fraction The fraction of this program's available CPU time used by the GC since the program started.
# TYPE go_memstats_gc_cpu_fraction gauge
# HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata.
# TYPE go_memstats_gc_sys_bytes gauge
# HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and still in use.
# TYPE go_memstats_heap_alloc_bytes gauge
# HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used.
# TYPE go_memstats_heap_idle_bytes gauge
# HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use.
# TYPE go_memstats_heap_inuse_bytes gauge
# HELP go_memstats_heap_objects Number of allocated objects.
# TYPE go_memstats_heap_objects gauge
# HELP go_memstats_heap_released_bytes Number of heap bytes released to OS.
# TYPE go_memstats_heap_released_bytes gauge
# HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system.
# TYPE go_memstats_heap_sys_bytes gauge
# HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection.
# TYPE go_memstats_last_gc_time_seconds gauge
# HELP go_memstats_lookups_total Total number of pointer lookups.
# TYPE go_memstats_lookups_total counter
# HELP go_memstats_mallocs_total Total number of mallocs.
# TYPE go_memstats_mallocs_total counter
# HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures.
# TYPE go_memstats_mcache_inuse_bytes gauge
# HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system.
# TYPE go_memstats_mcache_sys_bytes gauge
# HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures.
# TYPE go_memstats_mspan_inuse_bytes gauge
# HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system.
# TYPE go_memstats_mspan_sys_bytes gauge
# HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place.
# TYPE go_memstats_next_gc_bytes gauge
# HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations.
# TYPE go_memstats_other_sys_bytes gauge
# HELP go_memstats_stack_inuse_bytes Number of bytes in use by the stack allocator.
# TYPE go_memstats_stack_inuse_bytes gauge
# HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator.
# TYPE go_memstats_stack_sys_bytes gauge
# HELP go_memstats_sys_bytes Number of bytes obtained from system.
# TYPE go_memstats_sys_bytes gauge
# HELP go_threads Number of OS threads created.
# TYPE go_threads gauge
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
# HELP process_max_fds Maximum number of open file descriptors.
# TYPE process_max_fds gauge
# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
# HELP process_resident_memory_bytes Resident memory size in bytes.
# TYPE process_resident_memory_bytes gauge
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
# HELP process_virtual_memory_max_bytes Maximum amount of virtual memory available in bytes.
# TYPE process_virtual_memory_max_bytes gauge
# HELP promhttp_metric_handler_requests_in_flight Current number of scrapes being served.
# TYPE promhttp_metric_handler_requests_in_flight gauge
# HELP promhttp_metric_handler_requests_total Total number of scrapes by HTTP status code.
# TYPE promhttp_metric_handler_requests_total counter