mirror of
https://github.com/tiennm99/tjahzi.git
synced 2026-10-15 12:15:11 +00:00
88 lines
5.6 KiB
Markdown
88 lines
5.6 KiB
Markdown
# About
|
|
[Tjahzi](http://www.thorgal.com/personnages/tjahzi/) is a set of Java tools and appenders for logging to [Grafana Loki](https://grafana.com/oss/loki/).
|
|
|
|
[](https://github.com/tkowalcz/tjahzi/blob/master/LICENSE)
|
|
[](https://github.com/tkowalcz/tjahzi/commits/master)
|
|
[](https://app.circleci.com/pipelines/github/tkowalcz/tjahzi?branch=master)
|
|
|
|
Latest releases:
|
|
|
|
[](https://search.maven.org/search?q=g:pl.tkowalcz.tjahzi)
|
|
[](https://search.maven.org/search?q=g:pl.tkowalcz.tjahzi)
|
|
[](https://search.maven.org/search?q=g:pl.tkowalcz.tjahzi)
|
|
|
|
If you find the project useful (or not useful for some reason) please let me know to encourage further development. ⭐ Stars are also welcome ⭐ ;).
|
|
|
|
## Features
|
|
|
|
Tjahzi allows pushing log data to Loki. It uses protobuf format to push log lines with timestamps and labels to Loki. This
|
|
is implemented in the `core` component. On top of that we built appenders for `log4j2` and `Logback`. These feature:
|
|
|
|
1. Logging does not allocate objects nor take any locks.
|
|
1. Sending data to Loki in batches allocates as little as possible.
|
|
1. We also provide a no-dependency versions of these appenders.
|
|
1. Includes in-house implementation of protobuf wire format for Loki to reduce dependencies and improve performance.
|
|
|
|
These points apply to log4j2 appender. Logback design does not allow allocation free operation, but we go to great
|
|
lengths to reduce allocations and even provide an optimized version of pattern layout.
|
|
|
|
Log4j2 appender is currently used to ship logs from tens of servers at about 10GB of logs per day.
|
|
|
|
## Getting started
|
|
|
|
#### Log4j2
|
|
For `log4j2` appender quick start guide and detailed discussions see this [README.md](log4j2-appender/README.md).
|
|
|
|
#### Logback
|
|
For `Logback` appender quick start guide and detailed discussions see this [README.md](logback-appender/README.md).
|
|
|
|
## Core design principles
|
|
|
|
Logging should be lightweight and not interfere with main business tasks of threads that happen to log a message.
|
|
Asking the logging subsystem to log a message should be as CPU efficient as possible.
|
|
That's a truism. Apart from computation itself there are many other causes of jitter (varying speed of code execution).
|
|
Thread can be slowed down by excessive allocations, by initialization code running in constructors of unnecessarily allocated objects,
|
|
by garbage collector activity that is triggered by it. There can be configuration refresh checks on logging path, inter thread signaling etc.
|
|
|
|
To avoid these effects we strive to adhere to the following principles (and document any violations):
|
|
|
|
1. Lock free and wait free API
|
|
2. Allocation free in steady state
|
|
|
|
You can compare this with [design principles of Aeron](https://github.com/real-logic/aeron/wiki/Design-Principles) which are close to our hearts.
|
|
|
|
## Architecture
|
|
|
|
```
|
|
[via 10kB thread local buffer]
|
|
│
|
|
┌─────────────┐ Log ←──┘
|
|
│ Application │----------------┐
|
|
│ Thread 1 │ │
|
|
└─────────────┘ │
|
|
│
|
|
. ▼
|
|
┌────────────────────┐ ┌─────────┐ ┌─────────┐
|
|
. Log │ │ │ Log │ │ I/O │
|
|
----------------▶│ Log buffer ├--→---→--┤ shipper ├--→---→--┤ thread │
|
|
. │ │ │ agent │ │ (Netty) │
|
|
└────────────────────┘ └─────────┘ └─────────┘
|
|
. ▲
|
|
│
|
|
┌─────────────┐ Log │
|
|
│ Application │----------------------┘
|
|
│ Thread N │
|
|
└─────────────┘
|
|
```
|
|
|
|
For those interested wiki contains some information on [log buffer sizing](https://github.com/tkowalcz/tjahzi/wiki/Log-buffer-sizing).
|
|
|
|
## Monitoring
|
|
|
|
Check out [this wiki article](https://github.com/tkowalcz/tjahzi/wiki/Monitoring).
|
|
|
|
# LICENSE
|
|
|
|
This work is released under MIT license. Feel free to use, copy and modify this work as long as you credit original authors.
|
|
Pull and feature requests are welcome.
|