Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ jobs:
run: |
set -x

cargo run --example native
cargo run --features="bridge-log" --example log_with_logger

cargo run --features="starter-log" --example simple_stdout
Expand Down
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ All notable changes to this project will be documented in this file.

## Unreleased

### Improvements

* Add instance-based native logging macros with lazy typed fields using the existing record and value model.
* Make native logging the primary documented API while retaining the optional `log` compatibility bridge.

### Breaking changes

* Bump minimum supported Rust version (MSRV) to 1.91.0.
Expand Down
119 changes: 58 additions & 61 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,88 +20,85 @@ Logforth is a versatile, extensible, and easy-to-use logging framework for Rust

## Getting Started

Add `log` and `logforth` to your `Cargo.toml`:
Add Logforth to your application:

```shell
cargo add log
cargo add logforth -F starter-log
cargo add logforth
```

## Simple Usage

Set up a basic logger that outputs to stdout:
Build a logger and pass it to the native logging macros:

```rust
use logforth::{Level, LevelFilter, append};

fn main() {
logforth::starter_log::stdout().apply();
let logger = logforth::builder()
.dispatch(|d| d
.filter(LevelFilter::MoreSevereEqual(Level::Info))
.append(append::Stderr::default()))
.build();

log::error!("This is an error message.");
log::info!("This is an info message.");
log::debug!("This debug message will not be printed by default.");
logforth::info!(logger, "service started");
logforth::debug!(logger, "this message is filtered out");
logger.flush();
}
```

By default, all logging except the `error` level is disabled. You can enable logging at other levels by setting the [`RUST_LOG`](https://docs.rs/logforth-filter-rustlog/*/logforth_filter_rustlog/index.html) environment variable. For example, `RUST_LOG=all cargo run` will print all logs.
The native API is always available and does not depend on the `log` crate. There is no implicit global logger or global maximum level: each logger's dispatches decide what to emit. An unfiltered dispatch accepts every level; a logger without dispatches emits nothing.

## Advanced Usage
## Events and structured fields

Configure multiple dispatches with different filters and appenders:
Use `trace!`, `debug!`, `info!`, `warn!`, `error!`, or `fatal!`, or `log!(logger, level, ...)` for any Logforth `Level`. `fatal!` records severity; it does not terminate the process or flush.

```rust
fn main() {
logforth::starter_log::builder()
.dispatch(|d| d
.filter(LevelFilter::MoreSevereEqual(Level::Error))
.append(append::Stderr::default()))
.dispatch(|d| d
.filter(LevelFilter::MoreSevereEqual(Level::Info))
.append(append::Stdout::default()))
.apply();

log::error!("This error will be logged to stderr.");
log::info!("This info will be logged to stdout.");
log::debug!("This debug message will not be logged.");
}
use logforth::kv::Value;

let logger = logforth::builder()
.dispatch(|d| d.append(logforth::append::Stderr::default()))
.build();
let queue = String::from("background");
logforth::info!(logger, {
"queue" => queue,
"completed" => 3u64,
"healthy" => true,
"details" => Value::debug(&[1, 2, 3]),
}, "batch complete");
logforth::info!(logger, { "queue.depth" => 0 });
```

Configure OpenTelemetry appender to export logs to an OpenTelemetry backend ([full example](https://github.com/scopedb/percas/blob/d01db13b/crates/server/src/telemetry.rs#L131-L227)):
Fields use standard `From` conversions into the existing `kv::Value`: scalars retain their types, and strings and existing values are borrowed. Use `Value::none()` for an absent value. Use `Value::display` or `Value::debug` for text and `Value::list` or `Value::map` for nested values. No logging-specific conversion trait is required.

Macros prefilter before evaluating messages, keys, fields, or conversions. Full-record filters may still reject an event afterward. The target defaults to the calling module, and source module/file/line/column identify the actual call site. Existing `RustLogFilter` module directives apply directly. Custom targets remain available through `RecordBuilder` and the `log` bridge.

Event fields describe one event. Existing `Diagnostic` implementations supply dispatch, thread, task, or trace context. These macros do not introduce logger-bound fields, merging precedence, or a context propagation API. Appenders keep their existing field/context handling. For independent filtering or output policy, build independent loggers; use `Arc<Logger>` to share a pipeline.

See the [native example](examples/src/native.rs) for static context and typed fields. Existing text/JSON layouts, rolling files, async appenders, and OTLP consume the same records. [Appender documentation](https://docs.rs/logforth-append-opentelemetry) covers OTLP configuration.

## Lifecycle and dependency logs

Stop logging producers, call `logger.flush()`, and then tear down resources required by appenders. Flush does not disable the logger or provide a transactional delivery guarantee. Independent loggers must each be flushed.

The optional `bridge-log` feature forwards dependency logs into a native logger. Install the bridge once at startup, retaining the `Arc<Logger>` for native calls and shutdown:

```rust
fn main() {
let static_diagnostic = {
let mut static_diagnostic = StaticDiagnostic::default();
static_diagnostic.insert("node_id", node_id);
static_diagnostic.insert("nodegroup", nodegroup);
static_diagnostic
};

let runtime = async_runtime();
let filter = make_rust_log_filter(&opentelemetry.filter);
let appender = runtime.block_on(async {
let exporter = opentelemetry_otlp::LogExporter::builder()
.with_tonic()
.with_endpoint(&opentelemetry.otlp_endpoint)
.with_protocol(opentelemetry_otlp::Protocol::Grpc)
.build()
.expect("failed to initialize opentelemetry logger");

append::opentelemetry::OpentelemetryLogBuilder::new(service_name, exporter)
.label("service.name", service_name)
.build()
});

logforth::starter_log::builder()
.dispatch(|b| {
b.filter(filter)
.diagnostic(FastraceDiagnostic::default())
.diagnostic(static_diagnostic)
.append(appender)
})
.apply();
use std::sync::Arc;
use logforth::append;
use logforth::bridge::log::LogBridge;

fn main() -> Result<(), log::SetLoggerError> {
let logger = Arc::new(logforth::builder()
.dispatch(|d| d.append(append::Stderr::default()))
.build());
log::set_boxed_logger(Box::new(LogBridge::new(logger.clone())))?;
log::set_max_level(log::LevelFilter::Trace);
logforth::info!(logger, "native event");
log::info!("dependency event");
logger.flush();
Ok(())
}
```

Read more demos under the [examples](examples) directory.
Add `log` and enable `logforth/bridge-log` for this example. `bridge-log-serde` continues to support Serde fields from the `log` facade. Native Serde capture is outside this API; the existing `Value` model supports explicit nested fields. The `starter-log` helpers remain available for applications using the compatibility facade.

## Features

Expand Down Expand Up @@ -191,7 +188,7 @@ Users can also provide their own MDC by implementing the [`Diagnostic`] trait.

### Bridges

So far, Logforth provides out-of-the-box integration with the `log` crate. You can use Logforth as the backend for any crate that uses the `log` facade.
The optional `log` bridge forwards dependency logs into the same dispatch graph as native events. It preserves the original level, target, source location, and supported structured fields.

## Documentation

Expand Down
7 changes: 6 additions & 1 deletion core/src/filter/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,12 @@ pub enum FilterResult {

/// A filter that can be applied to log records.
pub trait Filter: fmt::Debug + Send + Sync + 'static {
/// Whether the record is filtered by its given metadata.
/// Prefilter an event by level and target before its payload is evaluated.
///
/// Return `Neutral` if the decision needs message, fields, or source location,
/// and inspect those in [`Self::matches`]. `Reject` must mean no matching
/// record could be accepted with these criteria. An accepted prefilter does
/// not bypass the full-record filter pass.
fn enabled(&self, criteria: &FilterCriteria, diags: &[Box<dyn Diagnostic>]) -> FilterResult;

/// Whether the record is filtered.
Expand Down
14 changes: 14 additions & 0 deletions core/src/kv.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@

//! Key-value pairs in a log record or a diagnostic context.

mod convert;

use std::borrow::Borrow;
use std::borrow::Cow;
use std::collections::HashMap;
Expand Down Expand Up @@ -363,6 +365,10 @@ impl<'a> Iterator for MapValueIter<'a> {
}

/// A borrowed value in a key-value pair.
///
/// Standard [`From`] conversions preserve scalar types and borrow strings,
/// and existing values. Use [`Self::display`] or [`Self::debug`]
/// to explicitly capture a value as text.
#[derive(Debug, Clone, Copy)]
pub struct Value<'a>(ValueState<'a>);

Expand All @@ -382,6 +388,7 @@ enum ValueState<'a> {
Map(&'a [(Key<'a>, Value<'a>)]),
Debug(&'a dyn fmt::Debug),
Display(&'a dyn fmt::Display),
Owned(&'a ValueOwned),
}

impl fmt::Debug for ValueState<'_> {
Expand All @@ -401,6 +408,7 @@ impl fmt::Debug for ValueState<'_> {
ValueState::Map(v) => v.fmt(f),
ValueState::Debug(v) => fmt::Debug::fmt(v, f),
ValueState::Display(v) => fmt::Display::fmt(v, f),
ValueState::Owned(v) => fmt::Debug::fmt(v, f),
}
}
}
Expand Down Expand Up @@ -431,11 +439,17 @@ impl Value<'_> {
ValueState::Map(m) => ValueView::Map(MapValue(MapValueState::Borrowed(m))),
ValueState::Debug(d) => ValueView::Debug(DebugValue(d)),
ValueState::Display(d) => ValueView::Display(DisplayValue(d)),
ValueState::Owned(v) => v.view(),
}
}
}

impl<'a> Value<'a> {
/// Borrow an owned value without copying its strings or nested values.
pub fn borrowed(value: &'a ValueOwned) -> Self {
Self(ValueState::Owned(value))
}

/// Create a value representing the absence of data.
pub fn none() -> Value<'a> {
Value(ValueState::None)
Expand Down
94 changes: 94 additions & 0 deletions core/src/kv/convert.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
// Copyright 2024 FastLabs Developers
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use super::Value;
use super::ValueOwned;

impl<'a> From<&&'a str> for Value<'a> {
fn from(value: &&'a str) -> Self {
Self::str(value)
}
}

impl<'a> From<&'a Value<'_>> for Value<'a> {
fn from(value: &'a Value<'_>) -> Self {
*value
}
}

impl<'a> From<&'a ValueOwned> for Value<'a> {
fn from(value: &'a ValueOwned) -> Self {
Self::borrowed(value)
}
}

impl<'a> From<&'a str> for Value<'a> {
fn from(value: &'a str) -> Self {
Self::str(value)
}
}

impl<'a> From<&'a String> for Value<'a> {
fn from(value: &'a String) -> Self {
Self::str(value)
}
}

impl<'a> From<&&'a String> for Value<'a> {
fn from(value: &&'a String) -> Self {
Self::str(value)
}
}

impl<'a> From<&&'a Value<'_>> for Value<'a> {
fn from(value: &&'a Value<'_>) -> Self {
**value
}
}

impl<'a> From<&&'a ValueOwned> for Value<'a> {
fn from(value: &&'a ValueOwned) -> Self {
Self::borrowed(value)
}
}

macro_rules! scalar {
($constructor:ident, $repr:ty, $($ty:ty),+ $(,)?) => {
$(impl From<$ty> for Value<'_> {
fn from(value: $ty) -> Self {
Self::$constructor(value as $repr)
}
}

impl From<&$ty> for Value<'_> {
fn from(value: &$ty) -> Self {
<Self as From<$ty>>::from(*value)
}
}

impl From<&&$ty> for Value<'_> {
fn from(value: &&$ty) -> Self {
<Self as From<$ty>>::from(**value)
}
})+
};
}

scalar!(bool, bool, bool);
scalar!(char, char, char);
scalar!(i64, i64, i8, i16, i32, i64, isize);
scalar!(u64, u64, u8, u16, u32, u64, usize);
scalar!(i128, i128, i128);
scalar!(u128, u128, u128);
scalar!(f64, f64, f32, f64);
2 changes: 2 additions & 0 deletions core/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ pub use self::trap::Trap;
mod error;
pub use self::error::*;

mod macros;

mod logger;
pub use self::logger::*;

Expand Down
11 changes: 9 additions & 2 deletions core/src/logger/log_impl.rs
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,10 @@ impl Logger {
}

impl Logger {
/// Determine if a log message with the specified metadata would be logged.
/// Test whether any dispatch might accept an event with this level and target.
///
/// This is a conservative prefilter. Full-record filters can still reject
/// the event. Native macros call this before evaluating messages and fields.
pub fn enabled(&self, criteria: &FilterCriteria) -> bool {
self.dispatches
.iter()
Expand All @@ -52,7 +55,11 @@ impl Logger {
}
}

/// Flush any buffered records.
/// Flush buffered records through every configured appender.
///
/// Stop producers before flushing during shutdown, and keep resources needed
/// by appenders alive until this returns. This does not disable the logger.
/// Errors use the existing stderr fallback. There is no implicit exit hook.
pub fn flush(&self) {
for dispatch in &self.dispatches {
for err in dispatch.flush() {
Expand Down
Loading