📘 log-scanner — Generic Streaming Log Scanner Framework

A lightweight and extensible framework that reads any streaming log source (journalctl, file tail, TCP stream, Kafka consumer, etc.) and converts raw log lines into structured events using user-defined adapters.

Designed for high reusability and clean separation between:

• Log Source (journalctl, file, socket…)
• Scanner Core
• Adapters / Parsers
• Event Handlers

Used as a base library for log-driven monitoring systems such as Prometheus exporters.

📌 Overview

log-scanner is a minimal but powerful Go library that continuously reads log streams and transforms them into structured events through an adapter system.

It does not depend on log formats and does not assume any specific consumer logic.

Instead, it provides:

• A generic scanner (scanner.Start)
• A plug-in adapter (Adapter interface)
• A generic event model (model.Event)
• A handler callback for downstream processing

This makes it suitable for log monitoring, ETL ingestion, metric exporters, alert systems, and automated log analyzers.

✨ Features

• Works with any streaming log source
  • journalctl (-f)
  • tail -f (Not implimeted)
  • stdin (Not Implemented)
  • io.Reader (Not implimeted)
  • socket/TCP stream (Not implimeted)
• Pluggable adapter to parse log lines
• Graceful shutdown via context.Context
• Zero-dependency core (standard Go only)

Architecture

[ Log Stream ] → [ Scanner ] → [ Adapter.ParseLine ] → [ Handler(Event) ]

Component	Responsibility
scanner.Start	Reads log lines & drives lifecycle
adapter.Adapter	Convert raw log line → structured event
model.Event	Generic structured output
handler func(Event)	User processing logic

Installation

go get github.com/cosmostation/log-scanner@latest

Usage

A minimal example:

package main

import (
    "context"
    "os"
    "fmt"

    "github.com/cosmostation/log-scanner/scanner"
    "github.com/cosmostation/log-scanner/model"
)

func main() {
    ctx := context.Background()

    // adapter example
    ad := &MyAdapter{}

    scanner.Start(ctx, os.Stdin, ad, func(e *model.Event) {
        fmt.Printf("Event: %+v\n", e)
    })
}

Reference

Example source code using a log-scanner

• monad-exporter