Laravel Monolog Handler for Logflare

For our API, we’ve been happily using NewRelic’s monolog enricher for a while, which sends our application logs to NewRelic at the end of each request, making it light and fast for our system not to be bothered by it.

Until it stopped working with the upgrade to Composer 2, and they knew about it for several months and still didn’t do a single thing to fix it.

So I decided to move to Logflare.

Logflare is a fast, light, scalable, and powerful logging aggregator. It has a simple and usable interface, basically the exact opposite of NewRelic.

Chase Granberry is the founder of Logflare and was happy to help me when I had some challenges, making it yet another opposite of NewRelic, where any complaint ends up in “we’ll take your feedback into account”. But enough venting about New Relic, let’s get to the good stuff.

Logflare accepts log intake via various methods, including JavaScript, Elixir, Cloudflare. One of them is a helpful, yet very generic RESTful API endpoint. It’s accepting structured JSON log records, which is exactly what we need.

To send logs to Logflare, we need the following updates:

• updating config/logging.php with our new logger;
• adding the Logflare source and API Key to .env;
• create a new Monolog Logger class;
• create a new Monolog Handler class;

Let’s dive in.

First, sign up for a Logflare account. It takes a minute. After signing up, create a new Source, and get the source ID and API key. We’ll need this to be able to send logs from Laravel.

Then, create a new logging channel in config/logging.php:

return [
    'channels' => [
        'stack' => [
            'driver' => 'stack',
            'channels' => ['daily', 'logflare'],
            'ignore_exceptions' => false,
        ],
        // ... after your usual logging channels
        // add our logflare channel:
        ‘logflare’ => [
            ‘driver’ => ‘custom’,
            ‘via’ => App\Logging\LogflareLogger::class,
            ‘api_key’ => env(‘LOGFLARE_API_KEY’),
            ‘source’ => env(‘LOGFLARE_SOURCE’, ‘YOUR-DEFAULT-SOURCE-HERE’),
        ],
    ],
];

We’re using the stack channel, so all the logs go locally on our server for future references, while we’re also sending logs to Logflare for quick access. More about why we’re doing this below.

I’m keen on keeping our default source in the configuration file, overridable from the .env file. The API key, however, should never be committed to your repository. For any service you might be using. Never, ever, commit API keys.

So let’s add these to .env:

LOGFLARE_API_KEY=YOUR-API-KEY
LOGFLARE_SOURCE=YOUR-SOURCE-ID

Next, let’s create the Logger class. For us, it’s a new file called LogflareHandler.php saved in our app/Logging folder:

<?php

namespace App\Logging;

use Monolog\Logger;
use Illuminate\Http\Request;
use Monolog\Handler\BufferHandler;
use Monolog\Handler\FingersCrossedHandler;

/**
 * Custom LogFlare Logger
 *
 * This Logger sends information to Logflare, but in a
 * very smart way:
 *  - complete logs are sent if an ERROR or higher is triggered
 *  - Logger: WARNING messages are passing through regardless
 */
class LogflareLogger
{
    protected $request;
    protected $sequence;

    public function __construct(Request $request = null)
    {
        $this->request = $request;
    }

    public function __invoke(array $config)
    {
        $this->sequence = uniqid('seq', true);

        $logger = new Logger('logflare');
        $handler = new LogflareHandler(Logger::DEBUG, $config);
        $bufferHandler = new FingersCrossedHandler(
            $handler, // upstream handler
            Logger::ERROR, // activation strategy
            0, // bufferSize
            true, // bubble
            true, // stopBuffering
            Logger::WARNING,
        );

        $logger->pushHandler($bufferHandler);

        collect($logger->getHandlers())->each(function ($handler) {
            $handler->pushProcessor(function ($record) {
                $record['time'] = new \DateTime();
                $record['extra'] = array_merge($record['extra'] ?? [], [
                    'seq' => $this->sequence,
                    'level' => $record['level'],
                    'level_name' => $record['level_name'],
                    'hostname' => gethostname(),
                    'user_id' => auth()->user() ? auth()->user()->id : null,
                    'origin' => $this->request->headers->get('origin'),
                    'ip' => $this->request->server('REMOTE_ADDR'),
                    'user_agent' => $this->request->server('HTTP_USER_AGENT'),
                    'context' => $record['context'] ?? null,
                ]);
                return $record;
            });
        });

        return $logger;
    }
}

Some notes about what we are doing above, and why.

We’re using the FingersCrossedHandler for two very important reasons. First, it batches up log records, thus avoiding slowing down our API whit the activity of logging messages.

And second, it only sends them forward (to Logflare) if we have logged an ERROR-level message or above. So if we only have DEBUG messages to log, they aren’t even sent to Logflare.

This is very useful when things go wrong. Which is the main reason we have logging in the first place.

If any request generates an error, not only we’re getting the error into Logflare, but we’re also getting all the breadcrumbs (debug messages) that lead to that error, and all the breadcrumbs that follow. This proved to be very useful, and smart.

The last parameter of the FingersCrossedHandler constructor is the $passthruLevel, meaning the level that we always want to be sent forward, regardless of what’s being logged before or after. In our case, we’re getting all the WARNING logs without the debugs. Magic!

Another thing that we’re doing here is enriching the records that we’re logging with very useful information, such as:

seq is a sequence number, basically a unique ID for the request. Useful to be able to distinguish (or search) logs that originated from a specific request; without it, all the log messages would be mixed and we would not know which log message came from which request. Especially since when we get an error, we want to see all the messages for that specific request.

level and level_name are the debug levels being logged.

hostname is the hostname of the machine that generated the log message. Useful to distinguish between production servers, staging, or local development. We’ve set up Logflare in a way where all the log messages go to the same source (bucket), and from there, using rules, they’re being dispatched to the production, staging, or local development sources.

user_id If the request is authenticated, include the user ID.

origin The request’s Origin header.

ip The IP of the request.

user_agent The User-Agent that sent the request. This will show up as Symfony for CLI commands or Queue jobs.

context This contains the context of the debug messages, as logged when calling \Log::debug(‘message’, $context); The second parameter is an array that may contain useful information regarding the message being logged.

The final piece of the puzzle is the LogflareHandler.php file, which sends the records batched to Logflare in a single request, at the end of the ongoing request.

<?php

namespace App\Logging;

use Monolog\Handler\AbstractProcessingHandler;
use Monolog\Logger;
use GuzzleHttp\Client as GuzzleClient;

class LogflareHandler extends AbstractProcessingHandler
{
    protected $config = [];

    public function __construct($level = Logger::DEBUG, array $config = [])
    {
        parent::__construct($level);
        $this->config = $config;
    }

    /**
     * {@inheritdoc}
     */
    public function handleBatch(array $records): void
    {
        $batch = [];
        foreach ($records as $record) {
            $batch[] = [
                'timestamp' => $record['time']->format(DATE_ISO8601),
                'log_entry' => $record['level_name'] . '|' . $record['message'],
                'metadata' => $record['extra'],
            ];
        }

        $client = new GuzzleClient();
        try {
            $client->request(
                'POST',
                sprintf('https://api.logflare.app/logs?api_key=&source=%s', $this->config['source']),
                [
                    'json' => ['batch' => $batch],
                    'headers' => [
                        'Content-Type' => 'application/json; charset=utf-8',
                        'X-API-KEY' => $this->config['api_key'],
                    ]
                ]
            );
        } catch (\Exception $e) {
            // silently drop logging errors...
            // throw $e;
        }
    }

    protected function write(array $record): void
    {
        $this->handleBatch([$record]);
    }
}

Important to note here is that to batch up requests, we’re sending a batch attribute with the array of log records.

Each log record has the following attributes:

• timestamp - This is important to send because we want Logflare to show the date and time of the logging, not the date/time that our batch of logs was sent to Logflare. So we’re overriding the default (current) timestamp with the value that we saved when we logged the message.
• log_entry - This is the actual log message.
• metadata - All the enriched information that we’ve been saving in LogflareHandler with each log record.

Besides that, it’s just a simple Guzzle POST request. We’re ignoring all potential sending errors, as we don’t want our application to crash if it cannot send log messages.

We’re using the stack channel which logs all messages in a local file as well as sending them to Logflare, it’s not the end of the world if Logflare cannot accept some messages due to maintenance or connectivity issues.

Having this set up should be everything you need to see your logs coming into Logflare.

I hope this helped, as Logflare is rather new and I couldn’t find documentation on Logflare and Monolog anywhere online.

–Photo by Elia Pellegrini on Unsplash, Logflare is a trademark of Logflare LLC, Laravel is a Trademark of Taylor Otwell.