Strange invalid json is received by Seq

[ceregador]

Hi.
We use Serilog.Sinks.Seq to log NSeviceBus events.
At starting of NServiceBus host Seq receives following invalid json request (copied from invalid-BadRequest-....json file):

{"Events":
[{"Timestamp":"2016-08-17T18:02:01.2348376+03:00","Level":"Information","MessageTemplate":"{Text:l}",
"Properties":{"Text":"Using license supplied via fluent API.","SourceContext":"NServiceBus.Licensing.LicenseManager","MachineName":"SSSSS","EnvironmentUserName":"SSSSSS"},
"Renderings":{"Text":[{"Format":"l","Rendering":"Using license supplied via fluent API."}]}},
imestamp":"2016-08-17T18:02:01.3689267+03:00","Level":"Information","MessageTemplate":"Activating persistence '{0}' to provide storage for '{1}' storage.","Properties":{"0":"NHibernatePersistence",...

There are few missing characters: "{T before second event (imestamp).
We can't determine cause of this strange behaviour.

What can be reason of this?

[nblumhardt]

Hi Sergey,

Are you using the bufferBaseFilename (durable buffering mode) of the Seq sink? This might be possible if there's somehow been corruption of the disk buffer file in that case.

[ceregador]

Hi Nicholas,
Yes, we use durable buffering mode.
We tried to clear the directory but it's reproduced with empty buffer files directory, too.

[nblumhardt]

Thank you for the follow-up. Under the hood, the buffer file is written by the Serilog rolling file sink, that in turn uses the Serilog file sink. Could you please let me know which versions of these packages are being used?

• Serilog
• Serilog.Sinks.File
• Serilog.Sinks.RollingFile
• Serilog.Sinks.Seq

When you say the issue is reproduced on an empty buffer files directory, is it at a particular point in the log file that this occurs, or is it seemingly "at random"?

Thanks!

[nblumhardt]

Also, have you specified a value for bufferFileSizeLimitBytes? Looking at the implementation now, I suspect doing so could cause problems, though I don't think they'd manifest in the same way as your {T truncation.

[ceregador]

Package versions:
Serilog - 2.0.0
Serilog.Sinks.File - 2.2.0
Serilog.Sinks.RollingFile - 2.2.0
Serilog.Sinks.Seq - 2.0.0.

It's always reproduced for the same event (about "activating persistence"), so this is a particular point in the log file.

We don't specify a value for bufferFileSizeLimitBytes, only bufferBaseFilename.

[nblumhardt]

Thanks for the further information, I think I've tracked it down.

https://github.com/serilog/serilog-sinks-seq/blob/dev/src/Serilog.Sinks.Seq/Sinks/Seq/HttpLogShipper.cs#L424 compensates for the existence of a byte-order mark (BOM) in the file, however as of Serilog.Sinks.File v2.2 the default encoding no longer emits it (serilog/serilog-sinks-file#10).

Bug fix on its way.

[nblumhardt]

This should be fixed as of Serilog.Sinks.Seq 2.0.1, now on NuGet.

Unfortunately, you'll need to manually convert any buffered .json files to have the Unicode BOM before logging will kick off again. Let me know if you need a hand with it, or if you continue to see any issues.

Thanks again for the report, much appreciated.

[ceregador]

Hi Nicholas,
I checked with updated package - the problem is not reproduced anymore.
Thanks!

[nblumhardt]

Excellent, thanks for confirming!