In System.Text.Json is it possible to specify custom indentation rules?

Question

Edit: I made an issue at the .Net runtime repo yesterday which was closed to by "layomia" with this message: "Adding extension points like this comes with a performance cost at the lower-level reader and writer and does not present a good balance between perf and functionality/benefit. Providing such configuration is not on the System.Text.Json roadmap."

When setting JsonSerializerOptions.WriteIndented = true indentation looks like this when writing json...

{
  "TILESET": "tilesets/HOW_TO_GET_TILESET_NAME_?",
  "TILES": {
    "TILE_1": {
      "NAME": "auto_tile_18",
      "TEXTURE_BOUNDS": [
        304,
        16,
        16,
        16
      ],
      "SCREEN_BOUNDS": [
        485,
        159,
        64,
        64
      ]
    }
  }
}

Is there a way to change the automatic indentation to something like this...

{
  "TILESET": "tilesets/HOW_TO_GET_TILESET_NAME_?",
  "TILES": 
  {
    "TILE_1": 
    {
      "NAME": "auto_tile_18",
      "TEXTURE_BOUNDS": [304, 16, 16,16],
      "SCREEN_BOUNDS": [485, 159, 64, 64]
    }
  }
}

dbc · Accepted Answer · 2022-01-05T23:27:04.580

This is not possible currently with System.Text.Json (as of .NET 5). Let's consider the possibilities:

JsonSerializerOptions has no method to control indentation other than the Boolean property WriteIndented:

Gets or sets a value that defines whether JSON should use pretty printing.
Utf8JsonWriter has no method to modify or control indentation, as Options is a get-only struct-valued property.

In .Net Core 3.1, if I create a custom JsonConverter<T> for your TEXTURE_BOUNDS and SCREEN_BOUNDS lists and attempt set options.WriteIndented = false; during serialization, a System.InvalidOperationException: Serializer options cannot be changed once serialization or deserialization has occurred exception will be thrown.

Specifically, if I create the following converter:

class CollectionFormattingConverter<TCollection, TItem> : JsonConverter<TCollection> where TCollection : class, ICollection<TItem>, new()
{
    public override TCollection Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
        => JsonSerializer.Deserialize<CollectionSurrogate<TCollection, TItem>>(ref reader, options)?.BaseCollection;

    public override void Write(Utf8JsonWriter writer, TCollection value, JsonSerializerOptions options)
    {
        var old = options.WriteIndented;
        try
        {
            options.WriteIndented = false;
            JsonSerializer.Serialize(writer, new CollectionSurrogate<TCollection, TItem>(value), options);
        }
        finally
        {
            options.WriteIndented = old;
        }
    }
}

public class CollectionSurrogate<TCollection, TItem> : ICollection<TItem> where TCollection : ICollection<TItem>, new()
{
    public TCollection BaseCollection { get; }

    public CollectionSurrogate() { this.BaseCollection = new TCollection(); }
    public CollectionSurrogate(TCollection baseCollection) { this.BaseCollection = baseCollection ?? throw new ArgumentNullException(); }

    public void Add(TItem item) => BaseCollection.Add(item);
    public void Clear() => BaseCollection.Clear();
    public bool Contains(TItem item) => BaseCollection.Contains(item);
    public void CopyTo(TItem[] array, int arrayIndex) => BaseCollection.CopyTo(array, arrayIndex);
    public int Count => BaseCollection.Count;
    public bool IsReadOnly => BaseCollection.IsReadOnly;
    public bool Remove(TItem item) => BaseCollection.Remove(item);
    public IEnumerator<TItem> GetEnumerator() => BaseCollection.GetEnumerator();
    System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator() => ((IEnumerable)BaseCollection).GetEnumerator();
}

And the following data model:

public partial class Root
{
    [JsonPropertyName("TILESET")]
    public string Tileset { get; set; }
    [JsonPropertyName("TILES")]
    public Tiles Tiles { get; set; }
}

public partial class Tiles
{
    [JsonPropertyName("TILE_1")]
    public Tile1 Tile1 { get; set; }
}

public partial class Tile1
{
    [JsonPropertyName("NAME")]
    public string Name { get; set; }

    [JsonPropertyName("TEXTURE_BOUNDS")]
    [JsonConverter(typeof(CollectionFormattingConverter<List<long>, long>))]
    public List<long> TextureBounds { get; set; }

    [JsonPropertyName("SCREEN_BOUNDS")]
    [JsonConverter(typeof(CollectionFormattingConverter<List<long>, long>))]
    public List<long> ScreenBounds { get; set; }
}

Then serializing Root throws the following exception:

Failed with unhandled exception: 
System.InvalidOperationException: Serializer options cannot be changed once serialization or deserialization has occurred.
   at System.Text.Json.ThrowHelper.ThrowInvalidOperationException_SerializerOptionsImmutable()
   at System.Text.Json.JsonSerializerOptions.set_WriteIndented(Boolean value)
   at CollectionFormattingConverter`2.Write(Utf8JsonWriter writer, TCollection value, JsonSerializerOptions options)
   at System.Text.Json.JsonPropertyInfoNotNullable`4.OnWrite(WriteStackFrame& current, Utf8JsonWriter writer)
   at System.Text.Json.JsonPropertyInfo.Write(WriteStack& state, Utf8JsonWriter writer)
   at System.Text.Json.JsonSerializer.Write(Utf8JsonWriter writer, Int32 originalWriterDepth, Int32 flushThreshold, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.JsonSerializer.WriteCore(Utf8JsonWriter writer, Object value, Type type, JsonSerializerOptions options)
   at System.Text.Json.JsonSerializer.WriteCore(PooledByteBufferWriter output, Object value, Type type, JsonSerializerOptions options)
   at System.Text.Json.JsonSerializer.WriteCoreString(Object value, Type type, JsonSerializerOptions options)
   at System.Text.Json.JsonSerializer.Serialize[TValue](TValue value, JsonSerializerOptions options)

Demo fiddle #1 here.

In .Net Core 3.1, if I create a custom JsonConverter<T> that creates a pre-formatted JsonDocument and then writes that out, the document will be reformatted as it is written.

I.e. if I create the following converter:

class CollectionFormattingConverter<TCollection, TItem> : JsonConverter<TCollection> where TCollection : class, ICollection<TItem>, new()
{
    public override TCollection Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
        => JsonSerializer.Deserialize<CollectionSurrogate<TCollection, TItem>>(ref reader, options)?.BaseCollection;

    public override void Write(Utf8JsonWriter writer, TCollection value, JsonSerializerOptions options)
    {
        var copy = options.Clone();
        copy.WriteIndented = false;
        using var doc = JsonExtensions.JsonDocumentFromObject(new CollectionSurrogate<TCollection, TItem>(value), copy);
        Debug.WriteLine("Preformatted JsonDocument: {0}", doc.RootElement);
        doc.WriteTo(writer);
    }
}

public static partial class JsonExtensions
{
    public static JsonSerializerOptions Clone(this JsonSerializerOptions options)
    {
        if (options == null)
            return new JsonSerializerOptions();
        //In .Net 5 a copy constructor will be introduced for JsonSerializerOptions.  Use the following in that version.
        //return new JsonSerializerOptions(options);
        //In the meantime copy manually.
        var clone = new JsonSerializerOptions
        {
            AllowTrailingCommas = options.AllowTrailingCommas,
            DefaultBufferSize = options.DefaultBufferSize,
            DictionaryKeyPolicy = options.DictionaryKeyPolicy,
            Encoder = options.Encoder,
            IgnoreNullValues = options.IgnoreNullValues,
            IgnoreReadOnlyProperties = options.IgnoreReadOnlyProperties,
            MaxDepth = options.MaxDepth,
            PropertyNameCaseInsensitive = options.PropertyNameCaseInsensitive,
            PropertyNamingPolicy = options.PropertyNamingPolicy,
            ReadCommentHandling= options.ReadCommentHandling,
            WriteIndented = options.WriteIndented,
        };
        foreach (var converter in options.Converters)
            clone.Converters.Add(converter);
        return clone;
    }

    // Copied from this answer https://stackoverflow.com/a/62998253/3744182
    // To https://stackoverflow.com/questions/62996999/convert-object-to-system-text-json-jsonelement
    // By https://stackoverflow.com/users/3744182/dbc

    public static JsonDocument JsonDocumentFromObject<TValue>(TValue value, JsonSerializerOptions options = default) 
        => JsonDocumentFromObject(value, typeof(TValue), options);

    public static JsonDocument JsonDocumentFromObject(object value, Type type, JsonSerializerOptions options = default)
    {
        var bytes = JsonSerializer.SerializeToUtf8Bytes(value, options);
        return JsonDocument.Parse(bytes);
    }
}

Fully indented JSON is generated despite the fact that the intermediate JsonDocument doc was serialized without indentation:

{
  "TILESET": "tilesets/HOW_TO_GET_TILESET_NAME_?",
  "TILES": {
    "TILE_1": {
      "NAME": "auto_tile_18",
      "TEXTURE_BOUNDS": [
        304,
        16,
        16,
        16
      ],
      "SCREEN_BOUNDS": [
        485,
        159,
        64,
        64
      ]
    }
  }
}

Demo fiddle #2 here.

And finally, in .Net Core 3.1, if I create a custom JsonConverter<T> that clones the incoming JsonSerializerOptions, modifies WriteIndented on the copy, then recursively serializes using the copied settings -- the modified value for WriteIndented is ignored.

Demo fiddle #3 here.

Apparently the JsonConverter architecture is going to be extensively enhanced in .Net 5 so you might re-test this option when it is released.

You might want to open an issue requesting this functionality, as there are multiple popular questions about how to do this with Json.NET (where it can be done with a converter):

Thank you for the comprehensive answer. I have opened an issue here: https://github.com/dotnet/runtime/issues/40731 — Moritz Schäfer, Aug 12 '20 at 19:33
@Alexey.Petriashev - You can add another answer if you want. — dbc, Dec 26 '20 at 20:32

score 2 · Answer 2 · answered Dec 27 '20 at 20:04

Faced with the same problem. I need to write arrays in one row for json simplicity.

Latest version is here: https://github.com/micro-elements/MicroElements.Metadata/blob/master/src/MicroElements.Metadata.SystemTextJson/SystemTextJson/Utf8JsonWriterCopier.cs

Solution:

I Use reflection to create clone of Utf8JsonWriter with desired options (see class Utf8JsonWriterCopier.cs)
To check that API was not changed Clone calls Utf8JsonWriterCopier.AssertReflectionStateIsValid, also you can use it in your tests

Usage:

Create NotIndented copy of Utf8JsonWriter
Write array
Copy internal state back to original writer

Sample:

if (Options.WriteArraysInOneRow && propertyType.IsArray && writer.Options.Indented)
{
    // Creates NotIndented writer
    Utf8JsonWriter writerCopy = writer.CloneNotIndented();

    // PropertyValue
    JsonSerializer.Serialize(writerCopy, propertyValue.ValueUntyped, propertyType, options);

    // Needs to copy internal state back to writer
    writerCopy.CopyStateTo(writer);
}

Utf8JsonWriterCopier.cs

/// <summary>
/// Helps to copy <see cref="Utf8JsonWriter"/> with other <see cref="JsonWriterOptions"/>.
/// This is not possible with public API so Reflection is used to copy writer internals.
/// See also: https://stackoverflow.com/questions/63376873/in-system-text-json-is-it-possible-to-specify-custom-indentation-rules.
/// Usage:
/// <code>
/// if (Options.WriteArraysInOneRow and propertyType.IsArray and writer.Options.Indented)
/// {
///     // Create NotIndented writer
///     Utf8JsonWriter writerCopy = writer.CloneNotIndented();
///
///     // Write array
///     JsonSerializer.Serialize(writerCopy, array, options);
///
///     // Copy internal state back to writer
///     writerCopy.CopyStateTo(writer);
/// }
/// </code>
/// </summary>
public static class Utf8JsonWriterCopier
{
    private class Utf8JsonWriterReflection
    {
        private IReadOnlyCollection<string> FieldsToCopyNames { get; } = new[] { "_arrayBufferWriter", "_memory", "_inObject", "_tokenType", "_bitStack", "_currentDepth" };

        private IReadOnlyCollection<string> PropertiesToCopyNames { get; } = new[] { "BytesPending", "BytesCommitted" };

        private FieldInfo[] Fields { get; }

        private PropertyInfo[] Properties { get; }

        internal FieldInfo OutputField { get; }

        internal FieldInfo StreamField { get; }

        internal FieldInfo[] FieldsToCopy { get; }

        internal PropertyInfo[] PropertiesToCopy { get; }

        public Utf8JsonWriterReflection()
        {
            Fields = typeof(Utf8JsonWriter).GetFields(BindingFlags.Instance | BindingFlags.NonPublic);
            Properties = typeof(Utf8JsonWriter).GetProperties(BindingFlags.Instance | BindingFlags.Public);
            OutputField = Fields.FirstOrDefault(info => info.Name == "_output")!;
            StreamField = Fields.FirstOrDefault(info => info.Name == "_stream")!;

            FieldsToCopy = FieldsToCopyNames
                .Select(name => Fields.FirstOrDefault(info => info.Name == name))
                .Where(info => info != null)
                .ToArray();

            PropertiesToCopy = PropertiesToCopyNames
                .Select(name => Properties.FirstOrDefault(info => info.Name == name))
                .Where(info => info != null)
                .ToArray();
        }

        public void AssertStateIsValid()
        {
            if (OutputField == null)
                throw new ArgumentException("Field _output is not found. API Changed!");
            if (StreamField == null)
                throw new ArgumentException("Field _stream is not found. API Changed!");
            if (FieldsToCopy.Length != FieldsToCopyNames.Count)
                throw new ArgumentException("Not all FieldsToCopy found in Utf8JsonWriter. API Changed!");
            if (PropertiesToCopy.Length != PropertiesToCopyNames.Count)
                throw new ArgumentException("Not all FieldsToCopy found in Utf8JsonWriter. API Changed!");
        }
    }

    private static readonly Utf8JsonWriterReflection _reflectionCache = new Utf8JsonWriterReflection();

    /// <summary>
    /// Checks that reflection API is valid.
    /// </summary>
    public static void AssertReflectionStateIsValid()
    {
        _reflectionCache.AssertStateIsValid();
    }

    /// <summary>
    /// Clones <see cref="Utf8JsonWriter"/> with new <see cref="JsonWriterOptions"/>.
    /// </summary>
    /// <param name="writer">Source writer.</param>
    /// <param name="newOptions">Options to use in new writer.</param>
    /// <returns>New copy of <see cref="Utf8JsonWriter"/> with new options.</returns>
    public static Utf8JsonWriter Clone(this Utf8JsonWriter writer, JsonWriterOptions newOptions)
    {
        AssertReflectionStateIsValid();

        Utf8JsonWriter writerCopy;

        // Get internal output to use in new writer
        IBufferWriter<byte>? output = (IBufferWriter<byte>?)_reflectionCache.OutputField.GetValue(writer);
        if (output != null)
        {
            // Create copy
            writerCopy = new Utf8JsonWriter(output, newOptions);
        }
        else
        {
            // Get internal stream to use in new writer
            Stream? stream = (Stream?)_reflectionCache.StreamField.GetValue(writer);

            // Create copy
            writerCopy = new Utf8JsonWriter(stream, newOptions);
        }

        // Copy internal state
        writer.CopyStateTo(writerCopy);

        return writerCopy;
    }

    /// <summary>
    /// Clones <see cref="Utf8JsonWriter"/> and sets <see cref="JsonWriterOptions.Indented"/> to false.
    /// </summary>
    /// <param name="writer">Source writer.</param>
    /// <returns>New copy of <see cref="Utf8JsonWriter"/>.</returns>
    public static Utf8JsonWriter CloneNotIndented(this Utf8JsonWriter writer)
    {
        JsonWriterOptions newOptions = writer.Options;
        newOptions.Indented = false;

        return Clone(writer, newOptions);
    }

    /// <summary>
    /// Clones <see cref="Utf8JsonWriter"/> and sets <see cref="JsonWriterOptions.Indented"/> to true.
    /// </summary>
    /// <param name="writer">Source writer.</param>
    /// <returns>New copy of <see cref="Utf8JsonWriter"/>.</returns>
    public static Utf8JsonWriter CloneIndented(this Utf8JsonWriter writer)
    {
        JsonWriterOptions newOptions = writer.Options;
        newOptions.Indented = true;

        return Clone(writer, newOptions);
    }

    /// <summary>
    /// Copies internal state of one writer to another.
    /// </summary>
    /// <param name="sourceWriter">Source writer.</param>
    /// <param name="targetWriter">Target writer.</param>
    public static void CopyStateTo(this Utf8JsonWriter sourceWriter, Utf8JsonWriter targetWriter)
    {
        foreach (var fieldInfo in _reflectionCache.FieldsToCopy)
        {
            fieldInfo.SetValue(targetWriter, fieldInfo.GetValue(sourceWriter));
        }

        foreach (var propertyInfo in _reflectionCache.PropertiesToCopy)
        {
            propertyInfo.SetValue(targetWriter, propertyInfo.GetValue(sourceWriter));
        }
    }

    /// <summary>
    /// Clones <see cref="JsonSerializerOptions"/>.
    /// </summary>
    /// <param name="options">Source options.</param>
    /// <returns>New instance of <see cref="JsonSerializerOptions"/>.</returns>
    public static JsonSerializerOptions Clone(this JsonSerializerOptions options)
    {
        JsonSerializerOptions serializerOptions = new JsonSerializerOptions()
        {
            AllowTrailingCommas = options.AllowTrailingCommas,
            WriteIndented = options.WriteIndented,
            PropertyNamingPolicy = options.PropertyNamingPolicy,
            DefaultBufferSize = options.DefaultBufferSize,
            DictionaryKeyPolicy = options.DictionaryKeyPolicy,
            Encoder = options.Encoder,
            IgnoreNullValues = options.IgnoreNullValues,
            IgnoreReadOnlyProperties = options.IgnoreReadOnlyProperties,
            MaxDepth = options.MaxDepth,
            PropertyNameCaseInsensitive = options.PropertyNameCaseInsensitive,
            ReadCommentHandling = options.ReadCommentHandling,
        };

        foreach (JsonConverter jsonConverter in options.Converters)
        {
            serializerOptions.Converters.Add(jsonConverter);
        }

        return serializerOptions;
    }
}

In System.Text.Json is it possible to specify custom indentation rules?

2 Answers2

Linked

Related