Json.net Async when writing to File

Question

Json.net has the async functions for converting an object to json like:

json = await JsonConvert.DeserializeObjectAsync<T>

But when I want to write an object to a json file it seems better to me to do it directly using a file Stream.

So i think it should be something like this:

 var fileStream = await file.OpenAsync(FileAccessMode.ReadWrite);

    using (IOutputStream outputStream = fileStream.GetOutputStreamAt(0))
    {
        using (StreamWriter sw = new StreamWriter(fileStream.AsStreamForWrite()))
        {
            using (JsonWriter jw = new JsonTextWriter(sw))
            {
                jw.Formatting = Formatting.Indented;

                JsonSerializer serializer = new JsonSerializer();
                serializer.Serialize(jw, obj);
            }
        }

But on the JsonSerzializer Object I can't find async methods. Also I think that IO operations shouldn't be placed in a own thread.

What is the recommended approach ?

score 16 · Accepted Answer · edited Apr 08 '21 at 15:51

16

Json.NET doesn't really support asynchronous de-/serialization. The async methods on JsonConvert are just wrappers over the synchronous methods that run them on another thread (which is exactly what a library shouldn't do).

I think the best approach here would be to run the file access code on another thread. This won't give you the full advantages of async (it will waste a thread), but it won't block the UI thread.

edited Apr 08 '21 at 15:51

Collin K

14,868
1
25
22

answered Mar 26 '13 at 21:55

svick

225,720
49
378
501

Thanks for pointing that out. I was already wondering how a stringoperation could be non threaded asynchron.. The Streaming could be made asynchron. Perhaps that would be a Feature for Json.net but obviously it isn't implented yet. So Thanks for your answer. – Boas Enkler Mar 27 '13 at 06:49
This would be a simple task to complete - making an async fork of JSON.Net. The problem though, is that it would take a long time - anyone got about 80 hours to spare? if not more? – Kind Contributor Apr 24 '15 at 12:08
There is some support in the meantime: http://james.newtonking.com/archive/2017/03/21/json-net-10-0-release-1-async-performance-documentation-and-more But since the extension API offered by JSON.NET is not at all prepared for async (`JsonConverter` etc.), I think a fork would not make much sense. I guess a different serialization library specifically built with async in mind would make more sense. And async de-serialization would require a push-style parser (event-driven) instead of a blocking pull-style parser... – Lucero Jul 24 '18 at 13:52

Manushin Igor · Answer 2 · 2018-07-24T14:22:56.617

9

See also this code, which uses right asynchronous way (e.g. it will not create huge byte arrays to avoid LOH memory allocations, it will not wait for IO operation complete).

// create this in the constructor, stream manages can be reused
// see details in this answer https://stackoverflow.com/a/42599288/185498
var streamManager = new RecyclableMemoryStreamManager();

using (var file = File.Open(destination, FileMode.Create))
{
    using (var memoryStream = streamManager.GetStream()) // RecyclableMemoryStream will be returned, it inherits MemoryStream, however prevents data allocation into the LOH
    {
        using (var writer = new StreamWriter(memoryStream))
        {
            var serializer = JsonSerializer.CreateDefault();

            serializer.Serialize(writer, data);

            await writer.FlushAsync().ConfigureAwait(false);

            memoryStream.Seek(0, SeekOrigin.Begin);

            await memoryStream.CopyToAsync(file).ConfigureAwait(false);
        }
    }

    await file.FlushAsync().ConfigureAwait(false);
}

Whole file: https://github.com/imanushin/AsyncIOComparison/blob/0e2527d5c00c2465e8fd2617ed8bcb1abb529436/IntermediateData/FileNames.cs

edited Jul 24 '18 at 14:22

answered Mar 09 '16 at 08:36

Manushin Igor

3,040
20
34

3

IMHO, this answer is better than the accepted answer, considering the accepted answer completely ignored the fact the user was attempting to perform IO and offers no real example. – James Haug Mar 22 '17 at 15:31
3

You pretend that your code "will not create huge byte arrays to avoid LOH memory allocations", but actually the `MemoryStream` internally does just that, it caches everything in a single byte array which is grown when needed. – Lucero Jul 24 '18 at 13:45
@Lucero, absolutely agree, we should use Microsoft.IO.RecyclableMemoryStream, will fix – Manushin Igor Jul 24 '18 at 14:17
3

@ManushinIgor Okay, that's better - even if it still requires the full data to be cached in memory. Note however that the link to the "Whole file" still has the old code. – Lucero Jul 24 '18 at 17:26
@Lucero, agree will fix on github a bit later – Manushin Igor Jul 25 '18 at 14:08

Stefan Steiger · Answer 3 · 2021-09-28T12:18:01.660

You cannot do that with JSON.NET/Newtonsoft.JSON.
However, you can instead use System.Text.Json.
To get the same behaviour as JSON.NET, just set IncludeFields and PropertyNameCaseInsensitive to true.

public static class JsonExtensions
{
    private static readonly System.Text.Json.JsonSerializerOptions _jsonOptions = 
        new System.Text.Json.JsonSerializerOptions
        {
            PropertyNameCaseInsensitive = true,
            IncludeFields = true
        }
    ;


    public static string ToJson<T>(this T obj)
    {
        return System.Text.Json.JsonSerializer.Serialize<T>(obj, _jsonOptions);
    }


    public static T FromJson<T>(this string json)
    {
        return System.Text.Json.JsonSerializer.Deserialize<T>(json, _jsonOptions);
    }


    public static async System.Threading.Tasks.Task ToJsonAsync<T>(this T obj, System.IO.Stream stream)
    {
        await System.Text.Json.JsonSerializer.SerializeAsync(stream, obj, typeof(T), _jsonOptions);
    }


    public static async System.Threading.Tasks.Task<T> FromJsonAsync<T>(this System.IO.Stream stream)
    {
        return await System.Text.Json.JsonSerializer.DeserializeAsync<T>(stream, _jsonOptions);
    }

}

and there you go.

Also, if you asynchronously want to serialize to string:

public static async System.Threading.Tasks.Task<string> ToJsonAsync<T>(this T obj)
{
    string ret = null;

    Microsoft.IO.RecyclableMemoryStreamManager streamManager = 
        new Microsoft.IO.RecyclableMemoryStreamManager();
    
    using (System.IO.MemoryStream ms = streamManager.GetStream())
    {
        await System.Text.Json.JsonSerializer.SerializeAsync(ms, obj, typeof(T), _jsonOptions);
        ms.Position = 0;

        using (System.IO.TextReader sr = new System.IO.StreamReader(ms, System.Text.Encoding.UTF8))
        {
            ret = await sr.ReadToEndAsync();
        }

    }

    return ret;
}

Json.net Async when writing to File

3 Answers3