#StackBounty: #c# #json #dynamic-programming Patch a JSON object using dynamic / ExpandoObject with System.Text.Json

Bounty: 50

Background

Recently, I was making some updates to an "older" library that would handle PATCH-style modifications to an object that is persisted in a JSON format on our document-storage databases (e.g., CosmosDB).

I took a fresh approach of this, and started on a blank slate and decided to make use of the DynamicObjectConverter which was introduced back in late 2020 to the System.Text.Json library.

The goal is to handle a PATCH operation to an existing JSON object.

For example from an existing JSON document with:

{
  "id": "e001",
  "name": "foo"
}

with a patch operation of

{ "name": "bar" }

the result:

{
  "id": "e001",
  "name": "bar"
}

I also wanted to be able to handle adding additional new properties to a collection of existing JSON documents (as a sweeping task) making patch updates across various documents that all have different schemas (hail schema-less DBs!). Such as adding a metadata, or isHidden property to all JSON documents.

The Extension Class

There are 5 methods to the extension class.

This question contains the complete class, you just need to put these 5 methods into a single static class.

DynamicUpdate() Method

The main extension method that extends the IDictionary<string, object type (which is commonly found in our code as ExpandoObject type implementation)

internal static JsonElement DynamicUpdate(
    this IDictionary<string, object> entity,
    JsonDocument doc,
    bool addPropertyIfNotExists = false,
    bool useTypeValidation = true,
    JsonDocumentOptions options = default)
{
    if (doc == null) throw new ArgumentNullException(nameof(doc));
    if (doc.RootElement.ValueKind != JsonValueKind.Object) 
        throw new NotSupportedException("Only objects are supported.");

    foreach (JsonProperty jsonProperty in doc.RootElement.EnumerateObject())
    {
        string propertyName = jsonProperty.Name;
        JsonElement newElement = doc.RootElement.GetProperty(propertyName);
        bool hasProperty = entity.TryGetValue(propertyName, out object oldValue);

        // sanity checks
        JsonElement? oldElement = null;
        if (oldValue != null)
        {
            if (!oldValue.GetType().IsAssignableTo(typeof(JsonElement))) 
                throw new ArgumentException($"Type mismatch. Must be {nameof(JsonElement)}.", nameof(entity));
            oldElement = (JsonElement)oldValue;
        }
        if (!hasProperty && !addPropertyIfNotExists) continue;
        entity[propertyName] = GetNewValue(
            oldElement, newElement, propertyName, 
            addPropertyIfNotExists, useTypeValidation, options);
    }
    using JsonDocument finalDoc = JsonDocument.Parse(JsonSerializer.Serialize(entity));
    return finalDoc.RootElement.Clone();
}
GetNewValue() Method

This method gets the value (recursively for object properties) and also deals with validation based on the passed in options as arguments.

private static JsonElement GetNewValue(
    JsonElement? oldElementNullable, 
    JsonElement newElement, 
    string propertyName,
    bool addPropertyIfNotExists,
    bool useTypeValidation,
    JsonDocumentOptions options)
{
    if (oldElementNullable == null) return newElement.Clone();
    JsonElement oldElement = (JsonElement)oldElementNullable;

    // type validation
    if (useTypeValidation && !IsValidType(oldElement, newElement)) 
        throw new ArgumentException($"Type mismatch. The property '{propertyName}' must be of type '{oldElement.ValueKind}'.", nameof(newElement));

    // recursively go down the tree for objects
    if (oldElement.ValueKind == JsonValueKind.Object)
    {
        string oldJson = oldElement.GetRawText();
        string newJson = newElement.ToString();
        IDictionary<string, object> entity = JsonSerializer.Deserialize<ExpandoObject>(oldJson);
        return DynamicUpdate(entity, newJson, addPropertyIfNotExists, useTypeValidation, options);
    }

    return newElement.Clone();
}
IsValidType() Method

This method handles the validation for types. (i.e. trying to replace a string with an int will return false.

private static bool IsValidType(JsonElement oldElement, JsonElement newElement)
{
    if (newElement.ValueKind == JsonValueKind.Null) return true;
    
    // 'true' --> 'false'
    if (oldElement.ValueKind == JsonValueKind.True && newElement.ValueKind == JsonValueKind.False) return true;
    // 'false' --> 'true'
    if (oldElement.ValueKind == JsonValueKind.False && newElement.ValueKind == JsonValueKind.True) return true;
    
    // type validation
    return (oldElement.ValueKind == newElement.ValueKind);
}
IsValidJsonPropertyName() Method

This method just a quick way to make sure there isn’t a totally malformed property name.

private static bool IsValidJsonPropertyName(string value)
{
    if (string.IsNullOrEmpty(value)) return false;

    // this is validation for our specific use case (C#)
    // note that the official docs don't prohibit this though.
    // https://datatracker.ietf.org/doc/html/rfc7159
    for (int i = 0; i < value.Length; i++)
    {
        if (char.IsLetterOrDigit(value[i])) continue;
        switch (value[i])
        {
            case '-':
            case '_':
            default:
                break;
        }
    }

    return true;
}
Overloaded DynamicUpdate() Method

An overloaded method so that passing in a JSON string is also possible for tests, etc.

internal static JsonElement DynamicUpdate(
    this IDictionary<string,
    object> entity,
    string patchJson,
    bool addPropertyIfNotExists = false,
    bool useTypeValidation = true,
    JsonDocumentOptions options = default)
{
    using JsonDocument doc = JsonDocument.Parse(patchJson, options);
    return DynamicUpdate(entity, doc, addPropertyIfNotExists, useTypeValidation, options);
}

How to use it

Here is a sample snippet to test the extension method.

string original = @"{""foo"":[1,2,3],""parent"":{""childInt"":1},""bar"":""example""}"; 
string patch    = @"{""foo"":[9,8,7],""parent"":{""childInt"":9,""childString"":""woot!""},""bar"":null}";
Console.WriteLine(original);

// change this value to see the different types of patching method
bool addPropertyIfNotExists = false; 

ExpandoObject expandoObject = JsonSerializer.Deserialize<ExpandoObject>(original);

// patch it!
expandoObject.DynamicUpdate(patch, addPropertyIfNotExists);
Console.WriteLine(JsonSerializer.Serialize(expandoObject));

Note: For the example above, the JSON is a string, but in practice reading in the document comes as some form of a UTF8 binary stream, which is where the JsonDocument shines.

Important Note: Keep in mind that property names are case sensitive, so foo and FoO are unique and valid property names. It would be trivial to add a method to support ignoring case, but in my use-case this is the desired use.

Question for code review

It would be interesting to know if there are any better design patterns that can minimize the back-and-fourth of serializing the inner objects and materializing it as a JsonElement that happens in recursive section of the code found in the GetNewValue() method.

For a large object nested JSON object, there is a lot of packing up and cloning of the JsonElement struct. It’s quite uncommon to have very "deep" properties in the wild, but I can’t help but wonder if there is a smarter approach to this.

Of course, any other feedback is welcome — always looking to improve the code!


Get this bounty!!!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.