#StackBounty: #json #aws-lambda #jsonpath #aws-step-functions #aws-event-bridge How to extract properties from my state machine output …

Bounty: 100

I’m having trouble passing one of the values returned by my AWS state machine as input to a Lambda using the EventBridge service.

I created a state machine in AWS Step Functions to model a specific problem in our domain. Once the state machine finishes, I want to perform another operation from inside one of my Lambdas. To make that work, I created a new rule using EventBridge: whenever the state machine finishes, it triggers my lambda with a specific Json input.

My problem is in how to extract properties from the state machine output and pass them as properties of the lambda input.

Say my state machine returns this:

{
    "usefulObject":{
        "usefulProperty":"value"
    },
    "anotherProperty":"anotherValue"
}

I want to receive the following payload in my lambda:

{
    "property":"value"
}

Initially, I thought I would be able to do this using the "Input Transformation" option on the EventBridge rule, something like:

  • InputTransformer Path:
{"propertyValue":"$.usefulObject.usefulProperty"}
  • InputTransformer Template:
{"property":<propertyValue>}

However, during testing, I realized that the event payload contains a lot more data than my state machine output. In reality, the state machine output is wrapped in this "event container" object, something like this:

{
    "version": "0",
    "id": "...",
    "detail-type": "Step Functions Execution Status Change",
    "source": "aws.states",
    "account": "...",
    "time": "2020-11-10T13:59:57Z",
    "region": "us-east-1",
    "resources": [
        "...myStateMachineArn..."
    ],
    "detail": {
        "executionArn": "...myStateMachineExecutionArn...",
        "stateMachineArn": "...myStateMachineArn...",
        "name": "ff72036a-2917-c657-80e7-2589b7b76d59",
        "status": "SUCCEEDED",
        "startDate": 1605016794597,
        "stopDate": 1605016797936,
        "input": "{n  "usefulObject":{n    "usefulProperty": "value"n  },n  "anotherProperty": "anotherValue"n}",
        "inputDetails": {
            "included": true
        },
        "output": "{n  "usefulObject":{n    "usefulProperty": "value"n  },n  "anotherProperty": "anotherValue"n}",
        "outputDetails": {
            "included": true
        }
    }
}

As you can see, in the actual event payload, my statemachine data is stored as a stringified Json value inside the output node. If I then change my Input Transformation path to something like:

{"propertyValue":"$.detail.output.usefulObject.usefulProperty"}

I get an empty result for property in the transformed input. Turns out, JsonPath is unable to traverse the stringified value as part of the Json payload and will fail on the search.

How can I extract the usefulProperty value out of that Json string in the event payload so that I can pass it to my lambda function? Is there a way to do it using purely JsonPath that I’m missing? Perhaps there is a way to configure AWS to not convert the payload into a string and just make it part of the whole event payload? Any other alternatives?


Get this bounty!!!

#StackBounty: #json #flutter #api #dart #flutter-layout Unable to send the intercepted sms using REST API in flutter dart

Bounty: 50

Problem:
I am trying to intercept the SMS message body and then send the message body to the database using a POST Call REST API every time I encounter an SMS. This whole interception and sending the message body should work in the background as well and that too automatically.

What I have achieved so far:
I am using the telephony plugin to intercept the message body and I am able to print the message body every time I receive one at the UI level but unable to call the API and send the SMS body.

Error:
Since I did not find a way to how to automatically call the API every time a new message is intercepted so I used a button to call it instead but even that did not work and it throws the error as

[error:flutter/lib/ui/ui_dart_state.cc(209)] unhandled exception: invalid argument(s) (onerror): the error handler of future.catcherror must return a value of the future's type

Also, I did not manage how do I intercept the SMS message body in the background.

To better understand this error I will be attaching a few of my code snippets:

API usage function:

String body = "";
  DateTime currentPhoneDate = DateTime.now();
  final telephony = Telephony.instance;
  interceptMessage() {
    final messaging = ApiService();
    messaging.interceptedMessage({
      "id": 50,
      "body": "$body",
      "senderName": "IDK",
      "timeStamp": "2021-10-02 12:00:55"
    })
      ..then((value) {
        if (value.status == "Success") {
          print('Message Intercepted');
        } else {
          print('Somethig went wrong');
        }
      });
  }

API Class:

Future<SmsResponse> interceptedMessage(dynamic param) async {
    var client = http.Client();

    String? token = await storage.readSecureToken('key');
    if (token == null) {
      throw Exception("No token stored in storage");
    }
    try {
      var response = await client
          .post(
            Uri.https("baseURL", "endpoint"),
            headers: <String, String>{
              'Authorization': 'Token $token',
            },
            body: param,
          )
          .timeout(Duration(seconds: TIME_CONST))
          .catchError(handleError);
      if (response.statusCode == 200) {
        print('Response Body: ${response.body}');
        final data = await jsonDecode(response.body);
        return SmsResponse.fromJson(data);
      } else if (response.statusCode == 401) {
        print("Unauthorized Request");
        return param;
      } else {
        print("Bad Input");
        return param;
      }
    } catch(e){
      print(e);
   }
  }

Telephony Plugin Usage:

 @override
  void initState() {
    super.initState();
    initPlatformState();
  }

  onMessage(
    SmsMessage message,
  ) async {
    setState(() {
      body = message.body ?? "Error reading message body.";
      print("$body");
    });
  }

  onSendStatus(SendStatus status) {
    setState(() {
      body = status == SendStatus.SENT ? "sent" : "delivered";
    });
  }

  Future<void> initPlatformState() async {
    final bool? result = await telephony.requestPhoneAndSmsPermissions;

    if (result != null && result) {
      telephony.listenIncomingSms(
        onNewMessage: onMessage,
        onBackgroundMessage: onBackgroundMessage,
        listenInBackground: true,
      );
    }
    if (!mounted) return;
  }

Handle Error Function

void handleError(error) {
    //hideLoading();
    if (error is BadRequestException) {
      var message = error.message;
      DialogHelper.showErroDialog(description: message);
    } else if (error is FetchDataException) {
      var message = error.message;
      DialogHelper.showErroDialog(description: message);
    } else if (error is ApiNotRespondingException) {
      DialogHelper.showErroDialog(
          description: 'Oops! It took longer to respond.');
    } else if (error is SocketException) {
      print(
          error); //Have to remove this part this is already being handled at the service level
    } else {
      print("All OK");
    }
  }

UI Level:

Text("$body");


Get this bounty!!!

#StackBounty: #json #scala #playframework #traversal How to parse an array of json in scala play framework?

Bounty: 50

I have an array of json objects like this

[
 {"events":[
            {
              "type":"message","attributes":[
                   {"key":"action","value":"withdraw_reward"}, 
                   {"key":"sender","value":"bob"}, 
                   {"key":"module","value":"distribution"}, 
                   {"key":"sender","value":"bob"}
            ]},
           {
             "type":"credit","attributes":[
                  {"key":"recipient","value":"ross"},
                  {"key":"sender","value":"bob"},
                  {"key":"amount","value":"100"}
           ]},
           {
             "type":"rewards","attributes":[
                  {"key":"amount","value":"100"}, 
                  {"key":"validator","value":"sara"}
           ]}
        ]
 },
   {"events":[
            {
              "type":"message","attributes":[
                        {"key":"action","value":"withdraw_reward"}, 
                   {"key":"sender","value":"bob"}, 
                   {"key":"module","value":"distribution"}, 
                   {"key":"sender","value":"bob"}
            ]},
           {
             "type":"credit","attributes":[
                  {"key":"recipient","value":"ross"},
                  {"key":"sender","value":"bob"},
                  {"key":"amount","value":"100"}
           ]},
           {
             "type":"rewards","attributes":[
                  {"key":"amount","value":"200"}, 
                  {"key":"validator","value":"Ryan"}
           ]}
        ]
 }
]

How to traverse through the types, check if it’s type "rewards" and then go through the attributes and verify if the validator is Sarah and fetch the value of the key amount? Pretty new to scala and play framework. Any help would be great. Thanks


Get this bounty!!!

#StackBounty: #reactjs #json #react-hooks #use-state updating object inside array inside object using prevState and the useState hook

Bounty: 50

I’d like to remove a nested object based on the id is equal to a passed prop. At the moment, the entire object is replaced. I’m missing something, when trying to update the state using useState probably with the way I’m looping my object?

UPDATE: The question was closed in response to available answers for updating nested objects. This question involves arrays which I believe are part of the issue at hand. Please note the difference in nature in this question with the forEach. Perhaps a return statement is required, or a different approach to the filtering on id..

my initial object looks like this:

{
  "some_id1": [
    {
      "id": 93979,
      // MORE STUFF
   
    },
    {
      "id": 93978,
      // MORE STUFF
    }
  ],
  "some_id2": [
    {
      "id": 93961,
      // MORE STUFF
    },
    {
      "id": 93960,
      // MORE STUFF
    }
  ]
}

and I go through each item as such:

for (const key in items) {
        if (Object.hasOwnProperty.call(items, key)) {
            const element = items[key];
            element.forEach(x => {
                if (x.id === singleItem.id) {
                    setItems(prevState => ({
                        ...prevState,
                        [key]: {
                            ...prevState[key],
                            [x.id]: undefined
                        }
                    }))
                }
            })
        }


Get this bounty!!!

#StackBounty: #c# #json #dynamic-programming Patch a JSON object using dynamic / ExpandoObject with System.Text.Json

Bounty: 50

Background

Recently, I was making some updates to an "older" library that would handle PATCH-style modifications to an object that is persisted in a JSON format on our document-storage databases (e.g., CosmosDB).

I took a fresh approach of this, and started on a blank slate and decided to make use of the DynamicObjectConverter which was introduced back in late 2020 to the System.Text.Json library.

The goal is to handle a PATCH operation to an existing JSON object.

For example from an existing JSON document with:

{
  "id": "e001",
  "name": "foo"
}

with a patch operation of

{ "name": "bar" }

the result:

{
  "id": "e001",
  "name": "bar"
}

I also wanted to be able to handle adding additional new properties to a collection of existing JSON documents (as a sweeping task) making patch updates across various documents that all have different schemas (hail schema-less DBs!). Such as adding a metadata, or isHidden property to all JSON documents.

The Extension Class

There are 5 methods to the extension class.

This question contains the complete class, you just need to put these 5 methods into a single static class.

DynamicUpdate() Method

The main extension method that extends the IDictionary<string, object type (which is commonly found in our code as ExpandoObject type implementation)

internal static JsonElement DynamicUpdate(
    this IDictionary<string, object> entity,
    JsonDocument doc,
    bool addPropertyIfNotExists = false,
    bool useTypeValidation = true,
    JsonDocumentOptions options = default)
{
    if (doc == null) throw new ArgumentNullException(nameof(doc));
    if (doc.RootElement.ValueKind != JsonValueKind.Object) 
        throw new NotSupportedException("Only objects are supported.");

    foreach (JsonProperty jsonProperty in doc.RootElement.EnumerateObject())
    {
        string propertyName = jsonProperty.Name;
        JsonElement newElement = doc.RootElement.GetProperty(propertyName);
        bool hasProperty = entity.TryGetValue(propertyName, out object oldValue);

        // sanity checks
        JsonElement? oldElement = null;
        if (oldValue != null)
        {
            if (!oldValue.GetType().IsAssignableTo(typeof(JsonElement))) 
                throw new ArgumentException($"Type mismatch. Must be {nameof(JsonElement)}.", nameof(entity));
            oldElement = (JsonElement)oldValue;
        }
        if (!hasProperty && !addPropertyIfNotExists) continue;
        entity[propertyName] = GetNewValue(
            oldElement, newElement, propertyName, 
            addPropertyIfNotExists, useTypeValidation, options);
    }
    using JsonDocument finalDoc = JsonDocument.Parse(JsonSerializer.Serialize(entity));
    return finalDoc.RootElement.Clone();
}
GetNewValue() Method

This method gets the value (recursively for object properties) and also deals with validation based on the passed in options as arguments.

private static JsonElement GetNewValue(
    JsonElement? oldElementNullable, 
    JsonElement newElement, 
    string propertyName,
    bool addPropertyIfNotExists,
    bool useTypeValidation,
    JsonDocumentOptions options)
{
    if (oldElementNullable == null) return newElement.Clone();
    JsonElement oldElement = (JsonElement)oldElementNullable;

    // type validation
    if (useTypeValidation && !IsValidType(oldElement, newElement)) 
        throw new ArgumentException($"Type mismatch. The property '{propertyName}' must be of type '{oldElement.ValueKind}'.", nameof(newElement));

    // recursively go down the tree for objects
    if (oldElement.ValueKind == JsonValueKind.Object)
    {
        string oldJson = oldElement.GetRawText();
        string newJson = newElement.ToString();
        IDictionary<string, object> entity = JsonSerializer.Deserialize<ExpandoObject>(oldJson);
        return DynamicUpdate(entity, newJson, addPropertyIfNotExists, useTypeValidation, options);
    }

    return newElement.Clone();
}
IsValidType() Method

This method handles the validation for types. (i.e. trying to replace a string with an int will return false.

private static bool IsValidType(JsonElement oldElement, JsonElement newElement)
{
    if (newElement.ValueKind == JsonValueKind.Null) return true;
    
    // 'true' --> 'false'
    if (oldElement.ValueKind == JsonValueKind.True && newElement.ValueKind == JsonValueKind.False) return true;
    // 'false' --> 'true'
    if (oldElement.ValueKind == JsonValueKind.False && newElement.ValueKind == JsonValueKind.True) return true;
    
    // type validation
    return (oldElement.ValueKind == newElement.ValueKind);
}
IsValidJsonPropertyName() Method

This method just a quick way to make sure there isn’t a totally malformed property name.

private static bool IsValidJsonPropertyName(string value)
{
    if (string.IsNullOrEmpty(value)) return false;

    // this is validation for our specific use case (C#)
    // note that the official docs don't prohibit this though.
    // https://datatracker.ietf.org/doc/html/rfc7159
    for (int i = 0; i < value.Length; i++)
    {
        if (char.IsLetterOrDigit(value[i])) continue;
        switch (value[i])
        {
            case '-':
            case '_':
            default:
                break;
        }
    }

    return true;
}
Overloaded DynamicUpdate() Method

An overloaded method so that passing in a JSON string is also possible for tests, etc.

internal static JsonElement DynamicUpdate(
    this IDictionary<string,
    object> entity,
    string patchJson,
    bool addPropertyIfNotExists = false,
    bool useTypeValidation = true,
    JsonDocumentOptions options = default)
{
    using JsonDocument doc = JsonDocument.Parse(patchJson, options);
    return DynamicUpdate(entity, doc, addPropertyIfNotExists, useTypeValidation, options);
}

How to use it

Here is a sample snippet to test the extension method.

string original = @"{""foo"":[1,2,3],""parent"":{""childInt"":1},""bar"":""example""}"; 
string patch    = @"{""foo"":[9,8,7],""parent"":{""childInt"":9,""childString"":""woot!""},""bar"":null}";
Console.WriteLine(original);

// change this value to see the different types of patching method
bool addPropertyIfNotExists = false; 

ExpandoObject expandoObject = JsonSerializer.Deserialize<ExpandoObject>(original);

// patch it!
expandoObject.DynamicUpdate(patch, addPropertyIfNotExists);
Console.WriteLine(JsonSerializer.Serialize(expandoObject));

Note: For the example above, the JSON is a string, but in practice reading in the document comes as some form of a UTF8 binary stream, which is where the JsonDocument shines.

Important Note: Keep in mind that property names are case sensitive, so foo and FoO are unique and valid property names. It would be trivial to add a method to support ignoring case, but in my use-case this is the desired use.

Question for code review

It would be interesting to know if there are any better design patterns that can minimize the back-and-fourth of serializing the inner objects and materializing it as a JsonElement that happens in recursive section of the code found in the GetNewValue() method.

For a large object nested JSON object, there is a lot of packing up and cloning of the JsonElement struct. It’s quite uncommon to have very "deep" properties in the wild, but I can’t help but wonder if there is a smarter approach to this.

Of course, any other feedback is welcome — always looking to improve the code!


Get this bounty!!!

#StackBounty: #postgresql #json #graph How to turn a complicated graph into Json Graph Format?

Bounty: 50

So having such normalized Postgres 13/14 graph of item joins:

CREATE TABLE items (
  item_id     serial PRIMARY KEY,
  title text
);
CREATE TABLE joins (
  id          serial PRIMARY KEY,
  item_id     int,
  child_id    int
);
INSERT INTO items (item_id,title) VALUES
  (1,'PARENT'),
  (2,'LEVEL 2'),
  (3,'LEVEL 3.1'),
  (4,'LEVEL 4.1'),
  (5,'LEVEL 4.2'),
  (6,'LEVEL 3.2');
INSERT INTO joins (item_id, child_id) VALUES
  (1,2),
  (2,3),
  (3,2),
  (3,5),
  (2,6);

How to turn it into a JSON Graph Format document containing item columns as fields?


Get this bounty!!!

#StackBounty: #node.js #json #typescript #express router handler returns an array of object but client doesn't get them in json tho…

Bounty: 50

I am implementing a express.js project with Typescript.

I have defined a enum and a interface :

export enum ProductType {
    FOOD = 'food',
    CLOTH = 'cloth',
    TOOL = 'tool'
}

export interface MyProduct {
    type: ProductType;
    info: {
        price: number;
        date: Date;
    };
}

One of my router handler needs to return an array of MyProduct to client. I tried this :

const productArr: MyProduct[] = // call another service returns an array of MyProduct
app.get('/products', (req, res) => {
    res.status(200).send({products: productArr});
});

I use Postman tested this endpoint, it responses with status 200 but with a default HTML page instead of the array of objects in JSON.

enter image description here

What do I miss? Is it because express.js can’t automatically parse the enum and interface to json object??

P.S. I have set up json parser, so it is not about that, other endpoints work fine with json response:

const app = express();
app.use(express.json());
...


Get this bounty!!!

#StackBounty: #parsing #node.js #json #typescript JSON4 parser in Typescript

Bounty: 50

Context

I may have accidentally gotten a little sidetracked during homeschooling, and wrote a JSON4 parser in Typescript during the down-time. This project started as an idea that I got while discussing something actually school-related and over the past few days grew into this.

I have no formal education on how compilers/parser/lexers work, except for the few hours I wasted on the internet reading up on the topic. This makes me especially curious if this implementation is somewhat correct – in the grand scheme of things.

I have dedicated myself to make it as spec-compliant as I deemed reasonable. I used this specification as a reference

In the GIT repository I pulled in a "JSON test suite", which I used to run tests against the code. This helped me fix several bugs, but I think there are still lots of edge-cases that are not handled.

To help anyone kind enough to review my code, I’ve published this thing in my "pet project" github repo, with everything already set up, so you just have to clone that and install using yarn.

Question

I am mainly interested in the technical implementation of the code. I tried to focus on readability, how did that work out?

  • How readable/understandable is the code?
  • How is the design/architecture of the code?
  • Are there any best-practices or techniques I could implement?

I know this is a lot of code, but there is no need to review all of it!

This thing works as is, the only thing that is still a little wanky is the string escaping.

Code

src/Json.ts

import { JsonValue } from './lib/types';

import { parse } from './lib/parser';
import { tokenize } from "./lib/lexer";
import { convertNodeToJsValue } from './lib/generator';

export class Json {
  static parse(source: string): JsonValue {
    const tokens = tokenize(source);
    const ast = parse(tokens);
    const jsValue = convertNodeToJsValue(ast);

    return jsValue;
  }
}

src/lib/util/JsonError.ts

import { Token } from "../Token";

function serializeTokens(tokens: Token[]) {
    return tokens.map((token) => {
        if (token.isString) {
            return `"${token.value}"`;
        }

        return token.value;
    }).join("");
}

function formatMessage(message: string, tokens: Token[]): string {
    const source = serializeTokens(tokens);

    return `${message}nnSource at the point of the Error:n${source}n^`;
}

export class JsonError extends Error {
    constructor(message: string, tokens: Token[]) {
        super(formatMessage(message, tokens));
    }
}

src/lib/types.ts

export type JsonScalar = boolean | number | string | null;

export type JsonObject = { [key: string]: JsonValue };

export type JsonArray = JsonValue[];

export type JsonValue = JsonObject | JsonArray | JsonScalar;

src/lib/Token.ts

type TokenType = "punctuation" | "boolean" | "string" | "number" | "null";

export class Token {
  public readonly type: TokenType;
  public readonly value: string;

  public constructor(type: TokenType, value: string) {
    this.type = type;
    this.value = value;
  }

  public get isString(): boolean {
    return this.type === "string";
  }

  public get isNumber(): boolean {
    return this.type === "number";
  }

  public get isBoolean(): boolean {
    return this.type === "boolean";
  }

  public get isNull(): boolean {
    return this.type === "null";
  }

  public get isScalar(): boolean {
    return this.isNull || this.isString || this.isNumber || this.isBoolean;
  }

  public get isPunctuation(): boolean {
    return this.type === "punctuation";
  }

  public get isArrayOpen(): boolean {
    return isPredefinedPunctuation("arrayOpen", this);
  }

  public get isArrayClose(): boolean {
    return isPredefinedPunctuation("arrayClose", this);
  }

  public get isObjectOpen(): boolean {
    return isPredefinedPunctuation("objectOpen", this);
  }

  public get isObjectClose(): boolean {
    return isPredefinedPunctuation("objectClose", this);
  }

  public get isComma(): boolean {
    return isPredefinedPunctuation("comma", this);
  }

  public get isColon(): boolean {
    return isPredefinedPunctuation("colon", this);
  }
}

export const PUNCTUATION_TOKENS = {
  comma: new Token("punctuation", ","),
  colon: new Token("punctuation", ":"),
  arrayOpen: new Token("punctuation", "["),
  arrayClose: new Token("punctuation", "]"),
  objectOpen: new Token("punctuation", "{"),
  objectClose: new Token("punctuation", "}"),
};

function isPredefinedPunctuation(
  key: keyof typeof PUNCTUATION_TOKENS,
  token: Token,
): boolean {
  return token.isPunctuation && PUNCTUATION_TOKENS[key].value === token.value;
}

src/lib/lexer.ts

import { PUNCTUATION_TOKENS, Token } from "./Token";

type TokenizerResult =
  | { matched: false; cursor?: number }
  | { matched: true; token: Token; cursor: number };

type Tokenizer = (source: string, cursor: number) => TokenizerResult;

export function tokenize(source: string): Token[] {
  let cursor = 0;
  const tokens: Token[] = [];

  const tokenizers: Tokenizer[] = [
    tokenizeWhitespace,
    tokenizePunctuation,
    tokenizeNull,
    tokenizeNumber,
    tokenizeString,
    tokenizeBoolean,
  ];

  while (cursor < source.length) {
    let didMatch = false;

    for (const tokenizer of tokenizers) {
      const result = tokenizer(source, cursor);

      if (result.matched) {
        didMatch = true;
        cursor = result.cursor;
        tokens.push(result.token);

        break;
      }

      // Even if a tokenizer did not match, it is free to move the cursor
      // this is usefull for example when parsing white-space, which does not result in a token
      if (!result.matched && typeof result.cursor === "number") {
        didMatch = true;
        cursor = result.cursor;

        break;
      }
    }

    if (!didMatch) {
      throw new Error(`Could not lex token: "${source.substr(cursor)}"`);
    }
  }

  return tokens;
}

function tokenizeWhitespace(source: string, cursor: number): TokenizerResult {
  const result = matchRegex(source, cursor, /^(?!f)s/);

  if (result.matched) {
    return {
      matched: false,
      cursor: result.cursor,
    };
  }

  return {
    matched: false,
  };
}

function tokenizeNull(source: string, cursor: number): TokenizerResult {
  return matchStaticToken(source, cursor, new Token("null", "null"));
}

function tokenizeBoolean(source: string, cursor: number): TokenizerResult {
  const booleanTokens = [new Token("boolean", "true"), new Token("boolean", "false")];

  for (const token of booleanTokens) {
    const result = matchStaticToken(source, cursor, token);

    if (result.matched) {
      return result;
    }
  }

  return {
    matched: false,
  };
}

function tokenizeNumber(source: string, cursor: number): TokenizerResult {
  const result = matchRegex(
    source,
    cursor,
    /^(?:(?!0d)(?!-0d)-?(?:0(?:.d+)|d+(?:d+)?(?:.d+)?)(?:(?:e|E)(?:-?|+?)d+)?)/,
  );

  if (result.matched) {
    return {
      matched: true,
      cursor: result.cursor,
      token: new Token("number", result.value),
    };
  }

  return {
    matched: false,
  };
}

function tokenizeString(source: string, cursor: number): TokenizerResult {
  const result = matchRegex(source, cursor, /^"(?:[^n"\]|\.)*"/);

  if (result.matched) {
    let value = result.value;

    if (
      // Check if we even have a value, dont run the following check if we dont
      value != null &&
      value.length > 0 &&
      // Check if the first or last character is a quotation mark, in which case we would need to remove them
      (value[0] === '"' || value[value.length - 1] === '"') &&
      // Ensure we do not remove quotation marks on a string with no content ("")
      !(value.length === 2 && value[0] === '"' && value[value.length - 1] === '"')
    ) {
      value = value.replace(/^"/, "").replace(/"$/, "");
    }

    return {
      matched: true,
      cursor: result.cursor,
      token: new Token("string", value),
    };
  }

  return {
    matched: false,
  };
}

function tokenizePunctuation(source: string, cursor: number): TokenizerResult {
  for (const token of Object.values(PUNCTUATION_TOKENS)) {
    const matchResult = matchLiteral(source, cursor, token.value);

    if (matchResult.matched) {
      return {
        matched: true,
        token,
        cursor: matchResult.cursor,
      };
    }
  }

  return {
    matched: false,
  };
}

/*
 * Helper Functions
 */

function matchRegex(
  source: string,
  cursor: number,
  regex: RegExp,
): { matched: false } | { matched: true; value: string; cursor: number } {
  const currentSource = source.substr(cursor);
  const match = currentSource.match(regex);

  if (!match || match.length === 0) {
    return {
      matched: false,
    };
  }

  if (match.length === 1) {
    const value = match[0];

    return {
      value,
      matched: true,
      cursor: cursor + value.toString().length,
    };
  }

  // We should not get here, regex passed in here should be designed
  // as such that there is only one Group match. (Use non-capture) groups.
  // Therefore it is most likely a programmer's error if we get here.
  throw new Error("Invalid regex matches");
}

function matchLiteral(
  source: string,
  cursor: number,
  token: string,
): { matched: false } | { matched: true; cursor: number } {
  if (source.substr(cursor, token.length) === token) {
    return {
      matched: true,
      cursor: cursor + token.length,
    };
  }

  return {
    matched: false,
  };
}

function matchStaticToken(source: string, cursor: number, token: Token): TokenizerResult {
  const lookahead = matchLiteral(source, cursor, token.value);

  if (lookahead.matched) {
    return {
      matched: true,
      cursor: lookahead.cursor,
      token,
    };
  }

  return {
    matched: false,
  };
}

src/lib/parser.ts

import { Token } from "./Token";
import { Node, ScalarNode, ObjectNode, ArrayNode } from "./nodes";
import { JsonError } from "./util/JsonError";

export function parse(tokens: Token[]): Node {
  const rootNode = parseSingle(tokens);

  if (tokens.length > 0) {
    throw new JsonError("Unexpected tokens at the end of source", tokens);
  }

  return rootNode;
}

export function parseSingle(tokens: Token[]): Node {
  if (tokens.length === 0) {
    throw new JsonError("Unexpected end of source!", tokens);
  }

  const initialToken = tokens[0];

  if (initialToken.isScalar) {
    return parseScalar(tokens);
  }

  if (initialToken.isObjectOpen) {
    return parseObject(tokens);
  }

  if (initialToken.isArrayOpen) {
    return parseArray(tokens);
  }

  throw new JsonError(
    `Could not parse token of type '${initialToken.type}' at this location`,
    tokens,
  );
}

function parseScalar(tokens: Token[]): ScalarNode {
  const { type, value } = tokens[0];

  if (type === "string") {
    validateString(value, tokens);
  }

  const scalar = new ScalarNode(type, value);

  tokens.shift();

  return scalar;
}

function parseArray(tokens: Token[]): ArrayNode {
  const arrayNode = new ArrayNode();

  // Removes the opening "[" token.
  tokens.shift();

  const firstToken = tokens[0];

  // Empty Array, exit early.
  if (firstToken && firstToken.isArrayClose) {
    tokens.shift();

    return arrayNode;
  }

  while (tokens.length > 0) {
    arrayNode.addChild(parseSingle(tokens));

    // The next token is either a comma or it is the closing bracket.
    // In both cases the token needs to be removed. We just need to keep it around
    // to check if it is a comma.
    const nextToken = tokens.shift();

    // If the next token "after" the value is not a comma, we do not expect
    // any more values. Technically we dont even need the comma, but we are stick
    // to the standard strictly.
    if (nextToken && nextToken.isComma) {
      continue;
    }

    if (nextToken && nextToken.isArrayClose) {
      return arrayNode;
    }

    throw new JsonError("Additional comma at end of array entries", tokens);
  }

  throw new JsonError(
    "Unexpected token at the end of an array entry, most likely a missing comma",
    tokens,
  );
}

function parseObject(tokens: Token[]) {
  const objectNode = new ObjectNode();

  tokens.shift();

  const firstToken = tokens[0];

  // Empty Object, exit early
  if (firstToken && firstToken.isObjectClose) {
    tokens.shift();

    return objectNode;
  }

  while (tokens.length > 0) {
    objectNode.addEntry(parseObjectEntry(tokens));

    const nextToken = tokens.shift();

    // If there is a comma, the json specifies that there *must*
    // be another entry on the object
    if (nextToken && nextToken.isComma) {
      continue;
    }

    // If the next token is not a comma, there are no more entries
    // which means that the next token *must* be a "}
    if (nextToken && nextToken.isObjectClose) {
      return objectNode;
    }

    throw new JsonError(
      "Unexpected token at the end of an object entry, most likely a missing comma",
      tokens,
    );
  }

  throw new JsonError("Unexpected end of source, while parsing object", tokens);
}

function parseObjectEntry(tokens: Token[]) {
  const [keyToken, seperatorToken] = tokens;

  if (!keyToken || !keyToken.isString) {
    throw new JsonError(
      `Unexpected token of type "${keyToken.type}" ("${keyToken.value}") on object key`,
      tokens,
    );
  }

  if (!seperatorToken || !seperatorToken.isColon) {
    throw new JsonError(
      `Unexpected token of type "${seperatorToken.type}" ("${seperatorToken.value}") as object key-value seperator`,
      tokens,
    );
  }

  tokens.splice(0, 2);

  return {
    key: keyToken.value,
    value: parseSingle(tokens),
  };
}

function validateString(value: string, tokens: Token[]) {
  const chars = value.split("");

  for (let index = 0; index < chars.length; index++) {
    const element = chars[index];

    if (
      element === "t" ||
      element === "n" ||
      element === "b" ||
      element === "f" ||
      element === "r"
    ) {
      throw new JsonError(
        "Invalid characters in string. Control characters must be escaped!",
        tokens,
      );
    }

    if (element !== "\") {
      continue;
    }

    if (chars.length <= index + 1) {
      throw new JsonError("Unexpected end of escape-sequence", tokens);
    }

    const escapeCharacter = chars[index + 1];

    if (escapeCharacter === "\" || escapeCharacter === "/") {
      index++;

      continue;
    }

    if (["b", "f", "n", "r", "t", '"'].includes(escapeCharacter)) {
      continue;
    }

    if (escapeCharacter === "u") {
      if (chars.length >= index + 6) {
        const unicodeEscapeSequence = chars.slice(index + 2, index + 6).join("");

        if (/^[0-9A-Fa-f]{4}$/.test(unicodeEscapeSequence)) {
          index += 5;

          continue;
        } else {
          throw new JsonError("Invalid unicode escape sequence", tokens);
        }
      } else {
        throw new JsonError("Unexpected end of escape-sequence", tokens);
      }
    }

    throw new JsonError("Unrecognized escape sequence", tokens);
  }
}

src/lib/generator.ts

import { JsonArray, JsonObject, JsonValue } from "./types";
import { Node, ObjectNode, ArrayNode, ScalarNode } from "./nodes";

export function convertNodeToJsValue(root: Node): JsonValue {
  if (root instanceof ScalarNode) {
    return scalarToJsValue(root);
  }

  if (root instanceof ObjectNode) {
    const result: JsonObject = {};

    for (const [key, value] of root.entries) {
      result[key] = convertNodeToJsValue(value);
    }

    return result;
  }

  if (root instanceof ArrayNode) {
    const result: JsonArray = [];

    for (const value of root.children) {
      result.push(convertNodeToJsValue(value));
    }

    return result;
  }

  throw new Error(
    `Unknown node type "${root.constructor.name}" found while deserializing tree`,
  );
}

function scalarToJsValue(node: ScalarNode) {
  const { type, value } = node;

  switch (type) {
    case "boolean":
      if (value === "true") {
        return true;
      }

      if (value === "false") {
        return false;
      }

      throw new Error(`Invalid boolean value "${value}" found.`);
    case "null":
      return null;
    case "number":
      return Number(value as string);
    case "string":
      return value;
    default:
      throw new Error(`Unknown node of type "${type}" cannot be converted to a js value`);
  }
}

src/lib/nodes/index.ts

export * from './Node';
export * from './ArrayNode';
export * from './ScalarNode';
export * from './ObjectNode';

src/lib/nodes/Node.ts

/**
 * Simple node base class.
 *
 * This class mainly exists to make typing a little easier.
 */
export class Node {}

src/lib/nodes/ScalarNode.ts

import { Node } from './Node';

export class ScalarNode extends Node {
  public readonly type: string;
  public readonly value: string;

  public constructor(type: string, value: string) {
    super();

    this.type = type;
    this.value = value;
  }
}

src/lib/nodes/ObjectNode.ts

import { Node } from './Node';

export class ObjectNode extends Node {
  public readonly entries: Map<string, Node>;

  public constructor() {
    super();

    this.entries = new Map();
  }

  public addEntry(entry: { key: string, value: Node }): void {
    this.entries.set(entry.key, entry.value);
  }
}

src/lib/nodes/ArrayNode.ts

import { Node } from './Node';

export class ArrayNode extends Node {
  public readonly children: Node[];

  public constructor() {
    super();

    this.children = [];
  }

  public addChild(value: Node): void {
    this.children.push(value);
  }
}

Thanks in advance 😄


Get this bounty!!!

#StackBounty: #postgresql #json #tree How to turn a set of flat trees into a single tree with multiple leaves?

Bounty: 50

So we have this beautiful postgres tree generator. Yet it kind of produces cuts of a tree not a whole tree all at once:

item_id jsonb_pretty
1   {
    "title": "PARENT",
    "item_id": 1,
    "children": {
        "title": "LEVEL 2",
        "item_id": 2,
        "children": {
            "title": "LEVEL 3.2",
            "item_id": 6
        }
    }
}
1   {
    "title": "PARENT",
    "item_id": 1,
    "children": {
        "title": "LEVEL 2",
        "item_id": 2,
        "children": {
            "title": "LEVEL 3.1",
            "item_id": 3,
            "children": {
                "title": "LEVEL 4.1",
                "item_id": 4
            }
        }
    }
}
1   {
    "title": "PARENT",
    "item_id": 1,
    "children": {
        "title": "LEVEL 2",
        "item_id": 2,
        "children": {
            "title": "LEVEL 3.1",
            "item_id": 3,
            "children": {
                "title": "LEVEL 4.2",
                "item_id": 5
            }
        }
    }
}

I want to get a single tree object out from it with an array like this:

1   {
    "title": "PARENT",
    "item_id": 1,
    "children": [{
        "title": "LEVEL 2",
        "item_id": 2,
        "children": [{
            "title": "LEVEL 3.2",
            "item_id": 6
        },
        {
            "title": "LEVEL 3.1",
            "item_id": 3,
            "children": [{
                "title": "LEVEL 4.1",
                "item_id": 4
            },
            {
                "title": "LEVEL 4.2",
                "item_id": 5
            }]
        }]
    }]
}

Here is the generator:

CREATE TABLE items (
  item_id     serial PRIMARY KEY,
  title text
);
CREATE TABLE joins (
  id          serial PRIMARY KEY,
  item_id     int,
  child_id    int
);
INSERT INTO items (item_id,title) VALUES
  (1,'PARENT'),
  (2,'LEVEL 2'),
  (3,'LEVEL 3.1'),
  (4,'LEVEL 4.1'),
  (5,'LEVEL 4.2'),
  (6,'LEVEL 3.2');
INSERT INTO joins (item_id, child_id) VALUES
  (1,2),
  (2,3),
  (3,4),
  (3,5),
  (2,6);

WITH RECURSIVE t(item_id, json, level) AS (
        SELECT item_id, to_jsonb(items), 1
        FROM items
        WHERE NOT EXISTS (
                SELECT 2
                FROM joins
                WHERE items.item_id = joins.item_id
        )
        UNION ALL
        SELECT parent.item_id, to_jsonb(parent) || jsonb_build_object( 'children', t.json ),
               level + 1
        FROM t
        JOIN joins AS j
                ON t.item_id = j.child_id
        JOIN items AS parent
                ON j.item_id = parent.item_id
        WHERE level < 7
)
SELECT item_id, jsonb_pretty(json)
FROM t
WHERE item_id = 1;

How one could change such a generator to produce one single tree in Postgres 13+?


Get this bounty!!!

#StackBounty: #php #json #laravel #voyager How to prefill voyager BREAD form with currently logged in user?

Bounty: 50

[SOLVED]
I have a simple listing in my laravel 5.7 project,and used voyager to manage its data.
and i want to make a fully working created_by and last_edit_by BREAD table but still have no idea to get the currently logged in user.

i currently using custom views(resources/views/vendor/voyager) with copy-pasted original codes,but still using original VoyagerBreadController.php with some additional import.

?php

namespace AppHttpControllers;

use IlluminateHttpRequest;
use IlluminateAuthAuthManager;
use IlluminateSupportFacadesAuth;
use AuthIlluminateFoundationAuthAuthenticatesUsers;
use AppWisata;
use AppUser;

namespace AppHttpControllersVoyager;

use TCGVoyagerHttpControllersVoyagerBreadController as BaseVoyagerBreadController;

class VoyagerBreadController extends BaseVoyagerBreadController
{
    //
}

and able to do this on my custom view

//get currently logged user
@php
  $currentUser = Auth::user()->nama;
@endphp

//printed output(for testing)
 {{$currentUser}}

and it works!,so i tried to set the BREAD option like this

{
    "default" : "{{$currentUser}}"
}

however,unlike my custom views it doesn’t returned the $currentUser value,just plain {{$currentUser}} in my tables.

please help me to make a fully functional created_by and last_edit_by table.

notes: im open to use any packages.

EDIT : it’s solved by duplicating the form just below the real dynamic form in my custom view(edit-add.blade.php),make a hidden type created_by & last_edited_by table & keep the BREAD option empty. and you can still adjust the visiblity without any problems.

//created_by form
<div class="form-group hidden  col-md-12 "><input type="hidden" class="form-control" name="created_by" placeholder="Created By" value="{{$currentUser}}"></div>
//last_edited_by form
<div class="form-group hidden  col-md-12 "><input type="hidden" class="form-control" name="last_edited_by" placeholder="Last Edited By" value="{{$currentUser}}"></div>

anyway , i still would be very thankful if someone give more efficient solution 🙂


Get this bounty!!!