#StackBounty: #laravel #elasticsearch Laravel Elasticsearch JSON Mapping Issue

Bounty: 50

I’m currently using Laravel v7.2, have the babenkoivan/scout-elasticsearch-driver installed (4.2) and am using AWS Elasticsearch 7.1. I have several tables mapped in my application that are working fine but am having issues with a nested mapping that was previously working and is now broken.

I’m saving data into a table and having that table data copied into AWS Elasticsearch. I’m using MySQL 5.6 so I am using a TEXT column to store JSON data. Data in the table looks as follows:

'id' => 1,
'results' => [{"finish":1,"other_id":1,"other_data":1}]

I have my model setup with the following mapping:

protected $mapping = [
        'properties' => [
            'results' => [
                'type' => 'nested',
                'properties' => [
                    'finish' => [
                        'type' => 'integer'
                    ],
                    'other_id' => [
                        'type' => 'integer'
                    ],
                    'other_data' => [
                        'type' => 'integer'
                    ]
                ]
            ],
        ]
    ];

And if it’s of any use, the toSearchableArray:

public function toSearchableArray()
    {
        $array = [
            'id' => $this->id,
            'results' => $this->results
        ];

        return $array;
    }

I have no problem creating this index and it worked up until about a couple of months ago. I don’t know exactly when, as it wasn’t a high priority item and may have occurred around an AWS ES update but not sure why this in particular would break. I receive the following error now:

{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"object mapping for [results] tried to parse field [results] as object, but found a concrete value"}],"type":"mapper_parsing_exception","reason":"object mapping for [results] tried to parse field [results] as object, but found a concrete value"},"status":400}

I’ve tried also storing the data in the table as such, thinking it was breaking due to the potential array, but it was to no avail:

'id' => 1,
'results' => {"finish":1,"other_id":1,"other_data":1}

I’m at a loss for what else to try to get this working again.

EDIT: Here is the entire model:

<?php
namespace AppModels;

use IlluminateDatabaseEloquentModel;
use ScoutElasticSearchable;


class ResultsModel extends Model
{

    use Searchable;

    protected $indexConfigurator = AppMyIndexConfiguratorResults::class;

    protected $searchRules = [
        //
    ];

    protected $mapping = [
        'properties' => [
            'results' => [
                'type' => 'nested',
                'properties' => [
                    'finish' => [
                        'type' => 'integer'
                    ],
                    'other_id' => [
                        'type' => 'integer'
                    ],
                    'other_data' => [
                        'type' => 'integer'
                    ]
                ]
            ],
        ]
    ];

    public function searchableAs()
    {
        return 'results_index';
    }

    public function toSearchableArray()
    {
        $array = [
            'id' => $this->id,
            'results' => $this->results
        ];

        return $array;
    }
    /**
     * The database table used by the model.
     *
     * @var string
     */
    protected $table = 'results_table';

}

Here is the AppMyIndexConfiguratorResults::class

<?php

namespace App;

use ScoutElasticIndexConfigurator;
use ScoutElasticMigratable;

class MyIndexConfiguratorResults extends IndexConfigurator
{
    use Migratable;

    protected $name = "results_index";
    /**
     * @var array
     */
    protected $settings = [
        //
    ];
}

This is all that is needed to have Laravel automatically update AWS ES each time the table is updated. For the initial load, I would SSH in and run the following command to have it create the index. This, as well as elastic:migrate and any update/insert into the table produces the mapping error.

php artisan elastic:create-index results_index


Get this bounty!!!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.