ci: elasticsearch: Upload script improvements

[golowanow]

Multiple improvements of the upload_test_results_es.py script:

• JSON objects flattening.

  This feature allows twister.json file preprocessing to simplify its Elasticsearch index structure for complex hierarhical objects, for example with memory footprint, or code coverage data.

  A new command line option --flatten is added to change testsuite data structure in regard of one of its list objects: either testcases or recording, so each item there becomes an independent data record inheriting all other testsuite properties, whereas the children object's properties are renamed with the parent object's name as a prefix: 'testcases_' or 'recording_' respectively. Only one testsuite property can be flattened this way per index upload. Other children objects will be treated accorging to the index structure.

  Related new command line options (with help text explanations): --flatten-dict-name, --flatten-list-names, --flatten-separator, --transpose-separator, --escape-separator

• A new command line option --transform is added to allow regexp group parsing in string propertites extracting additional derived properties.

• A new command line option --exclude is added to exclude testsuite properties not needed to store at Elasticsearch index.

• Branch name --run-branch and Workflow ID --run-workflow command line options as additional key fields to allow data from different branches, workflows and triggering events in the same index.

• A new command line option --map-file is added to apply an explicit index structure to the twister.json input data.

• Add bulk operation timeout parameter for heavy/long uploads.

Other changes:

• batch upload error handling and logging;
• inline documentation improvements;
• some corner case fixes on empty objects.

Previously mentioned in relation to memory footprint data collection.

Examples of use:

Collect data from tests with recording:

• timer accuracy (kernel.timer.timer, kernel.timer.timer_behavior_external)

./scripts/ci/upload_test_results_es.py \
        --flatten recording \
        --exclude path run_id \
        -i zephyr-tests-recording-timer \
	artifacts/**/twister.json

• kernel benchmarks (benchmark.kernel.latency.*, benchmark.user.latency.*)

./scripts/ci/upload_test_results_es.py \
        --flatten recording \
        --exclude path run_id \
        --transform "{ 'recording_metric': '(?P<recording_metric_object>[^\.]+)\.(?P<recording_metric_action>[^\.]+)\.(?P<recording_metric_details>[^ -]+)' }" \
        -i zephyr-tests-recording-benchmarks \
	artifacts/**/twister.json

Collect footprint data (Twister with --footprint-report and --enable-size-report)

./scripts/ci/upload_test_results_es.py \
        --flatten footprint \
        --exclude path run_id runnable retries execution_time build_time testcases \
        --flatten-list-names "{'children':'name'}" \
        --transform "{ 'footprint_name': '^(?P<footprint_area>([^\/]+\/){0,2})(?P<footprint_path>([^\/]*\/)*)(?P<footprint_symbol>[^\/]*)$' }" \
        -i zephyr-tests-footprint-metrics \
	artifacts/**/twister_footprint.json

Other possible use - keep code coverage data with #66345

[nashif]

You are adding lots of new functionality that is not being used anywhere. I would rather see this come along side whoever is going to use the new options, I guess this would be the footprint data.

[golowanow]

I've put into PR's description some examples for how this extended script is used for timer accuracy data, kernel benchmarks and footprint data.