mldev: introduce IO layout

Introduce IO layout in ML device specification.
IO layout defines the expected arrangement of model input and output
buffers in the memory.
Packed and Split layout support

mldev: introduce IO layout

Introduce IO layout in ML device specification.
IO layout defines the expected arrangement of model input and output
buffers in the memory.
Packed and Split layout support is added in the specification.

Updated rte_ml_op to support array of rte_ml_buff_seg
pointers to support packed and split I/O layouts.
Updated ML quantize and dequantize APIs to support rte_ml_buff_seg
pointer arrays.
Replaced batch_size with min_batches and max_batches in rte_ml_model_info.

Implement support for model IO layout in ml/cnxk driver.
Updated the ML test application to support IO layout and
dropped support for '--batches' in test application.

Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Shivah Shankar S <sshankarnara@marvell.com>

show more ...

app/mldev: support inference validation

Enabled support to validate inference output with reference
output provided by the user. Validation would be successful
only when the inference outputs are wi

app/mldev: support inference validation

Enabled support to validate inference output with reference
output provided by the user. Validation would be successful
only when the inference outputs are within the 'tolerance'
specified through command line option "--tolerance".

Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>

show more ...

app/mldev: support inference batches

Enabled support to execute multiple batches of inferences
per each enqueue request. Input and reference for the test
should be appropriately provided for multi-b

app/mldev: support inference batches

Enabled support to execute multiple batches of inferences
per each enqueue request. Input and reference for the test
should be appropriately provided for multi-batch run. Number
of batches can be specified through "--batches" option.

Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>

show more ...

app/mldev: add ordered inferences

Added an ordered test case to execute inferences with single
or multiple models. In this test case inference requests for
a model are enqueued after completion of a

app/mldev: add ordered inferences

Added an ordered test case to execute inferences with single
or multiple models. In this test case inference requests for
a model are enqueued after completion of all requests for
the previous model. Test supports inference repetitions.

Operations sequence when testing with N models and R reps,

(load -> start -> (enqueue + dequeue) x R -> stop -> unload) x N

Test case can be executed by selecting "inference_ordered" test
and repetitions can be specified through "--repetitions" argument.

Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>

show more ...

app/mldev: validate model operations

Added test case to validate model operations. Model ops test
is a collection of sub-tests. Each sub-test invokes the model
operations in a specific order.

Sub-t

app/mldev: validate model operations

Added test case to validate model operations. Model ops test
is a collection of sub-tests. Each sub-test invokes the model
operations in a specific order.

Sub-test A: (load -> start -> stop -> unload) x n
Sub-test B: load x n -> start x n -> stop x n -> unload x n
Sub-test C: load x n + (start + stop) x n + unload x n
Sub-test D: (load + start) x n -> (stop + unload) x n

Added internal functions to handle model load, start, stop and
unload. List of models to be used for testing can be specified
through application argument "--models"

Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>

show more ...