Subscribe to this thread
Home - General / All posts - Threads argument to functions
tjhb

7,457 post(s)
online
#16-Jul-17 07:39

It would be great to see an explanation of why it is necessary to have a ThreadConfig parameter for some special functions, how this parameter relates to the THREADS directive for the enclosing query statement, and why we need both.

The functions taking a ThreadConfig parameter are currently the Geom*Par table functions. Why can't they just use the THREADS directive for the query statement that contains them? What happens to threading when one of these functions is enclosed in an outer function that has no ThreadConfig parameter? How many threads does the outer function use, and how is that determined?

tjhb

7,457 post(s)
online
#16-Jul-17 08:18

Interpreting Adam's answer here (point 2), I think the answer can be put like this.

THREADS governs the number of threads allocated per batch. BATCH governs the number of objects scheduled per batch.

ThreadConfig governs the number of threads allocated per object. (Sometimes also the batch size, for compound objects such as multipoints.)

Ordinarily, it's best to concentrate on THREADS, i.e. on the number of objects in flight at a time.

For large or complex objects, it may also be beneficial to allocate more than one thread per object, but substantial benefit from this will be rare.

adamw


7,159 post(s)
#16-Jul-17 08:34

This is correct.

THREADS is used to parallelize query statements.

Thread configuration strings are used to parallelize individual function calls.

Ordinarily, you would use either one or the other, whichever suits the process better. It is completely normal to have a multi-statement query which uses THREADS in one statement and XxxPar functions with ThreadConfig in another statement.

Also: we'll be getting rid of BATCH and convert THREADS to take a JSON thread configuration, similar to the functions. This will allow us to take more parallelization parameters for query statements and unify the syntax.

Manifold User Community Use Agreement Copyright (C) 2007-2017 Manifold Software Limited. All rights reserved.