Subscribe to this thread
Home - Cutting Edge / All posts - Manifold System 9.0.163.10
adamw


10,447 post(s)
#24-Nov-17 17:34

9.0.163.10

manifold-future-9.0.163.10-x64.zip

SHA256: 2a806bbd67f12975167571428600b5411ee10aa9a53b5006753012ffb1e72e52

manifold-viewer-9.0.163.10-x64.zip

SHA256: ed5e457346524e11c133225fae97746f6a7ad7890fc340d826203ed0bfd5c533

adamw


10,447 post(s)
#24-Nov-17 17:35

Changes

The layout window avoids recomputing queries for components built on them after changes to the layout. The layout window shares the results of computing queries between layout frames: if the same component built on a query is used in multiple layout frames, the query will be computed only once.

(Fix) The layout window displays and prints images based on a table with a btree index on level-x-y fields (the format used by old Radian builds and occasionally used by some dataports when image data is actually stored that way).

(Fix) The query engine no longer fails to pass or receive tables from script functions. (Example uses of the feature follow.)

Expression objects in scripts can take table parameters. (Examples follow.)

Command objects in scripts can take table parameters. Inline queries called from scripts can take table parameters. (Examples follow.)

The PBF dataport reads data faster.

The PIX dataport recovers after invalid vector data.

The image servers dataport allows server URLs to include control sequences. URLs for built-in image servers have been expanded to include control sequences. Old-style URLs stored in data sources created before this build are supported, unless they have been edited manually (if so, the user has to re-create the data source).

An incomplete list of control sequences:

  • {x}, {y} - specify the X and Y coordinates of a tile,
  • {zoom} - specifies the zoom level of a tile (zoom level is a numeric value similar to an image level but going in the reverse direction: the image level of 0 is the most detailed level, while the zoom level of 0 is the least detailed level supported by the server),
  • {key} - specifies the API key if entered by the user,
  • {keyApp} - specifies the application key if entered by the user,
  • {lon}, {lat} - specify the longitude and latitude of the center of the requested tile,
  • {switch:a,b,c} - specify the server selector for load balancing, the choices after the semicolon are available alternatives.

A full list of control sequences will be in the documentation.

Creating a new image server data source allows selecting the 'Custom' image server and supplying a custom URL with the control sequences. Custom image servers use additional parameters such as the tile size (defaults to 256) and the zoom range (defaults to 0 through 20).

(There are many sources of custom image servers. This page, for example, lists many custom image servers designed for use with OSM: https://josm.openstreetmap.de/wiki/Maps. To try, say, the 'Hike & Bike' server, click it in the list to locate the URL, then use that URL with the custom image server in Future: http://{switch:a,b,c}.tiles.wmflabs.org/hikebike/{zoom}/{x}/{y}.png )

Connection strings for data sources generated by the UI have been cleaned up to avoid converting boolean and numeric values to strings.

New query function: ValueSequenceRandom - takes the number of values to generate and a seed value, and returns a table of floating-point random values between 0 and 1. If the seed value is zero or negative, it is ignored and the random generator uses a seed value unique for the session.

New query function: ValueSequenceRandomInt - takes the number of values to generate, an integer limit and a seed value, and returns a table of integer random values between 0 and limit-1. If the seed value is zero or negative, it is ignored and the random generator uses a seed value unique for the session.

(For both functions above, the random number generator is xorshift128+.)

(Fix) Moving or resizing a layout frame no longer snaps it to itself.

Printing a layout tracks progress.

End of list.

adamw


10,447 post(s)
#24-Nov-17 17:37

Passing and returning values from script functions to queries.

Example 1: passing an integer value and returning an integer value (this particular example was working before this build, it is here for completeness).

Script (named 'test'):

//C#

class Script

{

 

public static int F(int v)

{

  return v + 5;

}

 

static void Main()

{

}

}

Query:

--SQL9

FUNCTION f(x INT32INT32 AS SCRIPT [test] ENTRY 'Script.F';

SELECT mfd_id, f(mfd_id) FROM [mfd_root];

adamw


10,447 post(s)
#24-Nov-17 17:38

Example 2: passing a table and returning an integer value:

Script:

//C#

class Script

{

 

public static int G(Manifold.Table t)

{

  if (t == null)

    return 0;

  using (Manifold.Sequence seq = t.SearchAll(new string[] { "mfd_id" } ))

  {

    if (seq == null)

      return 0;

 

    // compute sum of mfd_id

    int accum = 0;

    while (seq.Fetch())

      accum += System.Convert.ToInt32(seq.GetValues()[0].Data);

    return accum;

  }

}

 

static void Main()

{

}

}

Query:

--SQL9

FUNCTION g(x TABLEINT32 AS SCRIPT [test] ENTRY 'Script.G';

VALUES (g(mfd_root));

adamw


10,447 post(s)
#24-Nov-17 17:39

Example 3: passing an integer value and returning a table, this requires telling the query engine what schema the result table is going to have in advance using a separate function:

Script:

//C#

class Script

{

 

static Manifold.Context Manifold;

 

public static Manifold.Table H_table()

{

  Manifold.Schema schema = Manifold.Application.CreateSchema();

  schema.AddField("val""int32");

  Manifold.Table output = Manifold.Application.CreateTable();

  output.Design(schema);

  return output;

}

 

public static Manifold.Table H(int x)

{

  Manifold.Table output = H_table();

  if (output == null)

    return null;

  Manifold.ValueSet values = Manifold.Application.CreateValueSet();

  values.AddValue("val");

 

  // compute several first powers

  values[0].Data = x;

  output.Insert(values);

  values[0].Data = x * x;

  output.Insert(values);

  values[0].Data = x * x * x;

  output.Insert(values);

  return output;

}

 

static void Main()

{

}

}

Query:

--SQL9

FUNCTION h(x INT32TABLE AS SCRIPT [test] ENTRY 'Script.H'

  ENTRYSCHEMA 'Script.H_table';

EXECUTE CALL h(5);

adamw


10,447 post(s)
#24-Nov-17 17:40

Example 4: passing a table and returning another table:

Script:

//C#

class Script

{

 

static Manifold.Context Manifold;

 

public static Manifold.Table I_table(Manifold.Table t)

{

  if (t == null)

    return null;

  Manifold.Schema schema = t.GetSchema();

  schema.AddField("rec_num""int32");

  Manifold.Table output = Manifold.Application.CreateTable();

  output.Design(schema);

  return output;

}

 

public static Manifold.Table I(Manifold.Table t)

{

  Manifold.Table output = I_table(t);

  if (output == null)

    return null;

  Manifold.Schema schema = t.GetSchema();

  Manifold.ValueSet fields = Manifold.Application.CreateValueSet();

  foreach (Manifold.Schema.Field field in schema.Fields)

    fields.AddValue(field.Name);

  using (Manifold.Sequence seq = t.SearchAll(fields.GetValueNames()))

  {

    if (seq == null)

      return output;

 

    // number records starting from zero

    int num = 0;

    while (seq.Fetch())

    {

      Manifold.ValueSet valuesNew = seq.GetValues().CreateCopy(true);

      valuesNew.AddValue("rec_num", num);

      ++num;

      output.Insert(valuesNew);

    }

  }

  return output;

}

 

static void Main()

{

}

}

Query:

--SQL9

FUNCTION i(x TABLETABLE AS SCRIPT [test] ENTRY 'Script.I'

  ENTRYSCHEMA 'Script.I_table';

EXECUTE CALL i(mfd_root);

adamw


10,447 post(s)
#24-Nov-17 17:41

Passing and returning values in the reverse direction, from queries to script functions.

Example 5: passing an integer value and returning an integer value (this also was working before this build, the example is here for completeness).

Script:

//C#

class Script

{

 

static Manifold.Context Manifold;

static void Main()

{

  Manifold.Application app = Manifold.Application;

  using (Manifold.Database db = app.GetDatabaseRoot())

  {

    using (Manifold.ExpressionParser parser = db.CreateExpressionParser())

    {

      Manifold.ValueSet source = app.CreateValueSet();

      source.AddValueType("par"typeof(int));

      string expr = "par + 100";

      using (Manifold.Expression expression = parser.CreateExpression(expr, source))

      {

        source[0].Data = 2;

        Manifold.ValueSet result = expression.Evaluate(source);

        app.Log(result[0].Data.ToString());

      }

    }

  }

  app.OpenLog();

}

 

}

adamw


10,447 post(s)
#24-Nov-17 17:41

Example 6: passing a table and returning an integer value:

Script:

//C#

class Script

{

 

static Manifold.Context Manifold;

static void Main()

{

  Manifold.Application app = Manifold.Application;

  using (Manifold.Database db = app.GetDatabaseRoot())

  {

    using (Manifold.ExpressionParser parser = db.CreateExpressionParser())

    {

      Manifold.ValueSet source = app.CreateValueSet();

 

      // it is not enough to tell query engine that the parameter type is

      // going to be table, we also have to tell it what the table schema is

      // going to be, we do that by passing a model table - or the table that

      // we will evaluate with

      source.AddValue("par", db.Search("mfd_root"));

 

      string expr = "(SELECT Max(mfd_id) FROM par)";

      using (Manifold.Expression expression = parser.CreateExpression(expr, source))

      {

        Manifold.ValueSet result = expression.Evaluate(source);

        app.Log(result[0].Data.ToString());

      }

    }

  }

  app.OpenLog();

}

 

}

adamw


10,447 post(s)
#24-Nov-17 17:42

Example 7: passing an integer value and returning a table (this is called "doing a query", it was working before this build):

Script:

//C#

class Script

{

 

static Manifold.Context Manifold;

static void Main()

{

  Manifold.Application app = Manifold.Application;

  using (Manifold.Database db = app.GetDatabaseRoot())

  {

    Manifold.ValueSet source = app.CreateValueSet();

    source.AddValueType("par"typeof(int));

    source[0].Data = 5;

    string expr = "SELECT * FROM CALL ValueSequence(0, par, 1)";

    using (Manifold.Table table = db.Run(expr, source))

    {

      using (Manifold.Sequence seq = table.SearchAll(new string[] { "Value" }))

      {

        while (seq.Fetch())

          app.Log(seq.GetValues()[0].Data.ToString());

      }

    }

  }

  app.OpenLog();

}

 

}

The same using a compiled query:

Script:

//C#

class Script

{

 

static Manifold.Context Manifold;

static void Main()

{

  Manifold.Application app = Manifold.Application;

  using (Manifold.Database db = app.GetDatabaseRoot())

  {

    Manifold.ValueSet source = app.CreateValueSet();

    source.AddValueType("par"typeof(int));

    string expr = "SELECT * FROM CALL ValueSequence(0, par, 1)";

    using (Manifold.Command command = db.RunCompile(expr, source))

    {

      source[0].Data = 5;

      Manifold.Table table = command.Run(source);

      using (Manifold.Sequence seq = table.SearchAll(new string[] { "Value" }))

      {

        while (seq.Fetch())

          app.Log(seq.GetValues()[0].Data.ToString());

      }

    }

  }

  app.OpenLog();

}

 

}

adamw


10,447 post(s)
#24-Nov-17 17:44

Example 8: passing a table and returning another table:

Script:

//C#

class Script

{

 

static Manifold.Context Manifold;

static void Main()

{

  Manifold.Application app = Manifold.Application;

  using (Manifold.Database db = app.GetDatabaseRoot())

  {

    Manifold.ValueSet source = app.CreateValueSet();

    source.AddValue("par", db.Search("mfd_root")); // pass schema

    string expr = "SELECT mfd_id FROM par";

    using (Manifold.Table table = db.Run(expr, source))

    {

      using (Manifold.Sequence seq = table.SearchAll(new string[] { "mfd_id" }))

      {

        while (seq.Fetch())

          app.Log(seq.GetValues()[0].Data.ToString());

      }

    }

  }

  app.OpenLog();

}

 

}

The same using a compiled query.

Script:

//C#

class Script

{

 

static Manifold.Context Manifold;

static void Main()

{

  Manifold.Application app = Manifold.Application;

  using (Manifold.Database db = app.GetDatabaseRoot())

  {

    Manifold.ValueSet source = app.CreateValueSet();

    source.AddValue("par", db.Search("mfd_root")); // pass schema

    string expr = "SELECT mfd_id FROM par";

    using (Manifold.Command command = db.RunCompile(expr, source))

    {

      Manifold.Table table = command.Run(source);

      using (Manifold.Sequence seq = table.SearchAll(new string[] { "mfd_id" }))

      {

        while (seq.Fetch())

          app.Log(seq.GetValues()[0].Data.ToString());

      }

    }

  }

  app.OpenLog();

}

 

}

This covers all possible scenarios.

tjhb
10,094 post(s)
#24-Nov-17 20:15

This covers all possible scenarios.

Sure does! I think I see now what you meant by planning a tweak as well as a fix. The "tweak" (examples 5, 6, 8) is fairly huge! The circle closes.

Beautiful examples, really easy to follow (and to use). Thanks so much Adam.

Dimitri


7,413 post(s)
#25-Nov-17 18:53

The new imageserver custom capability is great. It works perfectly with many, many TMS servers.

Attached is a map we're using internally that works great with 163.10 Future and Future Viewer that contains over 30 new servers from the JOSM link Adam published. They are the worldwide ones with a sampling of some country versions. The New Zealand LINZ servers are outstanding, for example.

You can add more of the various country servers by drilling into the link and copying/pasting the URL string for various TMS servers into an imageserver - custom data source.

Enjoy!

Attachments:
JOSM sources.map

artlembo


3,400 post(s)
online
#26-Nov-17 01:49

The ValueSequenceRandom works perfectly, and fast. Thank you for putting that in. Even though it is cool that we can write our own functions, having it directly in the SQL is much easier. Also, it is a perfect way to make estimates are really large data sets.

I'll put together a video on my blog to illustrate this. I have some ideas on how to approach things, but I'm going to talk with a friend who is a computational biologist who does gene sequencing to see how sound my idea is. Also, this might actually allow MF to be used in biology once I show it to him :-)

Manifold User Community Use Agreement Copyright (C) 2007-2021 Manifold Software Limited. All rights reserved.