georeference.org
Subscribe to this thread
Home - Cutting Edge / All posts - Manifold APIs web site
adamw


6,668 post(s)
#14-Mar-17 15:38

We reworked our documentation process to tighter integrate producing technical documentation with development. The ideal is to have new query functions or new methods in the object model, as well as changes to existing functions, objects and methods, appear in the technical documentation at about the same time they appear in the cutting edge builds. Another important goal is to better document the exact computation methods and formulae used.

The produced technical documentation is going to appear on a new web site:

Manifold APIs

We started with a couple of top-level objects in the .NET / COM object model. Hopefully, the few current pages convey what we want to achieve.

We will be extending the documentation heavily in the days and weeks ahead. All updates are going to be posted to this thread. We welcome your feedback on the content as it appears.

tjhb
7,093 post(s)
#14-Mar-17 16:33

This is great! Very thorough comments on the examples so far.

How about examples in languages other than C#. Basic, starter syntax for VB.net, IronPython and F# would be good, as well as occasional worked examples in one or more of those languages besides C#.

(If it were me I would ditch C# and write the whole thing in (Iron)Python, that being the language most commonly used for GIS scripting elsewhere (and for other reasons: duck typing, comprehensions... arguably closely aligned with SQL). But C# is the .NET language most commonly used generally, even though it is verbose and ugly.)

There is lots of (other) good news in your first paragraph, especially first and last sentences.

jkelly


1,230 post(s)
#15-Mar-17 00:02

Tim, in regards to programming languages, beauty is in the eye of the beholder.


James Kelly

http://www.locationsolve.com

tjhb
7,093 post(s)
#15-Mar-17 00:36

Good code can always be beautiful, even when expressed in an ugly language.

The initial C# examples on the API web site are certainly beautiful (even if they must be verbose--verbosity is not in the eye of the beholder).

There has to be a standard language for the API, and it probably must be C#.

I would just like extra examples in other languages.

Almost everyone coming to Radian/Manifold from scripting in ESRI or QGIS will know Python (not necessaarily only Python, but that will be their normal idiom for scripting).

tjhb
7,093 post(s)
#15-Mar-17 00:56

And yes, that's all partly tongue in cheek. I know it's partly subjective to say that C# is verbose and ugly but Python is beautiful. And IronPython is built on C#--if C# were not such a great language (along with other features of the framework), then IronPython could not thrive or even exist.

A point of perspective: the best development language may not be the best language for scripting.

Currently, the best language for GIS scripting [on the desktop] is Python. I think that's objectively true. [On the web it is JavaScript--or V8.]

tjhb
7,093 post(s)
#15-Mar-17 01:42

Plus, Radian already has an IronPython REPL. It does not yet have a C# REPL (though it could).

tjhb
7,093 post(s)
#15-Mar-17 01:23

Out of my depth, but it would also be great to see a basic example using Revolution R, aka Microsoft R Server. Just enough to get access to a dataport.

Pluralism is the essence of Radian.

adamw


6,668 post(s)
#15-Mar-17 07:29

We hear you on languages other than C# in general and Python in particular. We definitely need to have docs demonstrating how to write Radian scripts in IronPython. Python is very popular in the scientific community and in GIS, and we even have REPL for it, as you say below. We will try to provide that.

mdsumner


4,157 post(s)
#15-Mar-17 07:42

R initiatives should focus on DBI compatibility to leverage dplyr's abstractions over SQL. Most of this is easyish R package level code too.

Whether Microsoft R or GNU R is less important imo. RStudio already has the reactive interactive programming environments for ultra flexibility and would easily give the best alternative front-end to users tying JavaScript and R and Manifold together.

artlembo


2,704 post(s)
#15-Mar-17 11:29

another option for Python of course is to use:

import win32app.client

and instantiate Radian - it works for 8, so I suppose it will work for Radian. Of course this operates outside of the GUI, but you can then use Python 2.7, or whatever else you want.

KlausDE

5,968 post(s)
#14-Mar-17 17:03

Another important goal is to better document the exact computation methods and formulae used.

'like'-smiley missing, three of them.

Actually this is a prerequisite for use of Mfd-functions in scientific work where it's not enough to know that methods work as expected.

artlembo


2,704 post(s)
#14-Mar-17 18:14

great start - I'll be keeping my eye on this as it gets expanded.

jkelly


1,230 post(s)
#15-Mar-17 00:04

I will agree with all the others, this is great news. I particularly like the examples.


James Kelly

http://www.locationsolve.com

cartomatic

884 post(s)
#15-Mar-17 04:26

examples are a great idea.

+1 for the examples in other langs. it would be a nice touch although c# presents the apis clear enough.


maps made easy - www.cartomatic.pl || www.cartoninjas.net

adamw


6,668 post(s)
#16-Mar-17 17:23

We added documentation for the Database object in .NET / COM. Lots of examples.

There is a short overview chapter as well, designed for quick skimming.

adamw


6,668 post(s)
#20-Mar-17 12:23

We added documentation for the Table object in .NET / COM. All but one method have examples (the remaining method will have an example or two as well, we are missing a service function).

There is a short overview chapter, as usual.

tjhb
7,093 post(s)
#20-Mar-17 22:27

(1)

Would it be better for the Table.Search* functions to be named Table.Seek* or Table.Fetch* instead? I think that is at least true for Table.SearchAll, for the reason below.

In the case of Table.Search and Table.SearchBatch (and similarly Database.Search), the word "Search" is OK, it does make sense, since there is some matching going on, there are criteria. (I would still prefer "Seek" or "Fetch" in those cases, since those words seem to describe more accurately what is going, i.e. filtering a sequence, but that may be subjective and perhaps it doesn't matter.)

In the case of Table.SearchAll, there is no searching--no matching, no criteria. Here "Seek" or "Fetch" are both clearly better than "Search", which makes the actual purpose less obvious and possibly confusing. ("What are you searching for? I'm searching for everything!"--That doesn't really make sense.)

(2)

There seems to be some confusion (maybe just mine) in the documentation for Table.SearchBatch. I can't follow it.

First, we have

Parameters

index - name of a unique index. Case-insensitive.

keys - key values specifying the search criteria.

fields - list of fields to return. May be empty.

...

Notes

The method attempts to locate records that match the specified search criteria. [This could be omitted since it's obvious and/or implicit under Parameters.]

The index parameter is the name of a unique index. [Unnecessary repetition.] If the index is not found or is not unique, the method will fail. [Crucial.]

The keys parameter provides the key values specifying the search criteria. [Again just repetition.] The key values are interpreted according to the index type. [Crucial, and should possibly stand as a separate paragraph so that its relation to what follows is more obvious.]

...

The bits in bold are what I'm focussing on. [The comments in brackets above are secondary, not the main point here.]

So far so good, and clear.

Next there are two sections which explain that last sentence, "The key values are interpreted according to the index type", for BTREE / BTREEDUP / BTREEDUPNULL / BTREENULL indexes on the one hand, and RTREE on the other.

Wait a second... Here's the first confusing thing. The documentation has aleady said that the index named in the index parameter must be unique. Now we have the possibility that "the index" may not only be BTREE or BTREENULL, but also BTREEDUP or BTREEDUPNULL, that is, not unique. Huh? What does "the index" mean here then?

Going on:

BTREE / BTREEDUP / BTREEDUPNULL / BTREENULL indexes interpret key values as the starting point of the search.

If we weren't already confused about what "indexes" means here (it apparently can't mean the index used as the index parameter), maybe the meaning here would likewise be clear.

In general, how do "indexes" relate to the search fields? Well, this is exactly what the section is trying to explain! Yes, but it doesn't do a very good job! I have no idea.

The rest is (maybe, consequently) no clearer, at least to me...

For an index that uses a single field: ...

For an index that uses multiple fields: ...

If the index allows duplicates and there are multiple records sharing the values of the index fields on which the sequence should start, the sequence is guaranteed to read through all of these records.

The index may revert the ordering in one or more fields via field options.

[Should "revert" there mean "reverse", or something else? Anyway probably not "revert"--that is, to put something back the way it was.]

Next in the same section:

The values in the keys parameter that do not correspond to any of the fields used by the index are ignored.

This seems crucial, but I don't understand it. Does it mean that we can't search for any values not involved in some BTREE* index (possibly with duplicates)? (If so, this sentence should probably be promoted higher, possibly under Parameters). How does that index (or possibly, how do these indexes) relate to the index parameter, where the index must be unique?

Lastly here:

The order of records in the returned sequence proceeds from smaller to larger values of the index fields according to the index field options.

Here, I think "index fields" does refer to the index parameter, and "index field options" means, for example, ASC/DESC, CASE/NOCASE, ACCENT/NOACCENT, SYMBOLS/NOSYMBOLS.

If that's right, I was initially thrown off the right path by one of the examples further below:

// list all records Manifold.Values keys = new Manifold.Values(); app.Log("Listing all records ordering by name"); ListSequence(app, table.SearchBatch("name_x", keys, fields));

Here "ordering by name" could be better worded: it's actually ordering by the index "name_x", with whatever fields that includes (maybe just one, called "name") and the associated options (maybe none). (In other words, this is not necessarily or always the same as ordering by the field called "name".)

It would also be worth explaining out loud, either in this example, or higher up, that if an empty Manifold.Values() collection is passed as the keys parameter, then we get all records (if I'm reading that right).

adamw


6,668 post(s)
#21-Mar-17 08:37

Thanks a lot!

In order:

We hear you on 'SearchAll' sounding weird. We will see if we can come with a better name, but, frankly, we don't want to lose the 'Search' prefix. Also, we tried many different verbs for what we now call 'Search' and 'Search', while imperfect, fit best. (For one thing, we don't want to use 'Fetch' as we use it in sequences. You first 'Search' a table and then you get a sequence from which you 'Fetch' records one by one.)

The index passed to Table.SearchBatch does not have to be unique - that the chapter says it should be is an oversight. Thanks for catching this, we will fix it.

'An index' and 'the index' in the chapter refer to the index identified by the index parameter. 'Index field options' are things like ASC/DESC, etc, yes.

We'll change 'revert' to 'reverse'. Thanks again.

Does it mean that we can't search for any values not involved in some BTREE* index (possibly with duplicates)?

Exactly, we can not. If you have a table with fields City, Population, Area and you have an index on City, you can only use that index to search on values in City. If you want to search on City and Area, either build an index on them both or use a query which will use indexes where it can and perform its own searching on the rest. (Perhaps it makes sense to spell it in the docs.)

We will change the wording in the example from 'Listing all records ordering by name' to something like 'Listing all records in the order of the index ...', we agree it is better.

We kind of explain already that if the keys passed to a BTREE / BTREExxx index are empty, we get all records (we say that the sequence starts at the record with the smallest value from the point of view of the index and since it proceeds in order of increasing values, it will eventually get all records), but we will make it clearer. There is a point to be made here in that a btree index will go through all records, but an rtree index will skip records with NULL values in the geom field. We are also missing notes for rtree indexes on tiles.

Again, thanks a lot, this really helps.

tjhb
7,093 post(s)
#21-Mar-17 22:18

I'm very grateful that you waded through all of that Adam. In retrospect I could have been more succinct (but I was confused).

Once this is in the right place--

The index passed to Table.SearchBatch does not have to be unique

then the rest starts to look a lot clearer, also simpler. (Ex contradictione quodlibet--so a partly controlled detonation.)

I understand if a standardized Search* prefix is final. It's not so bad, and apart from Seek (not sure) I can't think of anything better. You're obviously right about Fetch, since these functions don't fetch, they return a Sequence object. From Python analogies I am thinking of this as an iterator or generator. Yield* would be a misuse (since that happens, exactly, iteratively). You will have considered Find*, Filter* and enough others to become boring. There may be no perfect fit--and really this doesn't matter much, once we become used to it. So long as the documentation can give the right hints to oil the understanding, it's fine.

Thanks again, and let me add that I've read the rest of the API documentation so far and didn't find anything else confusing. (That doesn't mean I actually understood it--but I felt as if I did!)

tjhb
7,093 post(s)
#22-Mar-17 03:34

Even so.

This is not what I want to post, but I think it needs to be said by someone.

Parameters

index - name of a unique index. Case-insensitive

...

Notes

The index parameter is the name of a unique index. If the index is not found or is not unique, the method will fail.

These statements should not have been posted to the new API website. They should have been picked up as errors.

Who was responsible for sanity checking?

It's not good enough to delegate sanity checks to users who are also learning. In practice that might work, but the excessive wasted time makes it wrong.

adamw


6,668 post(s)
#22-Mar-17 08:06

We use the same process for this documentation as we do for software - the changes are reviewed individually and the end product is tested as a whole. Every stage uncovers some issues which are then fixed. Unfortunately, there is no guarantee that the end result will have no issues whatsoever. Bugs happen.

We aren't just putting the documentation onto the web and sitting there waiting until someone submits a bug report. We'd likely eventually have found and fixed the issue in the notes for Table.SearchBatch by ourselves - but it is obviously better to be able to fix it earlier, and we are very grateful for all reports that help us do that. Thank you!

Dimitri


3,791 post(s)
online
#22-Mar-17 09:48

Who was responsible for sanity checking?

The same team that is in charge of creating perfect software that never has any bugs. :-) Still working on that.

Can't resist adding... anybody worried about pushing the edge and hitting an occasional bug or inaccuracy shouldn't be playing in the Cutting Edge sandbox... perhaps we could put a sticker on the API info "This is Cutting Edge"


Словом - иностранец. «Мастер и Маргарита» (М.Булгаков)

tjhb
7,093 post(s)
#22-Mar-17 21:56

You are both right, of course. The process issue, if there is one, was on my side. I should not have tried so hard to make sense of the text I was looking at. Everyone is fallible--not only me.

When it seemed not to make sense, I should have (OK, tried a bit harder, and maybe slept on it, then...) said so, rather than squaring the circle.

The worst is that I would have felt a bit foolish for missing something obvious, but no one really minds that, and even then the feedback might have been useful, since you want the obvious to be easy to see.

Thanks for your comments.

adamw


6,668 post(s)
#03-Apr-17 17:53

We added documentation for the Expression and ExpressionParser objects (new in 9.0.160.1) in .NET / COM and for the methods in other objects that create them.

The example for Expression.Evaluate is of particular interest and shows how to wrap query functions for seamless use from within a script (the example wraps GeomBuffer).

Manifold User Community Use Agreement Copyright (C) 2007-2011 Manifold Software Limited. All rights reserved.