Subscribe to this thread
Home - Cutting Edge / All posts - Manifold APIs web site
adamw

7,102 post(s)
#14-Mar-17 15:38

We reworked our documentation process to tighter integrate producing technical documentation with development. The ideal is to have new query functions or new methods in the object model, as well as changes to existing functions, objects and methods, appear in the technical documentation at about the same time they appear in the cutting edge builds. Another important goal is to better document the exact computation methods and formulae used.

The produced technical documentation is going to appear on a new web site:

Manifold APIs

We started with a couple of top-level objects in the .NET / COM object model. Hopefully, the few current pages convey what we want to achieve.

We will be extending the documentation heavily in the days and weeks ahead. All updates are going to be posted to this thread. We welcome your feedback on the content as it appears.

tjhb

7,364 post(s)
#14-Mar-17 16:33

This is great! Very thorough comments on the examples so far.

How about examples in languages other than C#. Basic, starter syntax for VB.net, IronPython and F# would be good, as well as occasional worked examples in one or more of those languages besides C#.

(If it were me I would ditch C# and write the whole thing in (Iron)Python, that being the language most commonly used for GIS scripting elsewhere (and for other reasons: duck typing, comprehensions... arguably closely aligned with SQL). But C# is the .NET language most commonly used generally, even though it is verbose and ugly.)

There is lots of (other) good news in your first paragraph, especially first and last sentences.

jkelly


1,234 post(s)
#15-Mar-17 00:02

Tim, in regards to programming languages, beauty is in the eye of the beholder.


James Kelly

http://www.locationsolve.com

tjhb

7,364 post(s)
#15-Mar-17 00:36

Good code can always be beautiful, even when expressed in an ugly language.

The initial C# examples on the API web site are certainly beautiful (even if they must be verbose--verbosity is not in the eye of the beholder).

There has to be a standard language for the API, and it probably must be C#.

I would just like extra examples in other languages.

Almost everyone coming to Radian/Manifold from scripting in ESRI or QGIS will know Python (not necessaarily only Python, but that will be their normal idiom for scripting).

tjhb

7,364 post(s)
#15-Mar-17 00:56

And yes, that's all partly tongue in cheek. I know it's partly subjective to say that C# is verbose and ugly but Python is beautiful. And IronPython is built on C#--if C# were not such a great language (along with other features of the framework), then IronPython could not thrive or even exist.

A point of perspective: the best development language may not be the best language for scripting.

Currently, the best language for GIS scripting [on the desktop] is Python. I think that's objectively true. [On the web it is JavaScript--or V8.]

tjhb

7,364 post(s)
#15-Mar-17 01:42

Plus, Radian already has an IronPython REPL. It does not yet have a C# REPL (though it could).

tjhb

7,364 post(s)
#15-Mar-17 01:23

Out of my depth, but it would also be great to see a basic example using Revolution R, aka Microsoft R Server. Just enough to get access to a dataport.

Pluralism is the essence of Radian.

adamw

7,102 post(s)
#15-Mar-17 07:29

We hear you on languages other than C# in general and Python in particular. We definitely need to have docs demonstrating how to write Radian scripts in IronPython. Python is very popular in the scientific community and in GIS, and we even have REPL for it, as you say below. We will try to provide that.

mdsumner


4,182 post(s)
#15-Mar-17 07:42

R initiatives should focus on DBI compatibility to leverage dplyr's abstractions over SQL. Most of this is easyish R package level code too.

Whether Microsoft R or GNU R is less important imo. RStudio already has the reactive interactive programming environments for ultra flexibility and would easily give the best alternative front-end to users tying JavaScript and R and Manifold together.


https://github.com/mdsumner

artlembo


2,818 post(s)
#15-Mar-17 11:29

another option for Python of course is to use:

import win32app.client

and instantiate Radian - it works for 8, so I suppose it will work for Radian. Of course this operates outside of the GUI, but you can then use Python 2.7, or whatever else you want.

KlausDE
6,010 post(s)
#14-Mar-17 17:03

Another important goal is to better document the exact computation methods and formulae used.

'like'-smiley missing, three of them.

Actually this is a prerequisite for use of Mfd-functions in scientific work where it's not enough to know that methods work as expected.

artlembo


2,818 post(s)
#14-Mar-17 18:14

great start - I'll be keeping my eye on this as it gets expanded.

jkelly


1,234 post(s)
#15-Mar-17 00:04

I will agree with all the others, this is great news. I particularly like the examples.


James Kelly

http://www.locationsolve.com

cartomatic

884 post(s)
#15-Mar-17 04:26

examples are a great idea.

+1 for the examples in other langs. it would be a nice touch although c# presents the apis clear enough.


maps made easy - www.cartomatic.pl || www.cartoninjas.net

adamw

7,102 post(s)
#16-Mar-17 17:23

We added documentation for the Database object in .NET / COM. Lots of examples.

There is a short overview chapter as well, designed for quick skimming.

adamw

7,102 post(s)
#20-Mar-17 12:23

We added documentation for the Table object in .NET / COM. All but one method have examples (the remaining method will have an example or two as well, we are missing a service function).

There is a short overview chapter, as usual.

tjhb

7,364 post(s)
#20-Mar-17 22:27

(1)

Would it be better for the Table.Search* functions to be named Table.Seek* or Table.Fetch* instead? I think that is at least true for Table.SearchAll, for the reason below.

In the case of Table.Search and Table.SearchBatch (and similarly Database.Search), the word "Search" is OK, it does make sense, since there is some matching going on, there are criteria. (I would still prefer "Seek" or "Fetch" in those cases, since those words seem to describe more accurately what is going, i.e. filtering a sequence, but that may be subjective and perhaps it doesn't matter.)

In the case of Table.SearchAll, there is no searching--no matching, no criteria. Here "Seek" or "Fetch" are both clearly better than "Search", which makes the actual purpose less obvious and possibly confusing. ("What are you searching for? I'm searching for everything!"--That doesn't really make sense.)

(2)

There seems to be some confusion (maybe just mine) in the documentation for Table.SearchBatch. I can't follow it.

First, we have

Parameters

index - name of a unique index. Case-insensitive.

keys - key values specifying the search criteria.

fields - list of fields to return. May be empty.

...

Notes

The method attempts to locate records that match the specified search criteria. [This could be omitted since it's obvious and/or implicit under Parameters.]

The index parameter is the name of a unique index. [Unnecessary repetition.] If the index is not found or is not unique, the method will fail. [Crucial.]

The keys parameter provides the key values specifying the search criteria. [Again just repetition.] The key values are interpreted according to the index type. [Crucial, and should possibly stand as a separate paragraph so that its relation to what follows is more obvious.]

...

The bits in bold are what I'm focussing on. [The comments in brackets above are secondary, not the main point here.]

So far so good, and clear.

Next there are two sections which explain that last sentence, "The key values are interpreted according to the index type", for BTREE / BTREEDUP / BTREEDUPNULL / BTREENULL indexes on the one hand, and RTREE on the other.

Wait a second... Here's the first confusing thing. The documentation has aleady said that the index named in the index parameter must be unique. Now we have the possibility that "the index" may not only be BTREE or BTREENULL, but also BTREEDUP or BTREEDUPNULL, that is, not unique. Huh? What does "the index" mean here then?

Going on:

BTREE / BTREEDUP / BTREEDUPNULL / BTREENULL indexes interpret key values as the starting point of the search.

If we weren't already confused about what "indexes" means here (it apparently can't mean the index used as the index parameter), maybe the meaning here would likewise be clear.

In general, how do "indexes" relate to the search fields? Well, this is exactly what the section is trying to explain! Yes, but it doesn't do a very good job! I have no idea.

The rest is (maybe, consequently) no clearer, at least to me...

For an index that uses a single field: ...

For an index that uses multiple fields: ...

If the index allows duplicates and there are multiple records sharing the values of the index fields on which the sequence should start, the sequence is guaranteed to read through all of these records.

The index may revert the ordering in one or more fields via field options.

[Should "revert" there mean "reverse", or something else? Anyway probably not "revert"--that is, to put something back the way it was.]

Next in the same section:

The values in the keys parameter that do not correspond to any of the fields used by the index are ignored.

This seems crucial, but I don't understand it. Does it mean that we can't search for any values not involved in some BTREE* index (possibly with duplicates)? (If so, this sentence should probably be promoted higher, possibly under Parameters). How does that index (or possibly, how do these indexes) relate to the index parameter, where the index must be unique?

Lastly here:

The order of records in the returned sequence proceeds from smaller to larger values of the index fields according to the index field options.

Here, I think "index fields" does refer to the index parameter, and "index field options" means, for example, ASC/DESC, CASE/NOCASE, ACCENT/NOACCENT, SYMBOLS/NOSYMBOLS.

If that's right, I was initially thrown off the right path by one of the examples further below:

// list all records Manifold.Values keys = new Manifold.Values(); app.Log("Listing all records ordering by name"); ListSequence(app, table.SearchBatch("name_x", keys, fields));

Here "ordering by name" could be better worded: it's actually ordering by the index "name_x", with whatever fields that includes (maybe just one, called "name") and the associated options (maybe none). (In other words, this is not necessarily or always the same as ordering by the field called "name".)

It would also be worth explaining out loud, either in this example, or higher up, that if an empty Manifold.Values() collection is passed as the keys parameter, then we get all records (if I'm reading that right).

adamw

7,102 post(s)
#21-Mar-17 08:37

Thanks a lot!

In order:

We hear you on 'SearchAll' sounding weird. We will see if we can come with a better name, but, frankly, we don't want to lose the 'Search' prefix. Also, we tried many different verbs for what we now call 'Search' and 'Search', while imperfect, fit best. (For one thing, we don't want to use 'Fetch' as we use it in sequences. You first 'Search' a table and then you get a sequence from which you 'Fetch' records one by one.)

The index passed to Table.SearchBatch does not have to be unique - that the chapter says it should be is an oversight. Thanks for catching this, we will fix it.

'An index' and 'the index' in the chapter refer to the index identified by the index parameter. 'Index field options' are things like ASC/DESC, etc, yes.

We'll change 'revert' to 'reverse'. Thanks again.

Does it mean that we can't search for any values not involved in some BTREE* index (possibly with duplicates)?

Exactly, we can not. If you have a table with fields City, Population, Area and you have an index on City, you can only use that index to search on values in City. If you want to search on City and Area, either build an index on them both or use a query which will use indexes where it can and perform its own searching on the rest. (Perhaps it makes sense to spell it in the docs.)

We will change the wording in the example from 'Listing all records ordering by name' to something like 'Listing all records in the order of the index ...', we agree it is better.

We kind of explain already that if the keys passed to a BTREE / BTREExxx index are empty, we get all records (we say that the sequence starts at the record with the smallest value from the point of view of the index and since it proceeds in order of increasing values, it will eventually get all records), but we will make it clearer. There is a point to be made here in that a btree index will go through all records, but an rtree index will skip records with NULL values in the geom field. We are also missing notes for rtree indexes on tiles.

Again, thanks a lot, this really helps.

tjhb

7,364 post(s)
#21-Mar-17 22:18

I'm very grateful that you waded through all of that Adam. In retrospect I could have been more succinct (but I was confused).

Once this is in the right place--

The index passed to Table.SearchBatch does not have to be unique

then the rest starts to look a lot clearer, also simpler. (Ex contradictione quodlibet--so a partly controlled detonation.)

I understand if a standardized Search* prefix is final. It's not so bad, and apart from Seek (not sure) I can't think of anything better. You're obviously right about Fetch, since these functions don't fetch, they return a Sequence object. From Python analogies I am thinking of this as an iterator or generator. Yield* would be a misuse (since that happens, exactly, iteratively). You will have considered Find*, Filter* and enough others to become boring. There may be no perfect fit--and really this doesn't matter much, once we become used to it. So long as the documentation can give the right hints to oil the understanding, it's fine.

Thanks again, and let me add that I've read the rest of the API documentation so far and didn't find anything else confusing. (That doesn't mean I actually understood it--but I felt as if I did!)

tjhb

7,364 post(s)
#22-Mar-17 03:34

Even so.

This is not what I want to post, but I think it needs to be said by someone.

Parameters

index - name of a unique index. Case-insensitive

...

Notes

The index parameter is the name of a unique index. If the index is not found or is not unique, the method will fail.

These statements should not have been posted to the new API website. They should have been picked up as errors.

Who was responsible for sanity checking?

It's not good enough to delegate sanity checks to users who are also learning. In practice that might work, but the excessive wasted time makes it wrong.

adamw

7,102 post(s)
#22-Mar-17 08:06

We use the same process for this documentation as we do for software - the changes are reviewed individually and the end product is tested as a whole. Every stage uncovers some issues which are then fixed. Unfortunately, there is no guarantee that the end result will have no issues whatsoever. Bugs happen.

We aren't just putting the documentation onto the web and sitting there waiting until someone submits a bug report. We'd likely eventually have found and fixed the issue in the notes for Table.SearchBatch by ourselves - but it is obviously better to be able to fix it earlier, and we are very grateful for all reports that help us do that. Thank you!

Dimitri

4,107 post(s)
online
#22-Mar-17 09:48

Who was responsible for sanity checking?

The same team that is in charge of creating perfect software that never has any bugs. :-) Still working on that.

Can't resist adding... anybody worried about pushing the edge and hitting an occasional bug or inaccuracy shouldn't be playing in the Cutting Edge sandbox... perhaps we could put a sticker on the API info "This is Cutting Edge"

tjhb

7,364 post(s)
#22-Mar-17 21:56

You are both right, of course. The process issue, if there is one, was on my side. I should not have tried so hard to make sense of the text I was looking at. Everyone is fallible--not only me.

When it seemed not to make sense, I should have (OK, tried a bit harder, and maybe slept on it, then...) said so, rather than squaring the circle.

The worst is that I would have felt a bit foolish for missing something obvious, but no one really minds that, and even then the feedback might have been useful, since you want the obvious to be easy to see.

Thanks for your comments.

adamw

7,102 post(s)
#03-Apr-17 17:53

We added documentation for the Expression and ExpressionParser objects (new in 9.0.160.1) in .NET / COM and for the methods in other objects that create them.

The example for Expression.Evaluate is of particular interest and shows how to wrap query functions for seamless use from within a script (the example wraps GeomBuffer).

adamw

7,102 post(s)
#22-May-17 18:40

We finished a mammoth update to the doc for the .NET object model.

Specifically:

  • Completed documentation of all existing objects. Added documentation for all missing objects. Added documentation for all changes made by 9.0.161.
  • Shortened examples whenever possible. Cleaned up log reports. Stopped generating model values for expressions and used ValueSet.AddValueType instead. Composed connection strings using PropertySet, etc.
  • Adjusted examples to note all accesses of external data in comments.

As a simple illustration, the number of examples went from 61 to 152.

We need to extend the overviews (they don't currently cover everything and they should), but the reference part is complete. It will get much easier to maintain the doc from now on.

Enjoy!

mdsumner


4,182 post(s)
#22-May-17 19:26

Wow very nice, thanks Adam


https://github.com/mdsumner

artlembo


2,818 post(s)
#22-May-17 19:43

great! It would be nice to see some implementation of the API within VBScript and Javascript as well - beyond the basic Application.MessageBox examples.

Also, will Radian (and eventually 9) have the ability to create a Form inside of the .map project the way 8 did? Or, is that something you will need to do through .NET. I still believe, quite strongly, that an internal form creation option extends the project nicely as illustrated in the outstanding Forms programming reference on the web site.

adamw

7,102 post(s)
#23-May-17 07:12

Custom forms have to use .NET.

We might add a form designer in the future. In the meantime, we will try to provide an example of designing custom forms in Visual Studio.

Forms are a bit of a volatile area in that there are parallel advances in other areas that affect it. Give it some time. We agree forms are very useful.

tjhb

7,364 post(s)
#23-May-17 03:44

It makes great reading.

I've started to translate some examples to IronPython. (I don't know how far I'll get. Maybe not all 152.)

Would it be worth setting up a GitHub repository for translations like this? (Others might want to work on other languages, e.g. VB.NET, VBScript, JScript.)

adamw

7,102 post(s)
#23-May-17 07:17

Thanks.

It might indeed be the right time to set up an official repository for the examples in different languages. Perhaps for community scripts in general.

We will discuss that.

tjhb

7,364 post(s)
#23-May-17 23:22

It would also be great to be able to write some examples in F#, once adjustments have been made for the later CodeDOM (and we know how to prepare the installation).

tjhb

7,364 post(s)
#24-May-17 02:57

I take that back. F# scripting is hardly a high priority. Relatively few people will use it. For now, anyone keen to use F# on the Radian object model can try a simple application from Visual Studio instead of scripting. Re-reading that earlier thread I suspect Manifold would rather implement F# compiler services later, than adjust for revised CodeDOM in the near term.

tjhb

7,364 post(s)
#23-May-17 23:43

By the way, it seems to me that the Radian object model has been built so intelligently and natively for .NET that it would be very limiting to stick with a COM-based scripting language such as VBSCript or JScript. (Would we just avoid code that uses generics, for example? How about using ValueSet.AddValueType. I suppose you must have mappings from explicit ECMA types to .NET types built in.)

VBScript and JScript were good default choices for scripting in Manifold 8. Since all scripting went via COM anyway (even when using a .NET language), well, why not keep it basic (unless you needed to write more robust code, or needed more powerful language features). But now that the COM limitation is unnecessary, the balance is shifted. Using a COM language to talk to the Radian model starts to look like a false economy.

For an easy starter language, IronPython looks like the obvious choice now. No fuss (apart from manual installation and a few tiny quirks), very familiar to GIS people (among many others), and with limitless room to grow in coding skill.

I expect V8 JavaScript will be an equally good choice, once its dedicated object model is up and running.

tjhb

7,364 post(s)
#24-May-17 01:51

Another example for first para above. In the ECMA languages we don't have the convenience of using (C#), Using (VB.NET), use (F#)or with (IronPython) blocks to ensure early disposal of resources that implement the IDisposable interface.* VBScript and JScript must instead rely on eventual garbage collection (I think).

(*IDisposable is currently implemented by the Command, Database, Expression, ExpressionParser, GeomBuilder, Root, Sequence, Table and TypeConverter objects. Except for GeomBuilder,** the C# API examples all use using blocks to wrap these resources.)

(**The examples don't wrap GeomBuilder in using blocks, even though it implements IDisposable. TileBuilder does not implement IDisposable, so it can't be wrapped in using. This may be because TileBuilder.EndTile and GeomBuilder.EndGeom dispose of the associated pixel/geometry data implicitly. As the release notes for 9.0.138 put it, when these methods are called, "The {TileBuilder|GeomBuilder} object used to create the {tile|geom} is cleared." The fact that GeomBuilder implements IDisposable but TileBuilder does not may be because a GeomBuilder contains references to Branch and Coord (and possibly Curve) objects, so that .EndGeom needs a .Dispose method to clean these up, whereas a TileBuilder has no resources to be disposed of besides returning the Tile itself. This is all a bit new to me so I don't know.)

adamw

7,102 post(s)
#24-May-17 08:07

Disposing objects in COM languages is done by calling the Dispose method explicitly. Example:

' VBScript

 

Sub Main

  Set app = Manifold.Application

  Set db = app.GetDatabaseRoot()

  Set t = db.Search("t")

  db.Dispose()

 

  ' use table

 

  t.Dispose()

End Sub

This disposes both the database and the table as early as possible.

TileBuilder does not implement IDisposable because the current implementation has no unmanaged resources to dispose of (managed memory can just be dropped on the floor). We might add a blank implementation of IDisposable to TileBuilder simply to future-proof it, so that you can still use the object in 'using' if you want to be sure you reclaimed all of unmanaged resources, whether the current implementation has them or not.

A general note on disposing objects:

Since scripts are written by people and people are imperfect, we had to account for scenarios when nothing is disposed of explicitly and the script just creates a bunch of objects, uses them, then happily ends with the result in hand and a big mess of objects still left undisposed in memory. We put quite a bit of effort into it because we didn't want scripts to be gradually cluttering the system and slowing it down over time. We had some success in that and the current code allows you to dispose nothing with the system automatically cleaning everything up after the script completes. It still makes a lot of sense to dispose objects you no longer need as the script is running, because they might be tying up unmanaged resources for no reason, but the pressure is lower.

The most important objects to dispose of timely are databases, commands, tables, sequences and expressions. Expression parsers are quite bulky as well, but you generally have at most one. And the root object, of course, but you definitely only have one. Everything else is less critical.

tjhb

7,364 post(s)
#24-May-17 10:15

Thanks very much for your comments Adam and for having waded through mine.

Disposing objects in COM languages is done by calling the Dispose method explicitly.

That makes perfect sense (obvious now that you have mentioned it, not to me before--I would have blindly used e.g. Set x = Nothing, but where it mattered that would have been incomplete).

It seems not too much to remember to do that, though if we forget, it seems you have us broadly covered. (I remember the release note about final cleanup, I understand better now.) Likewise we can call Dispose explicitly in .NET languages if we prefer that to using using (or analogues).

Replying partly to your post below: I hadn't meant to criticise the decision to keep COM languages on board, definitely not. I was thinking more from the user side: what best scripting language to recommend to someone starting out with Radian or Manifold 9. (And of course it is not as clear as I had thought. Thanks again.)

adamw

7,102 post(s)
#24-May-17 12:14

I hadn't meant to criticise the decision to keep COM languages on board, definitely not. I was thinking more from the user side: what best scripting language to recommend to someone starting out with Radian or Manifold 9.

Got it.

Regarding the scripting language to recommend to someone starting out / what I (personally) think about the "best" language to use to program Radian in general.

For now, I'd recommend C#. Partly because C# is available by default and IronPython isn't (if Microsoft shipped IronPython or some other kind of Python in .NET maybe it would have won).

After we complete the documentation for the V8 object model / clean and extend it the way we cleaned and extended the .NET object model, I will probably recommend Javascript. It is just easier for small tasks and it can achieve better performance. It could also have natural forms - we all saw what web pages can do, and while the form controls are HTML, the interactions are all done via Javascript snippets. There are lots of libraries, techniques and experience doing such a UI. External tools, books, etc. We do need to add some code to allow using HTML forms from Radian, but then you can use the same forms to ask things from users on the desktop and on the web (and that's going to be very smooth UI, it's not like before when someone saying "you are going to have the same UI on the desktop and on the web" meant "you are going to have a very limited UI" in both places), you can make them pretty via CSS, the sky is the limit.

adamw

7,102 post(s)
#24-May-17 07:47

The single most important COM language is VBScript. If it wasn't for VBScript, we'd stop paying attention to COM languages at this point.

VBScript is really unique in that unlike all other COM languages, it isn't really going away, it is phasing out very, very slowly. A lot of it has to do with VBScript for a long time being a natural stepping stone from batch files in Windows before PowerShell. It remains very popular even after PowerShell (with the operative notion being: why fix something that works, which is particularly true for things like admin scripts). VBScript managed to encroach into places where PowerShell couldn't - for example, MSI scripts are usually written in VBScript to this day. The language isn't evolving, but that's actually a plus in many cases. :-) It doesn't have to evolve, new functionality comes in the form of new objects. And COM surely isn't going away anytime soon, it is used as a base for new technologies all the time (WinRT, for example). In addition to all this, we used VBScript as the default language in Manifold 8.

So, we have to support at least VBScript.

That said, it is not too difficult to make the object model friendly to older languages. Where .NET uses a generic, older languages use dynamic dispatch. Generics are faster, true, but that's fine, if you want performance and you are losing it to not having generics, just switch to a language that has generics. We might have to add a helper object or method here and there, but it is not too much of a hassle.

artlembo


2,818 post(s)
#23-May-17 15:38

I like this idea. They should be fairly easy to pick off. But, just attempting to convert the ValueSet example to VBScript is giving problems. The code below should work, but says the for loop doesn't support this property or method

code' VBScript

Sub Main

    Set ManApp = Manifold.Application

    Set values = ManApp.CreateValueSet()

    values.AddValue "a", 1

    values.AddValue "b", 2

    for each val in values 

      ManApp.Log val.Name

    next

End Sub

adamw

7,102 post(s)
#23-May-17 16:36

We will check what's up (looks like a mapping issue from .NET to COM).

In the meantime, use VB.NET:

'VB.NET

 

Class Script

 

Shared Manifold As Manifold.Context

Shared Sub Main()

  Dim app As Manifold.Application = Manifold.Application

  Dim values As Manifold.ValueSet = app.CreateValueSet()

  values.AddValue("a", 1)

  values.AddValue("b", 2)

  Dim index As Integer

  For index = 0 To values.Count - 1

    app.Log(values(index).Name)

  Next

  app.OpenLog()

End Sub

 

End Class

artlembo


2,818 post(s)
#23-May-17 17:14

thanks, Adam. I will likely wait on the VBScript stuff to be fixed. Those are really easy for me to change out, so I won't mind redoing a couple of hundred of them in VBScript.

artlembo


2,818 post(s)
#25-May-17 13:21

just as an aside, while VB.NET is much more powerful than VBScript, in the above examples you can see the utility of VBScript in terms of its ease of implementation for a non programmer. The code is almost 40% shorter, there are no types to define, and is much more English-like.

That is why, for me, when I have to rip off a really quick script to do something, VBScript is still my language of choice. Also, statements like Dim app As Manifold.Application.... are much easier to write using Visual Studio due to its use of intellisense - it's a pain to write it freeform in 9 (not the end of the world, mind you, but still more cumbersome).

So, while it is good practice to define types, VBScript doesn't care about that, nor does it care around upper case, lower case, or camel case.

When I have some quick-and-dirty data problem I need to fix, nothing is faster to implement than VBScript.

jkelly


1,234 post(s)
#26-May-17 00:19

Hi Art,

I'd be interested to know your thoughts on why Javascript wouldn't be a better option than VBScript for your "quick and dirty" type scenarios?

Cheers


James Kelly

http://www.locationsolve.com

artlembo


2,818 post(s)
#26-May-17 02:12

yeah, javascript would be about the same for it. I find that VBScript is a little more English-like, but javascript is good for quick and dirty stuff.

tjhb

7,364 post(s)
#26-May-17 01:23

I'd be interested to know what a "quick-and-dirty" problem is. What is the objective, who is the client, why do we not care about code quality (data types, even case)? Is it fine if it is wrong, or later falls over? Why not write it well, so we can build on it tomorrow, in a week, a month, a year. Over time is clean code more work, or less?

artlembo


2,818 post(s)
#26-May-17 02:27

these are one-off things. Maybe something that is used more than one time - like 2 or 3 times. I literally bang out about 100 solutions a year for people that tell me "hey, can you do this for me..." VBScript is the perfect solution. Oftentimes, the calls are more desperate like:

Art, I'm screwed, I have a meeting with an investor in 2 days, and I need to show them [insert idea here]. Can you write me a script that does it?

Another use for me is to bang out a spatial algorithm that I think up, and then pass the VBScript code off to our programming staff, who then rewrite it in .NET when it has to be a long term solution.

Another use is when we are in the middle of a project, run into a jam, and I have to quickly script a solution to some road block.

Part of my role as a quantitative geographer is to come up with solutions, not spend tons of hours bullet proofing code - that is what my programmers do, they are fantastic, and better coders than me. So, I spend the time on dreaming up solutions, and others can turn it into .NET code if necessary.

In fact, last evening we had some researchers trying to work with a "big data" problem. I had to string together a number of steps: Simplify, Normalize Topology, Select features with a code of 0, Decompose those selected features to Triangles, Transfer the attributes of adjacent polygons to the triangles, dissolve the triangles, and export it out to a shapefile. They explained the problem to me, and then I spent about 20 minutes writing the script, and then they ran the script on about 30 drawings.

Hope that explains it.

adamw

7,102 post(s)
#26-May-17 06:46

I understand completely where you are coming from. I will offer, however, that VB.NET can be about as quick and dirty as VBScript.

Type specifications are optional:

' VB.NET

 

Class Script

 

Shared Manifold As Manifold.Context

Shared Sub Main()

  Dim app = Manifold.Application

  Dim values = app.CreateValueSet()

  values.AddValue("a", 1)

  values.AddValue("b", 2)

  Dim index

  For index = 0 To values.Count - 1

    app.Log(values(index).Name)

  Next

  app.OpenLog()

End Sub

 

End Class

And you can make variable declarations implicit as well via a one-line option:

' VB.NET

 

Option Explicit Off

 

Class Script

 

Shared Manifold As Manifold.Context

Shared Sub Main()

  app = Manifold.Application

  values = app.CreateValueSet()

  values.AddValue("a", 1)

  values.AddValue("b", 2)

  For index = 0 To values.Count - 1

    app.Log(values(index).Name)

  Next

  app.OpenLog()

End Sub

 

End Class

The body of the last script is the same as VBScript, the only extra lines of code are Class / End Class, Option and the declaration of the Manifold variable, and all of these except Option are auto-generated anyway. Now, I understand that it's not like you are in search of a new language to learn :-) but just in case you will ever have to go the way of VB.NET for other reasons, it might be as terse as VBScript is (and you can trade some of the terseness for performance / type safety / readability, etc).

tjhb

7,364 post(s)
#27-May-17 03:02

# IronPython

def Main():

    app = Manifold.Application

    values = app.CreateValueSet()

    values.AddValue('a', 1)

    values.AddValue('b', 2)

    for index in xrange(0, values.Count):

        app.Log(values[index].Name)

    app.OpenLog()

Even shorter than VBScript, and safer, and fully .NET.

Lorne

653 post(s)
#24-May-17 17:02

Is the example used in Database.SetProperties correct? It appears to be for SetProperty.

A few comments. These things may already have been noted.

1) If a Table in the Project Pane is deleted, should any Drawings tied to a Geom in that Table also be deleted... currently they are not.

2) If a component such as a table is deleted from the Project Pane, it's open window is also closed. However, if the table is deleted via a script, the window is not closed but does not display anything. If the same table is recreated by script, the open window still does not display anything. One must close the window and reopen it.

3) It would be nice to have some of the niceties of v8 in the scripting window. Examples: Hot key Comment and Uncomment and line number display. It would also be great if one could click on a script error and have the code window move to that position. It does appear that the cursor changes to the error line but if the error is not in the display window, then the window does not redisplay at the cursor location.

4) Radian Object model: I'm just completing transitioning a project over to Radian. It has been interesting. I’m not looking forward to converting many thousands of lines of code; however, the new object model is really nice. Having so much in the mfd_root and mfd_meta tables is really powerful.

adamw

7,102 post(s)
#24-May-17 17:31

In order:

The example for Database.SetProperties is indeed, incorrect. It works, but it doesn't call the function it illustrates. That's pretty easy to fix, the intent is clearly visible already. Thanks for noticing!

Deleting the table in the Project pane should not currently delete drawings linked to that table. We are planning to do that for drawings in the same MAP file. In the extreme, we can do that for all databases that are currently opened, but that's perhaps a bit of an overreach already.

We will check what's up with a table window not closing when the table is deleted using a script. Thanks again.

We will implement Comment / Uncomment in the script window. Maybe line numbers, too. I am not sure I understand the problem with the window not redisplaying when you double-click a script error at the bottom - could you elaborate a bit? (Do you mean the window is not scrolling to the cursor location? If so, we'll fix that.)

Thanks for the kind words on the object model. We'll keep adding helpful methods and objects.

Lorne

653 post(s)
#24-May-17 18:20

>>>Do you mean the window is not scrolling to the cursor location?

That is correct... the cursor/insertion point goes to the line containing the error, however, the window is not scrolling to that cursor location. If say, the current code window is displaying the first 50 lines of code, but the error is on line 700, the window does not scroll to 700 although you will find the insertion point has moved there.

Thanks Adam

adamw

7,102 post(s)
#25-May-17 08:52

Understood. We will scroll the script text to the error location.

A follow up on a different item.

If a component such as a table is deleted from the Project Pane, it's open window is also closed. However, if the table is deleted via a script, the window is not closed but does not display anything.

We can not reproduce this. If we create a table, open it, then write a script to delete it and run the script, the table disappears from the Project pane and the table window automatically closes.

Need more details.

Maybe it is the drawing window going blank if you delete the table? That's expected behavior. Although, if the drawing stays open and blank when the table is deleted, it would be logical for it to reconnect to the table when the table is recreated. That's an omission on our part, we should do this.

Lorne

653 post(s)
#25-May-17 13:39

Perhaps the problem I am seeing is related to running the code from a VB.Net form.

I tried writing a short form-based program to test this but it works fine.

I'll look at my application further. It's probably just my poor code.

I did attach some screenshots.

Another point: I don't think this is an error but is probably related to the behaviour discussed here--a progress dialog which opens when running code from the Project Pane.

If one runs form based code from the Project Pane, any component additions or deletions do not display in the Project Pane until after the form is closed.

If form based code is run from an open code window using the task bar run icon, one sees the components changing in the Project Pane while the code is running.

Attachments:
Table_Not_Closing.jpg

adamw

7,102 post(s)
#25-May-17 19:13

If one runs form based code from the Project Pane, any component additions or deletions do not display in the Project Pane until after the form is closed.

We will change this. Running a script from the Project pane will update the component list as if the script was run from the script window (ie, the component list will update as the script is running; we will perhaps still disallow clicking into the component until the script is closed, however - as a service for the script which wouldn't be happy if the user starts modifying the same components it writes).

adamw

7,102 post(s)
#26-May-17 07:28

A follow-up:

The table window for a component deleted via a script not closing is indeed related to (a) running the script from the Project pane, and (b) the script having a form that processes user input. As I mentioned in the previous post, we are planning to change the mechanics of running scripts from the Project pane to be equivalent to first opening a script and running it from a component window. This should make the table window close properly.

Lorne

653 post(s)
#26-May-17 12:28

Thanks Adam. The Tech team is tenacious

adamw

7,102 post(s)
#28-May-17 12:17

We completed the overviews, added one more example, cleaned up the example for Database.SetProperties, extended descriptions for several methods, etc.

We think the documentation for the .NET object model is more or less complete. We are going to keep it up to date. Please report any issues you find or improvements you would like to see via the usual channels, we'll be happy to implement them.

We might add an object model diagram like the one we had in Manifold 8. We will also likely set up a Github repository for examples in different languages, as suggested in the thread.

adamw

7,102 post(s)
#16-Jun-17 19:28

In addition to updating the API doc to the newest cutting edge build (9.0.161.3), we:

  • added an overview topic detailing the use of the object model from VBScript, and
  • translated a good number of examples to VBScript and IronPython.

The total number of examples in the reference topics is now 163. 83 of them are provided in C#, VBScript and IronPython, the rest just in C#.

Enjoy.

adamw

7,102 post(s)
#09-Jul-17 13:54

One more update to the API doc.

All script examples are now provided in C#, VBScript and IronPython (new covered areas: compiled queries, geoms, tiles, schemas, value sets, etc). There are several exceptions where an example does not have a VBScript version - mostly because the showcased function is not of interest to VBScript (like TypeConverter.GetTypeClr).

The IronPython examples have been adjusted to be closer to C# than to VBScript ones in that expensive objects are being disposed of explicitly. There are some differences because the 'with' construct in IronPython is similar but not quite the same as the 'using' construct in C#.

The documentation cites the required version of Radian as 9.0.162.1 which has not yet been released (coming soon), but in practice 9.0.162.0 will do just as well in the vast majority of cases. The requirement for 9.0.162.1 comes from a minor adjustment to Application.CreatePointObj / CreatePoint3Obj / CreatePoint4Obj whose arguments have been made optional for COM languages.

End of list.

tjhb

7,364 post(s)
#10-Jul-17 01:24

Really enjoying this, great stuff. Especially of course that

The IronPython examples have been adjusted to be closer to C# than to VBScript ones in that expensive objects are being disposed of explicitly. ...

Could you expand on the next comment a bit, or point me in the right direction?

There are some differences because the 'with' construct in IronPython is similar but not quite the same as the 'using' construct in C#.

I didn't know this. My halfway guess, looking at cases where 'using' is used but 'with' is not, is that the Python special methods for the Context Manager protocol (__enter__ and/or __exit__) do not behave well if the result of the with expression is None (and is assigned to a target).

In the example for Table.Search it would be more exactly like the C# example to write TrySearch as

# try locating record

def TrySearch(app, table, keys, prefix):

    try:

        with table.Search("mfd_id_x", keys, Array[str]([ "a" ])) as sequence:

            if sequence is None or not sequence.Fetch():

                app.Log(prefix + ": not found")

            else:

                app.Log(prefix + ": found, a = " + str(sequence.GetValues()[0].Data))

    except Exception as (e):

        app.Log(prefix + ", error: " + e.Message)

That seems to work OK (in the context of the rest of the example). Will it fail if table.Search() returns None (it isn't here)?

For the Table.Search, .SearchAll and .SearchBatch methods the documentation says

Return Value

Returns a sequence with a single record if the method found the record. Returns a sequence with no records or a null reference if the method did not find the record.

In what situations can search return a null reference rather than a sequence with no records? Does that depend on the database technology--or maybe whether progressive discovery is still underway?

Sorry for all the verbiage. I may be barking up completely the wrong tree (and it is a small detail).

(Added:)

Another example is Database.GetCache(). Here too the result may be a null reference--as it is for the root database, which helps.

So here, adjusting the IronPython example to use a (second) 'with' expression

def Main():

    app = Manifold.Application

    with app.GetDatabaseRoot() as db:

        with db.GetCache() as cache:

            if cache is None:

                app.Log("Root database does not have cache database attached")

            else:

                app.Log("Root database does have cache database attached")

    app.OpenLog()

results in error

'NoneType' object has no attribute '__exit__'

So basically, yes to the above.

(One remaining question I think: when can the Search() methods return a null reference.)

tjhb

7,364 post(s)
#10-Jul-17 01:56

tldr:

In IronPython, avoid using a 'with' block with an expression that may return None (or a null reference in C#), since the Context Manager protocol is not implemented for None. This results in an error whether or not the expression is assigned to a variable.

adamw

7,102 post(s)
#10-Jul-17 06:40

Exactly, the difference between 'using' and 'with' that I was referring to is in the handling of null objects - 'using' allows them, but 'with' does not.

Table.Search / SearchXxx functions are free to return a null object instead of an empty sequence, many tables do this if they can detect that the search is going to return no records early. This is a useful optimization.

adamw

7,102 post(s)
#10-Jul-17 07:14

An addition:

To be fair, 'with' not allowing null objects limits its usefulness quite a bit. Ie, in production code in IronPython, it would be tempting to start adding wrappers for sequence and similar objects like this (borrowing the code from one of the examples):

# IronPython

#

# Note: running script requires IronPython (IronPython.dll)

 

from System import Array

from System import Exception

 

# class for wrapping None for use in with

class SafeNone:

  Value = None

 

  # initialize

  def __init__(self, value):

    self.Value = value

 

  # enter

  def __enter__(self):

    return self.Value

 

  # exit, dispose of value checking for None

  def __exit__(self, excType, excVal, excTb):

    if not (self.Value is None):

      self.Value.Dispose()

      self.Value = None

    return False # do not stop exceptions

 

# try locating record

def TrySearch(app, table, keys, prefix):

  try:

    with SafeNone(table.Search("mfd_id_x", keys, Array[str]([ "a" ]))) as sequence:

#    with SafeNone(None) as sequence:

# ^^ uncomment to imitate table.Search returning None

      if (sequence is None) or (not sequence.Fetch()):

        app.Log(prefix + ": not found")

      else:

        app.Log(prefix + ": found, a = " + str(sequence.GetValues()[0].Data))

  except Exception as (e):

    app.Log(prefix + ", error: " + e.Message)

 

def Main():

  app = Manifold.Application

  with app.CreateTable() as table:

 

    # override table schema

    schema = table.GetSchema()

    schema.AddField("a""int32")

    table.Design(schema)

 

    # insert several records

    values = app.CreateValueSet()

    values.AddValue("mfd_id", 1)

    values.AddValue("a", 10)

    table.Insert(values)

    values[0].Data = 2

    values[1].Data = 20

    table.Insert(values)

    values[0].Data = 3

    values[1].Data = 30

    table.Insert(values)

 

    # search for record with no key

    keys = app.CreateValueSet()

    TrySearch(app, table, keys, "Searching with no key")

 

    # search for record with key that does not exist

    keys.AddValue("mfd_id", 500)

    TrySearch(app, table, keys, "Searching with key that does not exist")

 

    # search for record with key that exists

    keys[0].Data = 2

    TrySearch(app, table, keys, "Searching with valid key")

 

  app.OpenLog()

# result: Searching with no key: not found

# ....... Searching with key that does not exist: not found

# ....... Searching with valid key: found, a = 20

The examples don't do this and are using 'with' in places where the call is not expected to return None, but the code in real applications bigger than a couple thousand lines of code will frequently have no idea.

Life is imperfect...

tjhb

7,364 post(s)
#10-Jul-17 07:47

Brilliant.

As I said, I didn't know this limitation before (wish I had), and I'm ignorant (have not yet tested) whether the oversight is in the Python language or in the way IronPython hooks up to IDisposable.

Your SafeNone class shows that it's easy to work around post hoc--so ideally should Python code for 'with' be improved, or IronPython?

It's not difficult to approach IronPython developers, I imagine also not impossible to approach BDFL.

adamw

7,102 post(s)
#10-Jul-17 07:57

Your SafeNone class shows that it's easy to work around post hoc--so ideally should Python code for 'with' be improved, or IronPython?

Can't comment on that. It depends on the separation of freedoms between the Python language and its implementations (ie, if 'with' *has* to call __exit__ and is specifically designed to fail for objects that don't have it and NoneType *has* to not have it, and that's all dictated by Python rules, IronPython can do nothing).

Also, SafeNone can be made more generic by calling __exit__ instead of Dispose (and it should always return False - the return should be outside of 'if', that's a typo, I will correct it).

tjhb

7,364 post(s)
#10-Jul-17 09:16

I will pursue this in those channels (subject to my ability) because I think it matters, but will reply by email or in a separate thread. Thanks again for all this Adam.

tjhb

7,364 post(s)
#11-Jul-17 02:42

As simple as possible:

IronPython 2.7.7

def blank():

    return None

repr(blank())

    'None'

with blank():

    pass

Traceback (most recent call last):

    File "<string>", line 1, in <module>

AttributeError: 'NoneType' object has no attribute '__exit__'

CPython 2.7.13

def blank():

    return None

repr(blank())

   'None'

with blank():

    pass

Traceback (most recent call last):

  File "<pyshell#6>", line 1, in <module>

    with blank():

AttributeError: __exit__

CPython 3.6.1

def blank():

    return None

repr(blank())

   'None'

with blank():

    pass

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

AttributeError: __enter__

Possibly interesting that Python 3 bails on __enter__ while Python 2 (and IronPython) cough on __exit__.

But I think this shows it is an oversight in Python, not an implementation issue with IronPython.

That probably means that getting it changed is a question of arguing a PEP for Python 3.

adamw

7,102 post(s)
#11-Jul-17 08:22

FWIW, after reading the proposal for 'with' in Python, I think it would be simpler to persuade the IronPython folks to alter their mapping of IDisposable to __enter__ / __exit__ to handle nulls.

tjhb

7,364 post(s)
#12-Jul-17 02:53

Discussion started.

Manifold User Community Use Agreement Copyright (C) 2007-2017 Manifold Software Limited. All rights reserved.