Subscribe to this thread
Home - General / All posts - Manifold Future Wizards
dchall8
481 post(s)
online
#12-Oct-17 00:11

Are wizards in the future for 9 or are we going to have to become our own SQL wizards? Here's what I'm talking about.

I need to have meaningful labels for the areas in my map. The wizards in Manifold 8 make it fairly easy to create them. Mine need to be 3 lines deep and located in the centroid of the areas on a live basis for when the areas are split into pieces. In 8 with a couple of clicks I can change the font color using the labels toolbar. And I only want to see the labels when the area is selected. In Manifold Future it looks like the labels are currently limited to one line, font cannot be formatted, and locating them at the centroid is a separate process requiring a redrawing of centroids with every area split. And I can't find a way to add a field with a Boolean switch for selectedness.

But before I can do any labels I have to have the correct information in the table fields. When I opened the 8 map file the fields on the related table did not come through. So now I need to figure out how to relate the area table with the data table. Again, the wizard in 8 makes it easy to relate the table to the drawing.

One of the fields in my labels is acreage. I did figure out how to make a new field for area and how to make an active field converting area to acres. For me this was a wildly satisfying achievement. Now how do I round the decimal places down to 3? I'm looking at RoundDecs(<value>,<decimals>):<value>. I tried RoundDecs([WorkingParcels Table].[Acreage],3):Temp, but it did not like my Invalid object reference. I'm not certain what part of that is invalid, but I am frustrated enough to write and ask.

Also in 8 it is easy to update the calculated fields in the table. I don't see how to update it in Future.

Is there a way to save queries to the Project pane?

How can I reuse queries from Manifold 8? They need to link to a table.

rk
273 post(s)
#12-Oct-17 05:41

Call the fn like this.

RoundDecs([WorkingParcels Table].[Acreage],3)

The thing after colon indicates return type of the fn, not part of fn call syntax.

dchall8
481 post(s)
online
#12-Oct-17 14:14

Invalid object reference again.

Dimitri


4,991 post(s)
#12-Oct-17 15:00

Invalid object reference again.

Way too much information. From the above it's not possible to respond with anything but "You are doing something wrong again." It would be more productive to be able to state precisely what is being done wrong and how it should be corrected.

If you want people here to help you find the errors you are making, you have to tell them every last detail of what you do, such as the *entire* text of your query, along with a complete schema for every table involved. That's required to find things like spelling errors in field names, incorrect syntax that leads to referring to things that don't exist, etc.

I would also strongly, firmly urge you to take the advice that, I believe, has been offered many times in this forum: choose a transform template that is similar to what you want (see, for example Round to Decimals in the Transform Templates - Numeric topic) and press the danged Edit Query button to see a fully complete SQL example of how to use a particular function of interest. After you read that example and understand it perfectly, perhaps by making a few changes to it to see what those do, then go and write your own queries from the ground up.

After all, if you have a wizard ready to write SQL for you, why not use it? :-)

dchall8
481 post(s)
online
#12-Oct-17 19:10

Here's the entire text of the query...

RoundDecs([WorkingParcels Table].[Acreage],3)

Schema is attached. Is there supposed to be more to it? I do have a query from 8 to reformat several columns in one pass. I'll look at that to see if I can make it work in Future. No need to continue down this path until I've dug deeper into it.

In Manifold 8, as you know, you can right click on the field heading and format just about everything about how the numbers look in the table. I usually set to decimals showing 3 decimal places, default negatives, separate thousands, and right align. Then when I have the ViewBots showing with total acreage, the ViewBot sums all the computed acreages and compares to the acreages in our appraisal system to find large discrepancies.

I do recall seeing something about how the templates write code. I was not a Radian owner until this week, so there's a lot I've overlooked or not been able to try until now. I have a feeling I'm going to spend the afternoon clicking on the danged Edit Query buttons to learn how they work. Ground up probably means something different to me than what you intend. I'm thinking hamburger...which is how I feel after trying to get SQL to work. But I am willing to learn, so here goes.

Attachments:
Schema.jpg

Dimitri


4,991 post(s)
#12-Oct-17 20:36

Is there supposed to be more to it?

Yes. A function by itself is not a query. You don't need much more but you do need some. See what the edit query button does for the analogous transform template and you'll see what I mean.

I strongly recommend you get a copy of one of Chris Fehily's first rate books on SQL and cruise through it. SQL is not the least bit hard but you do have to honor what few demands it makes in terms of observing syntax. Chris is a great writer and has a knack for explaining SQL clearly and practically in the shortest way. He'll have you up and running in no time, with you thinking at the end, "wow, that was way easier than I thought..."

Note that you can use features in the Command Window to evaluate expressions without writing full SQL. It's a very useful feature, but there too you have to honor the syntax, like using the ? command to evaluate an expression. The SQL Functions topic uses that to show short snippets of examples without writing full queries.

Likewise, the transform dialog's Expression tab allows you to get away with writing expressions that are not full SQL queries, because the Expression tab helpfully surrounds them with necessary infrastructure. But don't confuse such short cuts with writing a real query.

By the way, one reason full info is necessary is because it is easy to overlook something simple, like a typo.

In Manifold 8, as you know, you can right click on the field heading and format just about everything about how the numbers look in the table. I usually set to decimals showing 3 decimal places, default negatives, separate thousands, and right align. Then when I have the ViewBots

More formatting for tables will get added as we cycle through formatting for tables. I love ViewBots too. I'd like something analogous to them to be in 9 as well.

tonyw
460 post(s)
#07-Jan-18 20:56

I'm following this thread, not for joining tables, but dchall8's original task of computing acres and rounding to 3 decimals. dchall8 wrote:

"Here's the entire text of the query... RoundDecs([WorkingParcels Table].[Acreage],3)"

The RoundDecs function was the key I was looking for, I used it like this to compute hectares.

RoundDecs((GeomArea([Geom], 0)/10000),3)

Regarding wizards, M9F is close, other than manually typing in zero for tolerance and typing in the "/10,000" and the "3" for 3 decimal places, the rest of the function is created by clicking on options presented in the Transform dialog. See the 2 attached screen shots. The first screenshot is adding the new field using Edit > Schema. The second are the steps I used to generate the function expression above. Double clicking [Geom] : geom inserted "[Geom]" into the expression.

I don't write code so this is a bit of an accomplishment for me.

I do have a few anomalies though, circled in blue in the screenshot "M9F Rounding areas to 3 decimals.jpg" where the resulting hectares don't match the area is square metres. 111.925 ha from 2,591,310 m2.

Attachments:
M9F Rounding area to 3 decimals.JPG
Schema.jpg

adamw

8,090 post(s)
#08-Jan-18 06:33

I guess the anomalies are cases where geometry values in the Geom field do not really have the area reported in the FEAT_AREA field.

Dimitri


4,991 post(s)
#08-Jan-18 07:21

where geometry values in the Geom field do not really have the area reported in the FEAT_AREA field.

To expand on that a bit... cleaning data and finding errors is a good use of Manifold. There are many data sets out there where you have a shapefile with polygons in it and data attributes that purport to give the area of the polygons but the area field in some cases contains a wrong value for area. Computing the area using GeomArea can find such errors.

A classic case how such errors can arise is a map of US states, where some states such as Michigan are represented with more than one polygon where each such part is a big percentage of the total. If the drawing is then unioned so that there is only one multi-branched area for each state, that is, one record for each state, but the data is not correctly aggregated so that the value in the attribute field is simply the value of the area for the first polygon unioned into the multi-branched area instead of the sum of the areas of the polygons that were unioned, well, you get an inaccurate attribute field.

Given the gross difference between what is in the attribute field and the area actually computed, it could be something like that happened.

But there are other ways in which reported attribute fields might differ from what is computed, for example, based on coordinate system.

If you want accurate "real world" computations of area the drawing must use a coordinate system where areas are accurately computed. Mercator, for example, is not an equal-area coordinate system because regions far from the Equator will have wildly inaccurate areas computed from the Mercator coordinates that define them.

I don't think that's going on in this case because most of the computed areas match the attribute in the drawing. Only some are off, and those few seem to be way off. So... I'd guess that whoever created that data set did some agglomeration to form parcels and did not always agglomerate the attributes correctly.

tonyw
460 post(s)
#08-Jan-18 08:25

Thanks for the idea, I now have two computed fields, one based on [geom] alone which should give my square metres and [geom]/10000 which gives me hectares. Now I can compare the digits as a check on my calculations, the digits should be the same with the decimal moved over 4 places.

So is [Geom] the M9F equivalent of M8's intrinsic fields where [geom] is based on, for areas, area calculated from the geometry and CRS?

Agreed, the [FEAT_AREA] field may have errors. The shapefile I was playing with is from someone else.

Attachments:
geom for intrinsic.JPG

adamw

8,090 post(s)
#08-Jan-18 08:33

9 has no intrinsic fields. A table can have multiple geometry fields, it is just that usually there is only one. The names can be anything, 'Geom' is just the default name suggested in the dialogs / used by dataports during importing and linking.

dchall8
481 post(s)
online
#08-Jan-18 15:59

9 has no intrinsic fields.

Uh oh. I absolutely rely on the ability to use the Selection (I) field to format my maps. For example I have 10 labels set for the parcels on my map. The text color for those labels is themed and set to invisible unless the area is selected. When areas are selected the font color changes to black (or yellow or whatever). As it happens when an area is selected the area line weight theme changes the outline from 2 points to 3 to help visualize the selection.

I also use the intrinsic Longitude (I) and Latitude (I) to sort the table from north to south or west to east. With that order I can do mass numbering for 50 (or thousands of) areas in a blink. Cemetery plots, for example, are almost always numbered in one direction or another. Selecting visually and ordering them by lat/long makes it trivial to number them. This ability has gotten me jobs to support ESRI shops who gave up on the creation of 3,000 to 10,000 parcel subdivisions.

How can I theme labels or order a table for numbering in M9? And if this question is premature in the development timeline, we can put it on hold. But if the answer is that it cannot be done, that would be good information to have.

Dimitri


4,991 post(s)
#08-Jan-18 18:21

Uh oh.

Why the "uh oh?" Intrinsic fields are like a crippled version of a computed field that can do only one thing... if that one thing is your deal and that's all you'll ever need, well, you might not think of them as crippled. But usually people want to do many things with their GIS and they want to be able to do that in a flexible and maintainable way.

Intrinsic fields are far less powerful, more limited and ultimately harder to maintain than even very simple SQL, for example, deployed in computed fields. The sorts of things you describe are easy to do in snippets of SQL, with less hassle, far greater maintainability and so on. Check out the Example: Add a Computed Field to a Table topic to see a small glimpse of what I mean.

Labels are being reworked right now, but within a day or so they'll have the same thematic formatting capabilities objects do, that is, the ability to be formatted based on a field. There are zillions of ways of ordering a table for numbering in 9, more formally done using ORDER BY (easy to remember...) or casually using Filters or just sorting by clicking the column heads of a table.

adamw

8,090 post(s)
#09-Jan-18 07:01

You can actually theme labels right now (9.0.164+) using the Style pane. As Dimitri says, we are adding a lot to that area, so stay tuned.

All intrinsic fields except those related to selections can be created as computed fields.

Intrinsic fields related to selections are slightly different because selections are no longer permanent like they were in 8. A simple replacement is saved selections - they are now just boolean fields, and the Select pane allows quickly saving / restoring current selection to / from them. You can base formatting to follow a particular saved selection and save the current selection to it whenever you want. You can also create a query to select just the selected records and create a drawing based on that query.

tonyw
460 post(s)
#09-Jan-18 16:59

Adam wrote:

Yes. That's what Area (I) was doing in 8 - taking the geometry value and computing its area. A computed field doing GeomArea does the same

Thanks Adam, good to know. [Geom] will be my friend.

dchall8
481 post(s)
online
#15-Jan-18 15:12

I've tried to reply to this for a week, but work keeps getting in the way, and then this text mysteriously disappears. But enough about my personal problems, I just watched the recent YouTube vids for New Labels, Per-Object Formatting, and New Layout Features in 9. I have also read a lot of the Transform help files and cannot seem to create a themed style at the end. I'm sure the process is simple, but I'm missing something. I have a table with a text scratchpad field used for various theming tasks. This field is called Codes. For all the parcels smaller than 100 acres the field has the text value of Clear. For parcels larger than 100 acres the field has the value Green. I would like to have the larger acreage parcels appear in a green color and the small ones as white or clear. I select the drawing, select the Style tab, select Options and Per-Record Style, set up per records with the Codes field selected, click the Drawing tab and click to the right side of the area fill color. Then a dialog appears saying (no field selected). So I select the Codes field and get a single value dropdown showing unique values and a value of 'Green' in the Value: dialog. Now what? If I click back on the color selector the dialogs revert back to the beginning with (no field selected).

Dimitri


4,991 post(s)
#15-Jan-18 15:37

For parcels larger than 100 acres the field has the value Green. I would like to have the larger acreage parcels appear in a green color and the small ones as white or clear. I select the drawing, select the Style tab, select Options and Per-Record Style,

Your task is just a plain, thematic formatting task, not a per-record style task. Thematic formatting is described in the Example: Format a Drawing using the Style Panel topic. Thematic formatting is a basic thing that's always been a part of Radian and then Future, so it appears in some videos as well.

In the above example they use a Number field with equal count method. Follow along in that example, but in your project use your Codes field with the unique values method. For Codes with a unique value of Clear specify transparent color or white. For Codes with a unique value of Green specify a green color. It's that easy.

There are videos on thematic formatting worth watching, like the format city size by population video.

Some background:

When you open a drawing or drawing layer the default operation of the Style panel is to apply the settings to all objects in the drawing. Style allows you to apply the same style properties to all objects the same way, or you can vary the style property applied depending on the value of a field for each object... that's classic thematic formatting as we've known and loved it since the last century. There are good topics and examples on that in the documentation.

What per-record style allows you to do is to override the style instructions that apply to the entire layer for only some objects, those which you specify. That's new, and how to rig up those exceptions is what the new video covers.

dchall8
481 post(s)
online
#15-Jan-18 18:42

That all came back to me as I read it, so thanks. Apparently I was following along closely earlier in the development, but moved on to other features before developing any muscle memory...or regular memory, of doing simple theming.

So with this in mind what I really want to do is color every parcel a single color of green except for those with a parcel ID number = 0. Zero is a place holder for those parcels which need my attention find the real values. By making those parcels a different color I can see them as I traverse the county on the map. I can make them all a single color of green from the palate, but when I click on the PROP_ID field and select unique values, all 31,000 parcels reset to the default gray fill color instead of the green color I just picked.

Dimitri


4,991 post(s)
#16-Jan-18 05:35

but when I click on the PROP_ID field and select unique values, all 31,000 parcels reset to the default gray fill color instead of the green color I just picked.

There are steps missing in the above. You have to tell us *every* step you are doing.

So with this in mind what I really want to do is color every parcel a single color of green except for those with a parcel ID number = 0.

OK, well, that's different than just coloring transparent if a Code field value is 'Clear' and coloring green if a Code field value is 'Green.' But that also is easy to do using thematic formatting for fill color.

The field you will use for thematic formatting is ID (let's assume that is the name of the parcel ID field you are creating).

Use equal intervals with 2 breaks and press the Tally button. You now have two intervals. The first one should have 0 as the number where the break starts and the next one will have some bigger number. Double-click into that bigger number and change it to 1. Set the color for the 0 interval to transparent and the color for the 1 interval to green. For the Fill rule, choose closest lower value.

What you've done is create a thematic format that has two intervals. The first interval starts at zero and and ends where the next interval starts. The next interval starts at 1 and covers all the rest of the values. You're coloring everything in the first interval transparent and everything else green. Press the update style button and you're done. Takes maybe ten seconds.

---

You can do a lot more with this once you get your head around it. It is far, far easier than you are making it out to be. Suppose for example you want everything with an ID of 0 to be transparent and all the rest of them colored in a series of shades depending on the size of the parcel. There are two ways of doing that.

One is to do a thematic format on ID using equal intervals or equal count with a reasonable number of intervals, say, 8, and color it with the CB spectral palette using a Fill rule of Interpolate. Then you turn on per-record style.

Open the drawing's table and select a record with an ID of zero. Back in the map, color that one record transparent. Back in the table, copy the JSON style string that appeared in the Style field for that record, select all records with an ID of zero and with the "selected records" box checked, use the Transform dialog to put that same JSON string into all of the Style cells for all of the selected records. You've just copied that per-record style into all records that have an ID of zero.

The result is that your drawing will be formatted beautifully in a range of colors using the CB spectral palette for all objects except those that have an ID of zero, which will be transparent.

If you don't know how to use the Select and Transform panes to instantly do the selection and copy moves I give above, please learn how to do that and don't cheat yourself by trying to use those panels without first spending some quality time to learn how to use them. If you learn how to use them the procedure I give above can be done in, like, five seconds. If you try to do it without first learning the basics it takes forever and a totally easy thing will be difficult and frustrating.

Dimitri


4,991 post(s)
#16-Jan-18 07:37

Just in case anybody will try doing this and needs every step...

Back in the map, color that one record transparent.

OK... you have one record selected. In the map the corresponding object is shown in red selection color. [making a note... time to bump up priority for zoom to selected features...]. Alt-click on that selected object and data for it appears in the Record panel. You can now set the style for that record/object in the Record panel's Style tab.

There are two ways of doing that.

Never got around to mentioning the other way. You do that purely in thematic formatting. First set up as many intervals as you want based on the ID field with the colors palette you want. Whatever is the lowest numbered interval, change that lowest number from 0 to 1. Next, hit the * button in the intervals list to add another interval with the value of 0. That adds automatically to the top of the list with a color of black. Change the color to transparent and hit the update style button. Done.

Come to think of it, that second way is even faster than the first way...

dchall8
481 post(s)
online
#17-Jan-18 17:36

I'm so glad I asked the question. THANK YOU! In 13 years I never thought about using equal intervals using only 2 interval breaks to theme my map. That did it in M9, but...Armed with that method I just rethemed the colors in my M8 project.

Okay do you mind if I stretch that idea? I wasn't clear in my question about changing the theme on elements when I select them. I want labels to be invisible until I select a parcel (ctrl-click). When the parcel is selected I want to see the active labels for that parcel. When the selection changes and the selected parcel is no longer selected, I want the formerly visible labels to disappear. In M8 I did this using the Selection (I) field. It has been hinted that the same M8 intrinsics are buried in the Geom element of the table and can be used; however, you have to know what you're looking for before you can use it. I cannot find it in the voluminous manual. I believe a new field would be needed where the value toggled from false to true when the object was selected. Then the theming would be based on the value of that field. Am I on the right track?

Dimitri


4,991 post(s)
#18-Jan-18 05:45

Am I on the right track?

Yes. Two ways to do that: save the selection and do a thematic format on that saved selection field (not as dynamic as you might want), or do a computed field that reports if the mfd_id value for that record is in the set returned by SelectionKeys for that table (computationally inefficient). The latter approach is OK for small data sets but probably way too inefficient for very big ones. But then again, with big data sets it is probably a fundamental mistake to approach them using point-and-click selection as a key interface strategy.

A bit of discussion for those interested in how this should evolve: playing around with dynamic selection often leads one into the interplay between casual, happy-go-lucky, interactive GIS and the much more serious world of Big Data.

9 does the big, serious data deal first because if you have that you can do anything, including casual stuff, but if you start with small-time, happy-go-lucky, casual stuff you cannot build big capabilities on that.

Having built the big foundation, 9 now is using the "anything is possible" capability of the Radian engine to focus on "interactive GIS," to make it easier to do simple things in a highly casual, ad hoc way. A good example is the increasing roster of abilities to more easily use Manifold-style selection in casual settings, where a more DBMS-centric person would automatically reach for SELECT and real SQL.

There is a slight risk to that in that once you make informal methods easier to apply people will often use them inappropriately instead of learning better, more powerful tools, sort of like how people use Excel in a wildly inappropriate manner as a DBMS instead of taking the slight extra step of learning a DBMS, not even a simple one like Access. Or, perhaps how an unskilled cook might slather ketchup all over everything instead of learning to prepare intrinsically appealing food.

But you can't let the risk that some folks will go overboard using casual facilities to stop you from making those available for when they are the right tool to use. A dollop of ketchup with an In-N-Out burger and fries? Sure. If somebody else wants to pour ketchup on a fine steak that's their problem. :-)

We could make casual use of selection more applicable in wider, more dynamic settings with just a few small tweaks, which no doubt we'll do as part of the "interactive GIS / cartography / presentation" campaign now underway for 9. Add an IsSelected function and you instantly have the computed field equivalent of Selection (I) from 8 with no need to figure out how to write a snippet of SQL that reports if a value is in a table. Add a Selection filter to drawings, like there is for tables, and you don't even need a computed field or thematic format on that. Click the filter on to show only unselected records and you are done.

I'd add one more element, saving of filters as properties for components. I like it when I can see everything about a component in a table, such whether it has a StyleOverrides property in the mfd_meta table. So, I'd like to see if a filter is in play by whether that property appears in the mfd_meta table for that component. That would also be a way of making such filter selections persistent, which is what you want if you are going to use them as a part of the control panel you set up for using a drawing.

tjhb

8,227 post(s)
#18-Jan-18 06:16

Nice answer Dimitri, but can you explain that last paragraph a bit more?

First, what do you mean by a filter?

Do you mean a table filter, such as <field> >= x?

How would that be updated, for an infinite dataset, live?

I don’t think that can be the point of what you’re saying but if it is, I think a property is not appropriate here at all. Stream elements + properties mixes paradigms badly (and can only slow things down).

And why would you want that property? (How many others?)

Dimitri


4,991 post(s)
#18-Jan-18 07:37

First, what do you mean by a filter?

I mean like the setting to show only selected items, all items or only unselected items. I suppose I also would mean a rule based filter such as <field> >= x.

How would it be updated for an infinite data set, live? The same way it is now, applied to whatever is in the open window. We can see examples of that going on today, with properties such as StyleOverrides that exist in the mfd_meta table but which function only in the context of what is displayed in an open window.

And why would you want that property? (How many others?)

All the others that one can tinker with in dialogs.

By the way, my answer was not quite correct because the second way I give of implementing a 9-ized intrinsic selection field as a computed field will not recompute on the fly as objects are selected/unselected interactively.

The right way is to leave the Select panel open to the Saved tab. Select an object and save that to a saved selection. OK. Select that saved selection in the Saved tab. Now, as you select or unselect objects in the map just click the Capture button and that updates the saved selection. Thematic format on that saved selection field so that True uses transparent.

You can then go back and forth using that saved selection field to make objects appear/disappear based on whether they are in that saved selection or not, and you can also have multiple selection sets in different saved selections that can been combined and recombined as well. Would probably make a useful video...

The above, of course, being easy assumes the operator has learned how to work Manifold and has acquired the skills an intermediate user should have in muscle memory. If the operator is still fumbling around and doesn't realize a Select panel can be left open or that it is one click to Capture, well, then the process will be clumsy. But that's just the clumsiness of a beginner which goes away in the high speed and flexibility available to a more skilled user.

I emphasize that latter point because Manifold is designed to be very fast and effective in the hands of somebody who has learned how to use it. Better to have lavish power and great efficiency for skilled users than to condemn everyone, forever, to limitations because beginners might be clumsy in the initial learning phase.

I have to admit I fell into that trap just now, by slanting what I wrote in a way that made it more analogous to how Release 8 did things. That was wrong, because although the 9 way is different once you learn how to do it correctly it is more powerful, more flexible and faster and easier to do what you want. What I should have written was, "here is how to do it in 9..."

tonyw
460 post(s)
#08-Jan-18 18:06

Adam:

9 has no intrinsic fields

Thanks Dimitri, I had read previously about the Geom field and that 9 wouldn't have Intrinsic Fields but didn't fully grasp the significance. Intrinsic fields were useful in M8, for me mainly for areas. As planning on a project progressed, areas of interest would change, get added to, have areas subtracted. In the end, the inevitabe question is "OK, now how big is it"? My definitive reference was to calculate hectares based on the Intrinsic field Area (I).

So if the Geom field: "...stores the coordinates that define the locations of points and the locations and shapes of lines and areas".

then in the absence of Intrinsic Fields, can we rely on the Geom field for area? It seems the area is a function of how the area is defined and inextricably linked to that definition in the Geom field? Maybe I'm missing something in the semantics?

Dimitri


4,991 post(s)
#08-Jan-18 18:23

To repeat, the ability to use small snippets of SQL in 9 is far beyond... way, way far beyond, intrinsic fields in 8. Whatever you did with intrinsic fields you can do far better in 9, with more options, the simplest of which and most akin to intrinsic fields in 8 being the use of calculated fields in 9. See the Example: Add a Computed Field to a Tabletopic for some elementary examples, including getting area. If you don't have 9 you can try out those examples in Viewer.

Edited to add quickly... the examples in that topic show a clear instance how intrinsics in 8 are limited compared to the use of computed fields with snippets of SQL in 9. The X(I) and Y(I) intrinsics in 8 give the "center" of the object. But if you have been doing GIS you know the notion of "center" is not so simple. If the object is a doughnut shaped area or a slim half-moon crescent shaped area or even just the US state of Florida, the "center" falls outside the object. Happy with that? Maybe yes, maybe no. Maybe you want an Inner centroid or a centroid by weight, as discussed in the ever-popular topic on centers.

But if you want those, too bad if you are using 8 because you don't have those with intrinsic fields in 8 that are limited to the list that is what it is, useful, but a short list. With 9 it's trivial to get what you want. Instead of GeomCenter in the computed field you use GeomCenterInner or GeomCenterWeight. In moments you have thousands of variations at your fingertips for whatever you want.

[Edited to correct typos.]

adamw

8,090 post(s)
#09-Jan-18 06:46

Answering this directly:

My definitive reference was to calculate hectares based on the Intrinsic field Area (I).

So if the Geom field: "...stores the coordinates that define the locations of points and the locations and shapes of lines and areas".

then in the absence of Intrinsic Fields, can we rely on the Geom field for area?

Yes. That's what Area (I) was doing in 8 - taking the geometry value and computing its area. A computed field doing GeomArea does the same (but can be adjusted to, say, round to the specified number of decimals, etc, and can in general do anything and you can have as many of those fields as you want doing different things opposite to choosing from a fairly small pre-defined set).

Dimitri


4,991 post(s)
#08-Jan-18 13:14

So is [Geom] the M9F equivalent of M8's intrinsic fields where [geom] is based on, for areas, area calculated from the geometry and CRS?

No. A geom contains the actual data. These are basic concepts that should be internalized before all else.

1. All info in 9 is stored in a table. The table is where the data is. One of the data types a table can store is geometry data, stored in type geom. A geom field stores the coordinates that define the locations of points and the locations and shapes of lines and areas.

2. Drawings don't contain any data. They are just a different way of viewing data in some table. A drawing's properties tell it what table to take data from, and what geom field in that table the drawing should use to draw the geometry for each record.

Read the Drawings topic for some good illustrations that show how that works.

3. Because drawings are just viewports of a different kind into a table, you can have multiple, different drawings that use data from the same table, either from the same geom or, if a table has more than one geom field, from different geoms.

Topics on that: Example: Two Drawings from the Same Table and Example: Multiple Drawings from the Same Table.

See also the new Example: Add a Computed Field to a Table topic, where there is an example of creating a computed field that generates circles in a drawing. There is also an example of a computed field that converts areas in square meters into areas in square feet.

Dimitri


4,991 post(s)
#12-Oct-17 14:48

A lot to discuss here...

Are wizards in the future for 9 or are we going to have to become our own SQL wizards?

There are very many wizards already there, in the form of transform templates where you can click the "edit query" button to have it write SQL for you (very wizardly...).

If you prefer something more free-form than the SQL the system writes for a template, use the query builder.

I need to have meaningful labels for the areas in my map.

Labels are still at Radian level, not where they are going. Keep your finger on the pulse of changes with Future. As formatting adds labels there will be far, far more capabilities with labels.

I have to have the correct information in the table fields. When I opened the 8 map file the fields on the related table did not come through. So now I need to figure out how to relate the area table with the data table. Again, the wizard in 8 makes it easy to relate the table to the drawing.

I don't quite follow the above but in general Radian / Future / 9 has far more capabilities than 8. There is no such thing, by the way, as an "area table" that is different from a "data table," so I don't understand what you mean by that. Everything is a table. A drawing is a viewport into a table. If there are any fields you can work with in a query using a drawing's name, those fields are in that drawing's table.

Also in 8 it is easy to update the calculated fields in the table. I don't see how to update it in Future.

Future automatically updates computed fields. I note that "calculated field" is not Manifold terminology. The right term is "computed field." Use the precise terminology so you can follow along with examples and report exactly what you've done in the forum.

You can add a field that has set field values, and you can do something related, but different, which is to add a computed field. There are two examples that show the difference:

Add a field to a table and fill it

Add a computed field to a table

If you just added a field to a table and filled it, but did not create a computed field, well, there is nothing to update because you've just filled that field with static values, not computed values. It's an ordinary field and will have white background. If you created a computed field as set forth in the second topic above the field will have a light gray background because it is not editable... the values in there are computed based upon some expression.

Is there a way to save queries to the Project pane?

Queries you create in the project pane are automatically saved. See the "to create a query" list of instructions at the very top of the Queries topic.

Do you mean, is there a way to save a temporary query that I wrote in the Command window into a query component in the project pane? Yes, of course:

Copy the query text. Create a new query in the project pane using whatever name you like. Open that query and paste the query text into that query. Done. As noted in the Queries topic changes made to query components are automatically saved.

As advised in the Queries topic, the usual way to create a new query is to create a new query in the project pane, to open the query and then to go forth and write whatever SQL you want in that query. But if you didn't do that but instead just launched the Command Window in an ad hoc way, no problem. Copy whatever SQL you've written and paste it into a query component.

How can I reuse queries from Manifold 8? They need to link to a table.

Open the query text and re-write it for Radian SQL. Depending on the query that can be trivial or it can be significant.

They need to link to a table.

Not sure what you mean by the above, as queries are operations on tables, so they all link to a table. Could you describe with more detail what you had in mind?

dchall8
481 post(s)
online
#12-Oct-17 23:54

Thank you for the reply. As for wizards, I prefer something where the options are already built into a dialog box, a la Manifold 8. For example if I want to format the text or numbers in a field, I can right-click the field name, select Format from a context menu, and make a few selections in a dialog box type of wizard. Other wizardly methods in Manifold 8 need no dialog box. If I want to sort a table based on a field I can click or shift-click on the field name. If I want to rotate a mapped object I can select the object and use the tools at the bottom of the window to rotate. If I want to number 5,000 parcels by a factor of 3, from south to north, it's only a few clicks. So I guess my question is will Future have dialog box or the intuitive types of transforms? If I have to program my own wizards, I'll likely stick with version 8. It's not as fast as Future, but it does what I need with no big learning curve.

On my other question, I lost you in the same place two times - must have been my wording. In Manifold 8 I use the Table Menu > Relations to connect two tables using a key field. One of these tables is my map parcel drawing table; the other is information table from our office computer. The map parcels table holds the geometry, a property identification code number (PID), and three other fields I have created. Two are scribble space for use on the fly. The other one is a computed acreage field. All the important information about the parcels comes from our office database. Once a week I query the office database to get a subset of data needed for the maps. I take that data through Excel to a .xls file. Then import the .xls to 32-bit Manifold, copy, and finally paste into 64-bit Manifold. Then I "relate" that table to my parcel map using PID as the key. This last step is what I'm trying to do here. I understand 'relate' as it applies in Manifold 8, but I don't see a template for it in Future.

adamw

8,090 post(s)
#27-Oct-17 08:50

A late reply:

Yes, we will have formatting for field values.

We already have sorting in cutting edge builds, including on multiple fields.

We allow rotating an object from the Transform pane (the Rotate template).

We don't have a specialized UI for relations, these are done using queries. Instead of using a dialog to add a relation from table P to table Q on shared field F, you write a query like:

--SQL9

SELECT * FROM P INNER JOIN Q ON P.F=Q.F;

...and operate off that table. In this particular place we did not carry over a feature from 8 because the design was a bit awkward (bringing fields from one table to another like 8 does has to make tons of assumptions and allowances - ie, what to do if a key value for a particular record matches more than one record? different scenarios call for different answers... there are also multiple valid ways to go when editing values brought from another table or when editing the values of shared fields, with the feature forcing a single choice) and there is a replacement feature which is only a bit more laborious to use (the initial query above is very simple) and is much more flexible (you can augment it easily to do things like summary records, reject matches based on values of non-key fields, etc).

dchall8
481 post(s)
online
#02-Nov-17 13:45

How do you undo a join like that? The beauty in 8 was that in less that 5 seconds I could undo last week's relation table and redo it with this week's updated table.

Dimitri


4,991 post(s)
#02-Nov-17 16:21

I could undo last week's relation table and redo it with this week's updated table.

I'm not sure I understand what you mean. Instead of an "undo" don't you mean you could just update the table with this week's data? You could do that in Future as well.

If you could give a specific example with step by step workflow we could talk about how, exactly, you would do that now.

dchall8
481 post(s)
online
#03-Nov-17 22:00

Maybe you have a better word than undo.

Undo a table relation In Manifold 8:

  1. Put focus on the drawing table which has been previously related to a second table.

  • Menubar>Table>Relations
  • Highlight the current relation

  • Click the red X in the Table Relations Tools

    That process brings me back to a pristine drawing table to which I can relate to the most recent data table from our master database. After you do it a few hundred times it goes very fast. Downloading the table and processing it through Excel and Manifold 8 (32 bit) takes the longest time.

  • tjhb

    8,227 post(s)
    #04-Nov-17 01:18

    Perhaps the answers here are becoming more complex than they need to be. I think you were perfectly clear when you said

    In Manifold 8 I use the Table Menu > Relations to connect two tables using a key field. One of these tables is my map parcel drawing table; the other is information table from our office computer. The map parcels table holds the geometry, a property identification code number (PID), and three other fields I have created. Two are scribble space for use on the fly. The other one is a computed acreage field. All the important information about the parcels comes from our office database. Once a week I query the office database to get a subset of data needed for the maps. I take that data through Excel to a .xls file. Then import the .xls to 32-bit Manifold, copy, and finally paste into 64-bit Manifold. Then I "relate" that table to my parcel map using PID as the key. This last step is what I'm trying to do here. I understand 'relate' as it applies in Manifold 8, but I don't see a template for it in Future.

    So let's imagine one table called [Parcel geometry], with fields [PID], [Geom], [a], [b] and [acreage]; and a second table called [Parcel data], with fields [PID], [x] and [y].

    In Manifold 8 you would open [Parcel geometry], choose Table > Relations, add a relation on [PID] -> [PID], and check [x] and [y] to "borrow" those fields from [Parcel data]. To refresh the relation for new data, you would delete the defined relation and start again.

    To implement the nearest equivalent for Manifold Future/9, following Adam's suggestion, you would use File > Create > New Query, and give it text like this (with adjustments for actual names).

    SELECT [t1].*, [t2].[x][t2].[y]

    FROM [Parcel geometry] AS [t1] LEFT JOIN [Parcel data] AS [t2]

    ON [t1].[PID] = [t2].[PID];

    Run the query. At a first approximation, that's all there is to it.

    When you have a new version of [Parcel data], just run the query again. There's nothing to undo or redo--the only thing to check is that the name or [Parcel data] has not changed (adjust the query text if necessary).

    (You could also incorporate automatically calculating acreage here, but let's leave to one side.)

    You will probably also want a drawing that shows the data from this joined table. That's slightly more complicated but not difficult.

    artlembo

    3,077 post(s)
    online
    #04-Nov-17 19:56

    that is good advice, and a cool way to create a sort of VIEW inside of MF. However, and I could be wrong, in doing this we can see the new drawing, and even symbolize the drawing, but I don't believe we can issue an Alt-click to get information on the drawing. I think this is due to the indexes not coming over in the query. Any thoughts on how we can use the info tool on this?

    tjhb

    8,227 post(s)
    #04-Nov-17 20:56

    You are exactly right Art.

    The join above converts the BTREE on mfd_id in the left table (Parcel geometry) into a BTREEDUP, as it must. (An inner join would do the same. The reason for the 'must' is that the compiler needs to know the schema, including indexes, before it processes the data. Because duplicates can occur, it must work on the basis that they might.)

    A BTREEDUP with no BTREE means no selectiing objects.

    There's not much we can do about this, using just the joined result.

    An aside:

    It seems worth noting that if the query table had no RTREE, and we tried to make a drawing based on it, we'd get a message, with the option to create a temporary RTREE on the query table to allow the drawing to work.

    Perhaps something similar could be done where the query table has no BTREE--again a message, and an offer to create a temporary BTREE on a field in the result table, although the attempt would have to fail if the chosen field did not contain unique values.

    End of aside.

    What we need to do here is create a new static table, with a defined BTREE index, using either CREATE TABLE... INSERT INTO..., or SELECT INTO... ALTER TABLE....

    Second aside:

    Which in practice is currently slightly tricky. If we know that values in mfd_id are unique, after the join, then all well and good. Sometimes we do know, but this is data dependent. (Here it depends on whether any row on the left can potentially be matched with multiple rows on the right. If FID is unique in both tables, then we know there will be no multiple matches, and mfd_id in the result must be unique.)

    It values in mfd_id are not unique after the join, or we don't know, then we will want to exclude mfd_id from the projection into the new table. That is, instruct CREATE TABLE to add a new mfd_id field, and a BTREE index on it, but leave out the source mfd_id field from the INSERT INTO, so that the new field will be filled automatically with unique values (and the index handled accordingly).

    A side effect is that we need to know, and list, all the source fields that we want, to exclude mfd_id. That's OK, it just limits what we can do generically by SQL alone. Which is just to say that a generic, bulletproof approach requires a script. OK.

    There is the earlier discussion about being able to exclude mfd_id (or other named fields) from the projection list by some new syntax. That would be good, but again there is a possible wrinkle that gets in the way of doing things generically.

    What if there is no mfd_id field in the source table(s)? This can be true for example for data imported from SQLite/GPKG, where the source FID field is used instead, and no mfd_id is created. Perhaps the proposed new syntax could exclude named fields if they exist, and ignore the exclusion(s) otherwise? That would be fine I think.

    (I have been thinking about these topics a lot in the last few days, a bit of thinking out loud here.)

    End of second aside.

    I'll post the code we need here (for now) in a moment.

    dchall8
    481 post(s)
    online
    #04-Jan-18 15:31

    I finally got around to trying this query as follows for my situation...

    code

    SELECT [t1].*, [t2].*

    FROM [WorkingParcels Table] AS [t1] LEFT JOIN [PACS 2018-01-03] AS [t2]

    ON [t1].[PROP_ID] = [t2].[prop_id];

    Using the relation wizard in M8 this process took seconds from start to finish. I'd call it instantaneous. Running that query in M9.0.164 the process took all afternoon to get 1/3rd of the way through the 31,000 records. The Working parcels table has 6 fields (including geom) that I want and 14 labels fields which I would like to get rid of. The PACS table has 42 fields that I need daily. I let the query run all night hoping there would be a record of how long it took to run, but it did not. The time to complete the query is not acceptable for daily use, but I wanted to see what it looked like at the end. I was expecting the results to be like in M8 - that is the Working parcels table to contain the fields from the joined PACS table, but it does not. The results in the Results tab of the Command window show the tables joined. I could export that and reimport the table, but that's getting still more convoluted.

    Is there something about the way this was written that it did not run instantly? Normally I update (relate) my tables once a week, but sometimes the boss wants updates, changes, and more updates in an afternoon.

    artlembo

    3,077 post(s)
    online
    #04-Jan-18 15:36

    the query looks good, but the time seems excessively long. Do you have an index on the prop_id fields?

    this is another good reason for the relation tool in M8, but that notwithstanding, I think if you added a btree index, things would go faster. And, the prop_id is a unique field, right?

    dchall8
    481 post(s)
    online
    #04-Jan-18 16:04

    Working parcels table has a mfd_id index as well as an obsolete Manifold 8 ID number. The M8 ID field is one of the fields I would like to delete, but I don't know how that's done. The help doesn't help with that. The prop_id field is unique in PACS but PROP_ID is not necessarily unique in Working parcels. When I find erroneous duplicate areas in Working parcels with the same PROP_ID, I take a second to Union them. There might be as many as 200 duplicate PROP_ID areas still to fix. But this inconsistency was not a problem for the relation tool in M8. With the M8 relation tool the PACS records attach to all of the areas with matching PROP_ID in Working parcels.

    I have seen y'all talking about btree and rtree indices, but those discussions are way over my head. I'm trying to keep my maps updated, not learn how to program the computer.

    artlembo

    3,077 post(s)
    online
    #04-Jan-18 19:02

    Having an index on the field should make the JOIN go much faster. I will have videos on that in a couple of weeks.

    In the meantime, you can check out this video where I am using indexes to select out of 13 million records.

    Indexes are very fundamental when working with large database tables. I assume that creating indexes will be made easier in the future, but if you read up on the schema/edit capabilities you will see that isn’t too bad now

    Dimitri


    4,991 post(s)
    #05-Jan-18 06:01

    The M8 ID field is one of the fields I would like to delete, but I don't know how that's done.

    Open the table and launch Edit - Schema. Highlight the field to be deleted by clicking on it and then press the Delete key or the Delete button in the Schema dialog's toolbar. Adios, field! :-)

    I have seen y'all talking about btree and rtree indices, but those discussions are way over my head. I'm trying to keep my maps updated, not learn how to program the computer.

    Indexes are not about programming the computer. They are a basic part of the infrastructure of any table you want to be editable, selectable and otherwise usable in the usual way. You must have an index to be able to edit a table or to select in it, and indexes also can dramatically speed up operations. Many dataports will automatically add an mfd_id field and mfd_id_x index for you, but some don't since they are often used in situations where informed users want control over such matters.

    You don't need to know much about indexes: ensure every table has a mfd_id field in it and, if an mfd_id_x index was not automatically created for that table when it was imported, create one as shown inthis topic. That is extremely easy to do in seconds and involves no programming. Given an mfd_id field with an index on it, which is pretty darned near automatically done for you in most cases, you never have to worry about indexes.

    If you are using the system in default ways you'll never have to think of rtree indexes, which normally are used only with spatial data like geoms. If you are creating your own spatial arrangements there is not much to think about, in that whenever you add a geom type field to a table that you later want to use in a drawing or whatever, take a moment to add an index to that table on the geom field that is an rtree index. There are many topics that show that process, also takes only a couple of seconds and totally easy to do. (I won't spoil the fun of reading the documentation by listing those topics here... :-)

    dchall8
    481 post(s)
    online
    #05-Jan-18 18:20

    Thanks for the tip about editing fields in the schema window. I bumped into the schema window looking for something else and got the table edited right after posting my previous question. So although I may seem hopeless I'm not always completely helpless.

    You don't need to know much about indexes:

    Apparently I need to know much more about them than I did when I lived in blissful ignorance from 2004 through 2017 (using Manifold 4-8). For now I think the point is (temporarily) moot because the query inner join needs more work to make it practical. For some reason it is still taking hours even after indexing both tables. In my ideal future there will be a table relations dialog box that obviates the steps as of the current M9.0.164.1. Maybe someone can write an add-in for that.

    (I won't spoil the fun of reading the documentation...

    That fun has already been spoiled by the documentation not being updated as fast as changes are being made in the program. This is not a complaint. I'm just pointing out that reading the fine manual is not as much fun as it once was just a few months ago I call it undocumented features and go on with life - the life of asking about them here.

    Dimitri


    4,991 post(s)
    #06-Jan-18 14:08

    the query inner join needs more work to make it practical. For some reason it is still taking hours even after indexing both tables.

    As compared to INNER JOIN in PostgreSQL or in flagship DBMS products from, say, Oracle, Microsoft, or IBM, INNER JOIN in Manifold is no less practical than any of them, which means it is very practical. It is as fast as or faster than INNER JOIN in the others, which means it is really fast.

    To help give you confidence in INNER JOIN, consider the performance reported by Adam... a tool that generates 29 million records as a result in two minutes is probably not the cause of a performance problem that generates only 248 records in two minutes.

    I haven't seen the text of your query so I have no way of knowing whether the way you used INNER JOIN in your query is either correct or is the best way to accomplish what you've intended. Taking a look at that query is where to begin. If it is a one-in-a-million bug, looking at the query text will help find it. If it is just a query missing a chance to do something really fast, well, looking at the text will find that as well. No downside.

    Let's begin with a close, precise and thorough look at the schema of the table, at what data the table contains, at the SQL text of the query, and a statement of what that query is supposed to accomplish. If something is inoptimal in the query, that's a very good thing because such issues can be fixed in minutes by adjusting the query.

    ---

    By the way, it really is true that for most purposes you don't need to know very much about indexes. Most people will use Manifold for years in interactive use who won't ever need to know anything about indexes than the need to add one to a table that doesn't have any indexes.

    Most people also will do GIS for a lifetime without ever needing to use an INNER JOIN, either. But if you are writing SQL and doing things like INNER JOINs that's not you. You'll need to know a bit more about both indexes and SQL than the average GIS user.

    Dimitri


    4,991 post(s)
    #06-Jan-18 14:28

    I'm just pointing out that reading the fine manual is not as much fun as it once was just a few months ago

    Well, sure, but that's the deal with pre-release software and, really, it is a compliment to how fast the product has evolved. If you are willing to work off release notes you can leverage every new build the moment it is released, but that means for rapidly-evolving software there's a lot to keep updated on.

    When working with any pre-release software, and that includes beta builds, Cutting Edge builds and pre-release builds of any kind, the "manual" is the collective series of build notes that were published in the forum. You have to stay caught up reading those so you can see how the product has evolved from the last synchronized product / manual ensemble.

    For example, the Radian Studio manual exactly matched the product when Radian was officially released in August. Since then there have been well over 300 cited improvements, most of which included many smaller improvements not explicitly called out and which collectively have required many thousands of changes in the documentation, including changes in the over 4000 screenshots currently in use.

    The manual has kept up reasonably well, typically lagging only a couple of weeks or so in most cases, and if you have read the release notes you can fairly easily tell where specific topics are not up to date. Of course, if somebody has been working with Cutting Edge builds or pre-releases without reading the release notes they risk hitting an out-of-date topic and confusion. To deal with that, search in the forum to find where that item was discussed in a release note (that's what I do). Avoiding such confusion is also why the company puts a lot of effort into keeping the documentation tracking the builds as rapidly as possible.

    When 9 is published as an official release and not a pre-release the manual will be totally in sync again. And then again with the very next Cutting Edge build the published website manual will tend to lag behind the latest and greatest as described in the Cutting Edge announcements. So if you are working with Cutting Edge builds or any other pre-release it is wise to read those announcements. Adam puts a lot of work into them and they are worth the effort.

    For all that, as with any other part of the product if you don't like something in the manual, send in a suggestion to get it improved, or, if you feel any topic in the manual has been overlooked or is lagging too far behind the current Cutting Edge build, send in a request to get it updated. Same for the API doc.

    dchall8
    481 post(s)
    online
    #08-Jan-18 15:26

    When 9 is published as an official release and not a pre-release the manual will be totally in sync again.

    When I first started my Air Force career (1974) we were developing hardware which was driven by software. Documenting the software had never been a priority in our field until about the time I got there. Nobody wanted to do it, but it was required by DoD. Most of my co workers hunted down ways to bypass or waive the documentation requirements, but I could see the handwriting and moved forward toward documentation. One of the tips I found useful was for estimating the time it takes to document software. They said if the contractor estimated a time of 2 man-months to write the documentation, then for my cost estimate I should double the number to 4 and change man-months to the next higher order of magnitude. So I budgeted for 4 man-years. My co-workers fell behind by not getting approval up front to waive the doc requirements, and they fell further behind by being underfunded. They all had overruns and my projects came in about on target. Even the contractors thought I was clairvoyant.

    adamw

    8,090 post(s)
    #05-Jan-18 12:28

    There might be as many as 200 duplicate PROP_ID areas still to fix. But this inconsistency was not a problem for the relation tool in M8.

    Duplicates are not a problem in 9 either - just create an index that allows duplicates (BTREEDUP or BTREEDUPNULL, the latter also allows NULLs) instead of an index that does not allow them.

    Overall, we are talking about indexes here because we are using SQL and SQL generally does less "magic" than visual tools do. A visual tool would have created temporary indexes if the tables do not already have them, SQL generally does not do that (it trusts the query writer).

    Thankfully, using indexes to improve the performance of a query is usually pretty straightforward. BTREExxx indexes improve the performance of operations like = and IN. RTREE indexes improve the performance of spatial tests like GeomContains. In your case, add a BTREEDUP or BTREEDUPNULL index on PROP_ID fields in both tables, that's it. You can add new indexes using Edit - Schema, or from a query using ALTER TABLE.

    dchall8
    481 post(s)
    online
    #05-Jan-18 17:37

    I added a btreeupnull index to the formerly non-indexed PACS table. In case nobody has counted, the process requires 12 mouse clicks as well as some typing in between some of those clicks. This process will have to be done on the updated table every week or as necessary. I also deleted the btree index from the Working parcels table and added a btreeupnull. That required an additional 4 mouse clicks over the same 12, but now that I have that table indexed, it should stay indexed...correct? After 2 minutes the query had processed 248 records out of the 31,000 records. And if I allowed it to run, I still would not have created the new table I need for daily use. For comparison, the relation process in M8 takes 10 mouse clicks and no typing, but the resulting usable table was joined essentially instantly.

    Is there a way to stop a query from running when it is clearly going to take hours instead of seconds? I just closed out of M9 to stop it.

    artlembo

    3,077 post(s)
    online
    #05-Jan-18 18:46

    Is it possible to post the table? That seems like a long time.

    Also, I think it would be nice to allow users to right-click a column and have a list pop up with one saying Add Index and that would expand to allow the user to select the index type.

    dchall8
    481 post(s)
    online
    #05-Jan-18 22:12

    What is the best way to post the table without losing the Manifold formatting? If I use a CSV export the entire file is 11 MB. I was thinking of attaching only 300 records to make it smaller and then realized that the essence of the field types would be lost.

    artlembo

    3,077 post(s)
    online
    #05-Jan-18 22:51

    Oh, I have a thought:

    are you running the query off of the CSV data source, or did you import the file in? Two things:

    1. Importing the file, and running a query against it as a local component will always be faster than trying to hit a CSV via a data source.

    2. I did notice that a query I tried to make on a a data source seemed particularly slow today, so you might want to email tech@manifold.net and describe what is happening.

    from looking at your query, it looks like it is a local component. But, I just wanted to see.

    you can email me the file (.map) or the .csv, and I can take a look at it. I can run the query as a local component as well. Maybe by looking at this, we can figure out what is going on, and perhaps email tech about what we find.

    adamw

    8,090 post(s)
    #06-Jan-18 08:39

    What is the best way to post the table without losing the Manifold formatting?

    You can copy and paste both tables and a query you are running into a new MAP file, export that as MXB (File - Export Project) and post that.

    dchall8
    481 post(s)
    online
    #08-Jan-18 15:32

    What size do I need to get the MXB file down below before it will upload to the forum? I'm getting errors on the upload. The file size is 11MB, but I can delete records to get it down. I thought upload size was in a FAQ somewhere, but I don't see it.

    I was not using a linked file. It's imported.

    adamw

    8,090 post(s)
    #09-Jan-18 07:09

    The limit is 4 MB. You can send the file via tech support (refer to the thread and say that you want to report a potential problem, do not require a reply, and want to send data that is larger than the forum allows - they will provide FTP account info), or upload it somewhere and post a link. Or delete records, yes - as long as the issue will still appear (query being slow).

    dchall8
    481 post(s)
    online
    #18-Sep-18 19:39

    I have not had a reason to revisit this topic for awhile, but now it's time. What I want is an easy way to make a relational database a la Manifold 8. When I try running the M9 query, as posted back in January, it takes hours and hours to run the query as opposed to the instantaneous response in M8.

    The MXB file exported from M9 is 11MB, so I cannot upload it to the forum. I have uploaded the MXB file to the following location, http://ge.tt/2ySz0jr2 for anyone who would like to take a look at it.

    danb


    1,649 post(s)
    online
    #18-Sep-18 19:57

    A couple of indexes wouldn't go a miss. I added a BTREE on [PACS 20180914] (PROP_ID) and a BTREEDUP on [WorkingParcels Table] (PROP_ID) and changed the query to a SELECT INTO query. It now runs in:

    2018-09-19 06:54:50 -- Query: [Relation query] (2.422 sec)

    On my box.


    Landsystems Ltd ... Know your land | www.landsystems.co.nz

    dchall8
    481 post(s)
    online
    #19-Sep-18 16:02

    Thanks Dan. That sounds encouraging(ish). Certainly that works and it works GREAT! The query execution speed is excellent, but the prep work is still not exactly the same "push button" style as found in M8. Here are my concerns now, and maybe these can be overcome.

    Every time I update the PACS data I'll have to redo the indexing before I can relate them. Is there a way to make that part of the query?

    Every time I run the query I'll get a new table called NewTable. Does the query repopulate the previous NewTable table every time? Rerunning the query does not seem to add more and more fields to NewTable so I'm hoping I can maintain one copy of NewTable and the NewTable Drawing that updates.

    Oops! With that last statement I realized I would be making map changes to NewTable Drawing and not to WorkingParcels Drawing. The query runs on the WorkingParcels Table, so changes I make to the PROP_ID field in NewTable table will not be included the old WorkingParcels Table. I hope I'm having a logic problem here and there a way to fix this. In M8 I always worked in the original WorkingParcels drawing and table all the time. Every week I "un-related" the two databases and re-related them with the new PACS table. So I was always in the same drawing.

    I have uploaded the improved version with the new indices to http://ge.tt/9cZ25kr2 and called it Relating tables2.mxb. I don't know if anyone needs it to help with the logic, but there it is.

    dchall8
    481 post(s)
    online
    #19-Sep-18 21:01

    Now there's another issue. When I try to create a new point, line, or area, after selecting either Save Changes or Add Record, I get a message, "Invalid key field value (null)." Then the contents pane switches to Record and shows that all the fields are filled with <NULL>, which I assume is for the new point I wanted to add. I checked the original file and I can add elements to that drawing, but not after running the query to create the new table. The original drawing is affected as is the new drawing from NewTable.

    tjhb

    8,227 post(s)
    #20-Sep-18 03:07

    It would be easy to explain what you are seeing here and the reasons for it, and to answer specific questions.

    But I doubt that would do any good, unless you show some willingness to read the documentation for Manifold 9 and to learn how it works.

    If you want Manifold 8, use Manifold 8. Otherwise...

    It would be easy for you to show *some* evidence of reading in your questions. What you get, and don't get (yet).

    Then (soon) you could also start to answer questions from others who are new to 9. Everyone needs help.

    dchall8
    481 post(s)
    online
    #20-Sep-18 16:12

    So if I had added words to the effect of, "I just spent hours reading the manual and I cannot clear this up," then you would have provided me with the apparently obvious answer?? Okay, I can do that. After 14 years of using Manifold I'm still learning how to use the forum without getting chastised or worse. You guys don't seem to have much tolerance for those of use who are database challenged. Dimitri would remind me that I need to read a book. I've tried, but at about page 5 I have blood shooting out of my eyeballs. Maybe there should be a separate forum for beginners and us database sloths.

    I don't normally discuss how many hours of manual reading I do to solve my issues, but adding the btree and btreedup indices took at least an hour of reading, learning terminology, and trial and error. I had tried it before but I did not know to put the index on the PROP_ID field. I didn't know I could do that until Dan dropped those clues. I had an entirely different problem I was going to ask before I fixed it and finally boiled it down to the logic question. As I wrote the first question out I gave myself further clues as to where to look in the manual for more information. After solving that then I had to learn what the recommended SELECT INTO command did. In just a few words, Dan sent me on a several hour journey through the manual and M9 schema. I learned what the btree and btreedup indices are, but I don't know how he knew which fields needed which index. So there's a lot of research going on in the background that I don't feel the need to write about. When the Invalid key problem came up, I revisited all I could find about creating new points, lines, and areas. The assumption in the manual is that it will work. There is no discussion about getting an Invalid key field error. As far as I can tell the manual does not discuss the possible causes of errors.

    Here's what I know: I had a project where I could add new elements, then I added indices as documented above and ran the blindingly fast query. After the query I cannot add new elements to either the original drawing or the new one. If there is a place in the manual where this topic is discussed, I'm more than willing to read it, but at this point my Manifold and database vocabulary is exhausted. I don't know what else to look for.

    If you want Manifold 8, use Manifold 8. Otherwise...

    I am using M8, but I want to learn to do the stuff I do in M9 as I think I can. At this point of development M9 is at the hobby stage - at least for my application. What I'm trying to do is keep up with the development so I'm not caught off guard later. It is mentally challenging for us older guys, and I try to keep it fun despite the frustrations of not knowing how to make it work. At first your simple looking query from a year ago seemed doable. Then it took half a day to run the query until I got the hint about btree and btreedup. That seems to produce an unusable drawing. I don't know enough about the problem to ask the right questions. I don't mind reading the fine manual, but it is frustrating when I have exhausted my knowledge of what to look for.

    Every now and then, and as features are added, I try to replicate simple tasks from M8. What I'm doing here is to Union all the parcels in the named neighborhoods to create a subdivision map of the county. I don't see the value in such a map, but I try to please. I cannot make the map in M9 until I can merge the simple parcel map with the detailed data table. I thought it would be easier to do this in M9, but there are obstacles I can't even see.

    dchall8
    481 post(s)
    online
    #20-Sep-18 23:57

    In the interest of moving this along, I started over and took a bold and reckless step of making the btreedup index into a btreedupnull index. Dan is probably wondering why his head is pounding. I'm not sure why this change works, but I don't get the errors when I add new elements to the drawing. In looking at the table all the new points have <NULL> values for everything. What's puzzling to me is that I could add new elements to the original Working Parcels drawing and it does not have a btreedupnull index.

    tjhb

    8,227 post(s)
    #21-Sep-18 03:08

    So if I had added words to the effect of, "I just spent hours reading the manual and I cannot clear this up," then you would have provided me with the apparently obvious answer?? Okay, I can do that.

    Of course not. That would make no difference.

    But if you had quoted, or given a link to, some text in the manual that you don't understand, then yes.

    The same if you had quoted some other thread on the forum.

    Do your homework, step by step. Don't blame anyone else.

    Everyone here is a volunteer, so anything you are offered is a straight-out gift.

    When you can, help other.

    It is mentally challenging for us older guys

    I couldn't agree more.

    I don't know enough about the problem to ask the right questions.

    Actually, you do.

    Every now and then, and as features are added, I try to replicate simple tasks from M8.

    Reframe that. Instead, try to achieve the same purposes you have achieved in 8. Put means completely aside, because 9 is different.

    tjhb

    8,227 post(s)
    #21-Sep-18 03:26

    What's puzzling to me is that I could add new elements to the original Working Parcels drawing and it does not have a btreedupnull index.

    You are getting there. What difference does the index make?

    Hints: (a) What field is it on? (b) What happens in that field when you add a new element?

    dchall8
    481 post(s)
    online
    #21-Sep-18 19:52

    Reframe that. Instead, try to achieve the same purposes you have achieved in 8.

    That's a better way to say what I meant. It is getting easier to do things as more transform templates become available.

    Hints: (a) What field is it on? (b) What happens in that field when you add a new element?

    I had to try some things to see what you were getting at. In the original table only the mfd_id field is indexed as btree, and a new number is generated with each added element. So the fact that the number is generated gets you past the inability to accept a null value in that field. In the NewTable the PROP_ID is indexed and new elements do not get a new value automatically. There are duplicate entries in PROP_ID, so the index has to allow duplicates, and since new elements will have null values for the PROP_ID field, it has to accept null values in the index.

    I think I asked enough questions to let you know where I needed hints. That works. What I did not realize was that a new element is created with null values. The concept of Null values was new to me with M9, and I never had to use them until now. I guess new elements come into M8 tables with null values, but you don't see them as <NULL> in the table, so I'll get it. Thanks for helping me over that hump.

    Now I'm searching the manual, because I stumbled over the words CREATE INDEX followed by SQL phrasing to complete the process. I want to beef up the query to simplify this.

    Also I'm still not sure how to keep from creating new NewTables every week as the NewTables get modified during. That's not a question or a plea for help. That's my logic issue.

    adamw

    8,090 post(s)
    #24-Sep-18 15:57

    On MFD_ID - this field in tables in MAP files is special in that you do not have to provide new values for it and the system will generate them for you. If you do provide your own value for MFD_ID when inserting a new record, the system will use that value, but if you provide nothing, the system will generate a unique value for MFD_ID automatically. No other fields in tables in MAP files are like that.

    Fields in tables on databases like PostgreSQL or SQL Server can be set up to generate values automatically as well, the set up and the limitations are specific to the database.

    adamw

    8,090 post(s)
    #24-Sep-18 15:42

    I learned what the btree and btreedup indices are, but I don't know how he knew which fields needed which index.

    An index on a field helps the database system find specific values in that field faster. A BTREExxx index sorts values from lowest to highest, is generally applied to numbers / strings / dates, and helps optimize searches for = as well as IN / BETWEEN AND / some other operators. An RTREE index sorts values spatially, can only be applied to geometry values, and helps optimize searches for functions specific to the database, in 9 these functions are GeomAdjacent / GeomContains / GeomIntersects / GeomTouches / GeomWithin, the list sometimes grows.

    Say, we have a query like: SELECT t.value1, u.value2 FROM t, u WHERE t.id=u.id:

    Let's assume all fields are numbers and so let's talk only about BTREExxx indexes. With that:

    • adding an index on u.value3 is not going to make the query faster because that field does not participate in the query at all,
    • adding an index on t.value1 is not going to help the query faster, because when the database executes the query it does not search t for specific values of value1, it searches t (or u) for specific values of t.id (u.id) and t.value1 just gets copied into the output,
    • adding an index on t.id tells the query engine 'hey, just in case, if you happen to have a value of t.id, the table t can return other values for that record fast' - the query engine is going to use that and the query is going to run faster than without the index,
    • adding an index on u.id in addition to the index on t.id is going to make the query even faster, and doubly so if the indexes are BTREE = disallow nulls and duplicates.

    So, returning to your question - which field need which index - this depends on what queries you are running and what is the nature of your data. This is a big topic worth books, but very shortly and imprecisely, if you have some field which you primarily use to identify records and to refer to records from other tables - like ID or RECORDNUM or something like that - and if you are using that field in constructs like WHERE t.ID = <another value possibly from another table>, you will most likely want to add a BTREExxx index on that. If the values in the field cannot contain NULLs, reflect that in the index type (use BTREE / BTREEDUP). Same if they can contain NULLs (use BTREENULL / BTREEDUPNULL). If the values in the field cannot coincide with each other, use that (use BTREE / BTREENULL). Same if they can coincide with each other (use BTREEDUP / BTREEDUPNULL).

    Same with RTREE - if you are doing spatial searches on a particular field, you will likely want to build an RTREE on that field.

    There is no discussion about getting an Invalid key field error.

    Regarding this: you are likely getting the error because one of the indexes in the table you are trying to add a record to has an index which requires non-NULL values and you are not providing these non-NULL values while inserting the record. Eg, if you have a BTREE / BTREEDUP index on a field, when you add a record - whether interactively by clicking in a window or via a query or via a script, whatever - you have to provide a non-NULL value for that field. For clicking in a window, you do this by entering the value in the Record pane.

    You can "solve" this by removing the BTREE / BTREEDUP index that you have and adding BTREENULL / BTREEDUPNULL index which allow NULL values, but the right question to ask is - do you want to allow the values of this field to be NULL or not? Because if you are fine with the values of that field being NULL, then OK, use BTREENULL / BTREEDUPNULL, that's the right type of indexes to use here. You won't be able to find records with NULL values in that field in queries, eg, SELECT * FROM t INNER JOIN u ON t.id=u.id will not return any records with NULL values in either t.id or u.id, the query will work as if records with NULL values in those fields just didn't exist, but maybe that's how you want it. But if you want queries like that to be able to find all records in the relevant tables, then you want to *disallow* such fields to have NULL values, so use BTREE / BTREEDUP and do provide a unique value for each new record that you add interactively - by going into the Record pane and typing in the unique value. If you want the system to help you provide such a unique value automatically, that's a fair request and something we can talk about.

    PS: Wow, I didn't realize the text is this long. Apologies.

    adamw

    8,090 post(s)
    #06-Jan-18 09:06

    In terms of speed, as others said, 248 records per 2 minutes (2 per second) seems too slow. If you post the data, we will check what's up.

    I just did a very simple test using 9.0.164 and the performance of your construct seems to be where it should be.

    This creates two tables (100,000 records each) and fills them with example values:

    --SQL9

    CREATE TABLE t1 (a INT32INDEX a_x BTREEDUP (a));

    INSERT INTO t1 (a)

      SELECT (value DIV 2) * 3 FROM CALL ValueSequence(1, 100000, 1);

    CREATE TABLE t2 (b INT32INDEX b_x BTREEDUP (b));

    INSERT INTO t2 (b)

      SELECT (value DIV 2) * 5 FROM CALL ValueSequence(1, 100000, 1);

    This then joins them, counts the number of records in the resulting table and writes that count into a new table (the last bit is there to measure the run time easier):

    --SQL9

    SELECT count(t1.a) INTO u FROM t1 LEFT JOIN t2 ON t1.a=t2.b;

    This second query runs in 0.447 seconds on a test system and the number of records it reports is ~120,000.

    Dimitri


    4,991 post(s)
    #05-Jan-18 20:18

    This process will have to be done on the updated table every week or as necessary.

    If you do it frequently, automate it with a query, just one click to run and it is done. I agree with Art... what you report is a very long time. Something cannot be right in that, just 248 records in 2 minutes? 248 million, yes, maybe, but 248?

    Dimitri


    4,991 post(s)
    #04-Nov-17 09:28

    Thanks for the explanation!! That clears things up. I think I had trouble visualizing the situation for two reasons:

    1. "Undo" usually means a specific command, typically launched with a Ctrl-Z, for example, that rolls back something that was done. I didn't understand you meant that to refer to deleting a relation from a table and then adding a relation again. So I was running around looking for an "Undo" command in 8 for relations that for the life of me I could not remember being there... :-)

    2. I was slightly surprised by this in Tim's post: " To refresh the relation for new data, you would delete the defined relation and start again. " - For some reason I thought that bringing data into a Release 8 table from an external source via a relation could be / would be refreshed without having to delete the relation and adding it again.

    But... the above is why it helps to ask for clarification! :-)

    Tim's right, by the way: the ability to just re-run the query in 9 is much, much better than having to manually delete a relation and add it again in 8.

    dchall8
    481 post(s)
    online
    #07-Nov-17 21:42

    This gets us back to the original question about wizards in 9 like there are in 8. Lets say that rerunning the query in 9 is better. I have 30 or more fields to join. In 8 it needed no separate query. I could relate tables as fast as I could click and get all the fields from each one. With 9 each table needs a custom query which is not intuitively obvious to a SQL plebe like me. I use Manifold to keep track of parcel ownership in a mapping environment - not as a tool to learn SQL.

    The Manifold 8 Toolbars are golden. The simplicity and power of the Query and Transform Toolbars is immensely helpful. I can do things really fast that my ESRI user friends cannot do at all (because they have not gone to that training). Those toolbars are always on at the bottom of the window, so I don't have to think about the process for finding the Transform pane. Secondly they automatically switch modes depending on whether a table or map is in focus. Third, most of the references in the Transform menu are understandable. In 9 it is often very cryptic (e.g. Parse GeoJSON, bounds, branch, Compose Point, copy (but no paste), etc.)). I can't imagine how many lines of SQL (that I don't have to learn) are buried in the 8 toolbars, and I appreciate it.

    I know about RTFM, but I already read much of the FM in 8 to get where I can use it. I was hoping 9 would not be a complete rewrite of the user interface. Using one interface for 13 years gives a person confidence and speed. So far 9 has different menus, mouse buttons, keyboard keys, and toolbars. What is the motivation to change every aspect of a program that is just the next version? That's what ESRI used to do. I was hoping 9 would be the same old Manifold we have come to love but with some tune ups made possible by the Radian Studio technology. I do like the ability to undock a window or table, but I don't like the inability to use mouse and kbd combinations to deselect background elements I have accidentally selected while trying to select a line.

    I realize I am expecting too much too soon with 9. I guess I'm worried that the current enthusiasm for the SQL abilities is going to overshadow the ease of use I've grown up with.

    Dimitri


    4,991 post(s)
    #08-Nov-17 09:19

    You raise so many good points, in good faith, let me try to address them. This is not intended in any tit for tat way but rather to try to open up the discussion to perhaps show that what you want is there before you.

    It is there in a different way, of course, which is the usual learning of new and improved methods which open the door to greater capability. Zipping about in a sports car really is faster and covers more ground than trotting on a horse. But as much fun as trotting on a horse may be, the skills which enable one with confidence to cover ground on a horse are fundamentally different than those required to drive, say, from San Francisco to Ensenada in a day, covering huge mileage on Interstate 5 and successfully punching through LA traffic en route.

    I use Manifold to keep track of parcel ownership in a mapping environment - not as a tool to learn SQL.

    That is a bit like saying "I use my sports car to get from San Francisco to Ensenada, not as a tool to learn how to work the steering wheel, the gas pedal and the brakes." Well, sure. Likewise you use Future as a tool to get tasks done, not to learn SQL. For that matter, you con't need to know SQL to drive Future, no more than you need to know how to work a clutch and manual shifter to drive a car with automatic transmission.

    Future has automatic transmission. No need for SQL, for most things, if you don't want it. But just like you can find endless examples of tasks in 8 that require SQL (8 has nice SQL, not as nice as Future, but still very cool...), just so for Future.

    There is a huge amount of work you can do in Future without knowing a drop of SQL, just by using things like the Select and Transform panels. As more and more templates are added there will be even more capabilities that don't require any SQL. So positioning the matter as 8 being SQL-free while Future requires SQL is totally wrong and misleading. If you think that way you are denying yourself the use of a hugely powerful tool that in most (not all, but most) respects is much easier to use than 8. A good example of that is previews. You don't get any previews in 8 but you do with Future, which really helps avoid making mistakes and also helps zeroing in on doing exactly what you want to do.

    But beyond all that if you want to learn SQL to lift your game to much higher capabilities, to do work more effectively and faster and with less effort, well sure, it is truly great and wonderful to spend the utterly trivial amount of time required to get going with SQL. That's true for 8, and it is also true for Future. That Future has far stronger SQL than 8 is a good thing, not a bad thing, because it enables you to do far stronger things with SQL in Future than you could do with SQL in 8. All that is very good, and is not remotely any sort of a flaw. It is genuine progress.

    I have never met anyone who has sat down with, say, one of Chris Fehily's books and then spent a solid few hours learning SQL from the beginning who did not say, "wow... this is really all very easy. I don't know what the fuss was about..." Anybody smart enough to be doing GIS at all, at any level, is more than smart enough to realize from such an experience that starting with simple use of SQL is just a beginning, and right away will see that you can add to it to make it as complex and elaborate as you like. Or not.

    But I have met zillions of guys who never sat down to learn SQL from easy, simple basics who complain how hard it is on the basis of thrashing around poking at buttons and trying to figure out what is going on from master class examples like those Adam or Tim contribute to this forum. For lack of investing a couple of hours up front to start learning the thing properly from the beginning they'll spend a lifetime encountering frustration and complaining.

    Now, it could be that you are the one guy, the one exception in hundreds, who has sat down for some quality time with a Fehily book, and yet despite spending a few hours or a day or whatever learning the very basics, still thinks

    SELECT name, address, city FROM clients WHERE city = 'London' 

    is so totally complex it could not be understood by ordinary mortals. But I'd bet not. You don't need SQL at all to use Future and to do wonderful things.

    But if you make the trivial effort to learn the world's most widely used means of interacting with data you can do even more. And not just with Future but with a few thousand other packages. Nothing wrong with that!

    In 8 it needed no separate query. I could relate tables as fast as I could click and get all the fields from each one. With 9 each table needs a custom query which is not intuitively obvious to a SQL plebe like me.

    You could relate tables as fast as you could click in 8 because you had taken the time to learn how to do that. Beginners cannot relate tables as fast as they can be clicked in 8. It turns out you can relate tables even faster in 9 with even fewer clicks, (double-clicks, actually) to add tables using the query builder. It's not obvious to you because you've not yet learned how to do it. Once you do, you'll see it is even easier than in 8 while being far more flexible, far more powerful and much easier to maintain or to alter to adapt to changing interests.

    Those toolbars are always on at the bottom of the window, so I don't have to think about the process for finding the Transform pane.

    "Process for finding?" ... over the top given the zero effort of clicking Transform in an always-present, always responsive Contents pane. That's like complaining about the "process for finding" the kitchen in your own home. :-)

    Secondly they automatically switch modes depending on whether a table or map is in focus.

    Same with the Select and Transform panes.

    Third, most of the references in the Transform menu are understandable. In 9 it is often very cryptic (e.g. Parse GeoJSON, bounds, branch, Compose Point, copy (but no paste), etc.)).

    There's no "Paste" in the transform toolbar in 8. Many of the transforms have the same names. Some don't because they are different. For example, 8 has no ability to parse GeoJSON. If you work with GeoJSON you know what that means and you are happy to have that capability.

    I think 8 has reasonable names for templates but I respectfully disagree that names such as "Span Excluding" or "Decompose to Convex Parts" are automatically self-documenting for people who don't RTFM. You have to read the fine manual to understand what they do, and then beyond that you need some experience and familiarity with them before they start sounding obvious. Same with new Radian templates.

    Some of those, by the way, have different names from 8 analogs because Radian tends to be more precise than 8. A word like "bounds" is slightly more precise than "boundaries," for example, in that "boundaries" tends to imply internal dimension that only areas have while "bounds" tends to cover "extremities" or "markers" in a way that slightly (just slightly) is a better fit to the end points of lines or the single coordinate location of points. It's not like after a moment you wouldn't immediately recognize Bounds in the list of templates for what you used Boundaries in 8.

    It would be better to address the very long list of possible templates and how they can be handled. Future has yet to add all the various token operators, for example, and one reason is that you end up with very long lists (even longer than the long lists in 8) of templates that can be cumbersome.

    I can't imagine how many lines of SQL (that I don't have to learn) are buried in the 8 toolbars, and I appreciate it.

    Same with Future. You have hundreds of templates with not a single line of SQL you need to learn to use them. But, unlike 8, if you want to take your game to the next level Future will show you the SQL behind each template so you can modify it to suit variations or to learn.

    What is the motivation to change every aspect of a program that is just the next version?

    If Future were just the next version of 8, I'd agree with you. Future isn't the next version of 8. It is a completely new and different product. It does far more than 8 with endless detail in greater accuracy, rigor, sophistication, flexibility, capacity and more. Pretty much any new technology is like that: your smartphone has basically every aspect different from a rotary dial landline telephone. Zipping around on a jet ski is very different than paddling a canoe. Flying a jet with a glass panel requires significantly different cockpit design than flying a biplane.

    Sometimes when technology changes vendors will try to package new things in old wrappers to ease the transition. The very first automobiles, for example, were modeled on horse carriages. Until it became clear that to really take advantage of the new technology and to provide an even higher level of comfort, ease of use, and convenience it was better to design automobiles to be automobiles and not powered horse carriages.

    My challenge to you is to learn the new environment so you can take advantage of the power and convenience it delivers. Then, if you find something lacking or inefficient, say what it is so it can be improved. But stuff like this...

    but I don't like the inability to use mouse and kbd combinations to deselect

    ... is not helpful because it is so untrue you may as well be talking about Excel and not Future.

    Deselect? Trivial: If you like keyboard stuff, use Shift-Ctrl-A (a quasi-standard Windows thing) or Ctrl-A Ctrl-I. Deselect with a mouse? That's what the Shift modifier is for. See the video.

    Any place where Future is not more effective than 8 is fair game for complaint and improvement. But let's keep such observations to real things and not complain about things which are not at all about Future.

    Keep in mind as well that by using Future you are participating in an open beta program, so you should cut the documentation a bit of slack for being behind a very rapidly moving thing. The RTFM effort is harder with betas because you have to cruise on the basis of release notes and introductory videos as well as more finely worked documentation typical of a finished Manifold product. But that's part of the deal for getting access to the hottest technology right away and being able to influence where it goes.

    Last but not least, to make useful comparisons such as "A is a much better approach than B" you really have to know both A and B. If somebody recommends A because that's what they know and to learn anything about B requires some effort, well, as far as that goes it is a fair comment but it is not the same as being able to say that A as a way of working is better than B as a way of working.

    The comments you make about relations in 8 being so much easier than SQL seem to me to be an example of that. What you are really saying is that what you know how to do in 8 is much easier than something which you have not learned. Well, OK, I'll agree with that as far as it goes, but it doesn't go any farther than announcing that something which has not been learned cannot help you. It doesn't touch the idea that once you learn a bit of SQL you can do what you want to do with far greater ease, more power and greater flexibility than using relations in 8.

    And that's really the idea for which Manifold is created, to provide greater ease, more power and greater flexibility, all for less cost. You do have to learn to use it, but hey, that's to be expected for any worthwhile new thing, whether it is carpentry, learning to drive a car or even learning to ride a bicycle.

    dchall8
    481 post(s)
    online
    #08-Nov-17 14:57

    Thank you for all that. In addition to being frustrated about the things Future doesn't yet do, I am frustrated that I cannot remember the simple stuff (Shift-Ctrl-Click). Perhaps for now I should stick to the things that Future can do (LiDAR) that Manifold 8 cannot and just enjoy the fortnightly-ish updates and videos as I can. As Future improves and I explore it more, maybe I can make it work for the daily grind before the final 9 release.

    Rather than importing a Manifold 8 project, I'm going to start with only the parcel drawing and build tools from there. That's how I started with Manifold 4, so I'll try that. Plus I need folders inside folders to be more organized, and importing a project scatters my 8 folder contents.

    Thank you again for your patience with me. I'll try to be patient with the Future open beta.

    Dimitri


    4,991 post(s)
    #08-Nov-17 15:42

    I'm going to start with only the parcel drawing and build tools from there.

    Sounds like a plan! Report progress here and ask any questions so we can pitch in to help.

    KlausDE

    6,214 post(s)
    #08-Nov-17 18:28

    I also thought about a sheet with all shortcuts. For instance I have forgotten the shortcut to open edit tools for curved segments and the key on a german keybord is not what help tells it should be. Guess I should have this laminated always at hand.

    pslinder1121 post(s)
    #09-Nov-17 00:31

    In 9 is Manifold planning to have buttons or some sort of GUI for each of the shortcuts? Or will some of the shortcuts be the only practical way of performing those functions?

    Dimitri


    4,991 post(s)
    #09-Nov-17 09:45

    the key on a german keybord is not what help tells it should be.

    The New Object Dialog is no longer enabled in Future editions, so Ctrl-/ won't work. Adding curved segments is something being migrated to the Coordinates tab of the Record panel so for now no entering of curved segments without rolling back to a prior build (trivial to do with portable installations...). ... all part of the fun of beta builds. :-)

    In 9 is Manifold planning to have buttons or some sort of GUI for each of the shortcuts?

    Can't resist... Yes. ...and...

    Or will some of the shortcuts be the only practical way of performing those functions?

    Yes.

    Given the very many Windows shortcuts as well as the Future-specific shortcuts, which are of interest to you?

    Some make sense to put in the GUI (File - Open or Ctrl-O) while others don't. After all the whole point of a keyboard modifier, say, to a mouse move is to not interrupt the flow of mousework with a click on some button or menu. It would be good to learn how you think that should be arranged.

    Remember, participating in an open beta is tall about saying what you think should be done. It's not to ask what will be coming, since that you find out by participating and seeing when it arrives.

    lionel

    457 post(s)
    #06-Jan-18 18:11

    as a normal user ( not developper ) of CMS joomla that use mvc pattern , i can create query without write raw code SQL. it is like javascript that have babel and typescript or ces with sass/less . All pre compiler generate raw code...... so for SQL there is ORM. i try to develop joomla extension ( component,module,plus in) 1 software so i aware of that but think debug tool should exist and google create sourcemap that give you à way to locate the line that give error in the precompile language. i don't know how orm can be debug in php sever side.......


    join image "Because my dad promised me" interstellar from Manifold: Time by Stephen Baxter. power Math destruction

    lionel

    457 post(s)
    #06-Jan-18 18:33

    and less in manifold9 software ( never reas subject about this ) . Perhaps in a desktop env that make no sense , since

    ---each time you add layer you losse some speed but sometime you don't need speed ( optimisation) and need to code fast .

    ---we are not on a web server where many many users ll call the ORM ( without knowing they use one ) because all matters is the results html page ( css js html raster vector .....)


    join image "Because my dad promised me" interstellar from Manifold: Time by Stephen Baxter. power Math destruction

    Manifold User Community Use Agreement Copyright (C) 2007-2017 Manifold Software Limited. All rights reserved.