Subscribe to this thread
Home - General / All posts - Query Builder behavior
dyalsjas96 post(s)
#04-Feb-18 16:36

Perhaps this has been discussed elsewhere on the forum and I've missed it.

As I work with the Query Builder, I've noticed that objects added to the query builder are not persistent between Query Builder / query edit sessions.

I'm working through Fehily SQL 2015, and tweaking the text listings / queries to look more like the examples from the book.

I'm adding tables to the right lower pane of the query builder to make it easy to double click tables or fields referenced in the query.

After saving the query, closing it, and reopening it, the lower right pane is empty.

Does it make sense to have objects added to the query builder pane be persistent?

KlausDE

6,228 post(s)
#04-Feb-18 17:07

That's for safety. Field names Amy hab been altered in the meantime and tables noch longer exist. This is btw the only way to wipe out this area.

dyalsjas96 post(s)
#04-Feb-18 17:41

I can understand that concern; however it seems unneeded since the query would fail if field names have been altered or tables have been deleted. Persistent objects (including tokens for missing or changed objects referenced in the query) in the query builder pane would likely make it easier to discern needed corrections rather than starting from scratch.

However, now we are moving into discussions that are more in the realm of development / data studios and less GIS... ...not unexpected with the current progression of Manifold capabilities.

Dimitri

5,035 post(s)
#05-Feb-18 05:59

Does it make sense to have objects added to the query builder pane be persistent?

I could see it either way, but probably not. Interfaces are easier when they don't require attention to exceptions. As the discussion has noted, when you do changes a persistent interface would have to highlight exceptions, which the user must notice and so on. Easier on users for it to be as it is: here it is, use it, no problem.

Keep in mind that it takes near-zero time to drop tables into the pane. Working with five or six tables and you don't want five or six drag/drop moves? No problem. Keep them all in a folder in the Project pane and then it is one motion to drag/drop that folder into the pane. All of the tables in the folder get dropped into the pane in one move.

So... if there is a question whether persistence is a gain or a loss given the above, the rule of thumb is to keep it simple, and put the energy of adjusting details into frying bigger fish.

tjhb

8,273 post(s)
online
#05-Feb-18 08:37

I hadn't registered that we can drag across a whole folder of tables! That's fantastic, who thought of that?

My initial thought in response to the idea of persistent contents in the tables panel was... isn't the main point that it's live? Just like a sushi train (if other people have those).

The best place to persist table definitions, field lists and so on between editing sessions is (in my opinion) in comments inside the query text (perhaps with notes). Totally under the writer's control.

And the query builder interface makes it so easy to transfer structural detail from the table panel into code. Why not keep it there?

(If only we could toggle multiple comment lines with one keystroke, and adjust the indent of multiple lines with another keystroke. So nicely done in 8. For 9 I still tend to copy back and forth from Notepad++.)

adamw


8,139 post(s)
#05-Feb-18 11:20

Maybe we could pre-scan query text and add the tables it uses to the list. This would only work for a Manifold query though and it would require the query to be valid (we'd stop at the first invalid statement).

tjhb

8,273 post(s)
online
#05-Feb-18 12:11

But why would that be useful? It just sounds like clutter.

If anything, it's only for the currently unfinished, invalid query text that I'd want to keep table data for the next session. Maybe that's just me.

dyalsjas96 post(s)
#05-Feb-18 15:52

My thought was that persistent objects in the query builder pane should be per query. I agree that dragging multiple tables or folders of tables is a great feature. My background thought for persistent objects in the query builder pane is that it would be easier to discern the objects and tables in a query if you are revisiting the query after a time away from the project. Another use case would be if you have shared a project .map file or have stored a query in a enterprise datastore.

KlausDE

6,228 post(s)
#05-Feb-18 16:19

I'm known for been lazy but nevertheless I think there is no way around a clean formating of the query text and detailed comments interspersed.

Dimitri

5,035 post(s)
#06-Feb-18 06:12

it would be easier to discern the objects and tables in a query if you are revisiting the query after a time away from the project. Another use case would be if you have shared a project .map file or have stored a query in a enterprise datastore.

I agree, of course, that there would be some utility for that. Now, as a though experiment I ask you to write out all the scenarios in which that might be used, how it could go wrong, and what the system should do about that. I'm not asking you to program it, I'm just ask you for a clear and complete thinking through of what you propose, so you actually propose the thing beyond a vague notion stage.

For example, by "persist" I suppose you mean that it is stored in the .map file, so that when you open that .map project you get a neat re-filling of the query builder with all the tables and fields and such as it was, right?

Super. So now you give that .map to a colleague on a different machine in Argentina and he or she opens it. Ain't no such data sources on that different machine. What happens? Is the pane full of tables and fields empty? But you've already advised the colleague that it is persistent, so what does that mean when it is empty? How is the colleague warned?

How about if you just move the project within your own hierarchy and now the absolute paths to the data sources are incorrect. What then? Suppose you've renamed something in one of the data sources. Is the query builder suppose to recurse through all projects on your computer and the billions it can reach through your network connectivity to update any changes that were made since the last time the project was opened? If it doesn't what it shows in the pane is wrong and misleading. Is that OK? Should the user be alerted somehow it is wrong and misleading?

All this is part of thinking such things through. It is not in any way intended to ridicule what, really, is a perfectly appealing desire. It is intended to open up the consideration of what goes into such an apparently simple notion. Each and every such question I raised above and the (probably) hundreds of other similar considerations that go into such a notion is absolutely, guaranteed possible to do. It all can be resolved into a very smooth implementation.

But the question is, at what cost, and for what benefit. If the benefit is slight and the costs are huge, are there other areas in which the resources should be applied first?

I don't mean that to discourage new ideas. But, as part of any new idea the next step after initial utterance of the thing is to dig deeper and to do the thought experiment to ask yourself "How would this actually work in practice..." so you can specify the detail of what you want. You have to do that anyway, so may as well do it at the beginning to know if a) maybe there are better ways to get what you want, and b) maybe the whole set of issues involved is not worth the benefit it brings.

dyalsjas96 post(s)
#07-Feb-18 00:19

Concur, and I have no basis to offer an opinion on the "effort / benefit" analysis to the Manifold development process.

Perhaps only data sources embedded in a .map file should be persistent.

It might make sense to make enterprise data sources persistent if the query is also stored on the same enterprise datastore.

It may never make sense to make web data sources persistent.

I'm not sure about the difficulty in identifying changes to persistent objects in the query builder pane.

Well coded and commented queries are always better, but SQL may not be the primary interface for all Manifold users (especially as the transform templates increase in variety).

Esri (through Model Builder) and Safe (through FME) both offer graphic programming interfaces. I personally wouldn't describe them as elegant, but they are functional, persistent, and mostly usable by a non-programmer.

The "geospatial programming studio" flavor that began with Radian Studio, is maturing in Manifold 9. The current interface is certainly clean and fast; and the emphasis on SQL is completely understandable in that the software is fundamentally based on the Radian SQL engine.

An analogy that makes sense to me right now is that ArcGIS is like a fully loaded luxury car with a 4 cylinder engine. Radian Studio was a National Hot Rod Association drag racer with 16 cylinders and just enough instrumentation to let a professional driver win races consistently. Manifold 9 is like a Shelby Mustang or Dodge Viper, enthusiasts love it. How do we get to Bughatti Veyron, or Aston Martin Vanquish where everyone wants it because it screams power and luxury?

I'm still uncertain to the priorities of Manifold development; enhancements to the graphic data interface, enhancements to the programming interface, or closing capability deltas between Manifold 8 and 9.

I appreciate the Edge build process immensely. The problems I see being solved in the forums are fascinating!!

Personally, I'm hoping for more elegance; I enjoy GIS more than data science.

tjhb

8,273 post(s)
online
#07-Feb-18 00:28

I was very interested by the background thought. That makes sense.

My background thought for persistent objects in the query builder pane is that it would be easier to discern the objects and tables in a query if you are revisiting the query after a time away from the project. Another use case would be if you have shared a project .map file or have stored a query in a enterprise datastore.

It's similar to the thought I had for this suggestion, from a very early beta thread (limited access). Attached as PDF.

Attachments:
20140918 GUI feature request - highlight query sources.pdf

tjhb

8,273 post(s)
online
#07-Feb-18 00:35

That old suggestion is probably not quite right either (I only included it to show that we are thinking in the same way), but my current thought is that the purpose is best met within query text, in comments (as Klaus also says), and not in the Tables panel, which has a different purpose.

If there are ways to make that easier--well, let's all think.

adamw


8,139 post(s)
#06-Feb-18 08:43

But why would that be useful? It just sounds like clutter.

To extend or fix an existing query that you want to build on. Ie, you wrote a query last week and it works fine but today you want to add a couple more fields to the SELECT list, or you want to add one more filter to WHERE.

That's just thinking out aloud, we don't think this is high priority because it is easy enough to drop the tables you want from the Project pane manually (as in, the time to locate them in the Project pane and then drop into the query builder is likely a very small portion of the time you're going to spend adjusting the query). Plus there is a fairly big con in that this might not work well if the query is broken (because some tables got renamed, etc), and, like you say, that's one of the big reasons to start editing it in the first place.

Manifold User Community Use Agreement Copyright (C) 2007-2017 Manifold Software Limited. All rights reserved.