Subscribe to this thread
Home - General / All posts - New video - Closest Rasters
Dimitri


7,413 post(s)
#08-Jan-20 12:44

There's a new Closest Rasters - Path video in the video page. This shows the last example in the new Transform: Closest Rasters topic. There's also a quick demo of Rename Related at the very end.

The main intent of the video is to show how quickly workflow can be done using the new "closest" tools in conjunction with other Manifold tools. It's easy to experiment, to learn by doing, when you know you can quickly try something to see what happens.

If you just do the GIS work without counting the time for extra discussions, it's easy in under five minutes to create a slope surface from a terrain elevation raster, to create a closest path length surface based on slope costs, to create watershed lines on that surface, to create a path source raster based on the slope, and then to trace areas in that raster to create a drawing of area objects to give you borders.

That's an incredible amount of stuff to get done in five minutes, and those five minutes include the time to style the rasters and drawing.

dchall8
1,008 post(s)
#08-Jan-20 20:11

That was fascinating. We had a historian come into the office for hillshade maps I was making from LiDAR dems. His interest was in locating the current land ownership for silver mines abandoned by the early European explorers, and also for the trails leading to the mines. The Indigenous Americans (IAs) had been wearing paths through the hills for thousands of years. The paths were used for hunting, travel, and trading between IA settlements. The explorers used the same trails but also ventured into the hills looking for natural caves to mine. My customer wanted to know the likely paths the miners might have used from the main roads to the mines. The mines didn't have silver, but when they were abandoned, they left tools, weapons, and graffiti. The trails also have artifacts which have been found by current land owners. So the historian ultimately wanted to get permission from the owners to enter and explore. The analysis presented in the video would have helped me help the historian to find the shortest, least cost, pathways from main road to mines.

antoniocarlos

609 post(s)
#09-Jan-20 12:28

Hi

I really enjoyed the video. Been waiting for these capabilities for some time. Some recommendations.

1. The time it takes could be reduced if one did not have to fiddle with styling or with renaming the outputs

2. I would like to see the output immediately after the process is finished (undocked).

3. Legends are a must.


How soon?

Dimitri


7,413 post(s)
#09-Jan-20 13:15

or with renaming the outputs

If you don't mind the default names, you don't have to change them. Personally, I'd like to see them in a box in the main pane, not within the Output button.

Not fiddling with Style is a fine goal, but other than showing it in a gray scale or with some default palette, what would you do as a default? It's rare that one size fits all to provide a sensible display for data, but I agree this would be worth exploring.

As always, send in suggestions for what you want.

Mike Pelletier

2,122 post(s)
#09-Jan-20 18:01

It is very nice how when changing the name to the drawing, the table name adjusts as well.

If a default style works 70% of the time reasonably well vs. the current default greeyscale working 10% of the time it is well worth it.

While these little things mean a lot in how enjoyable the software is to use when doing repetitive tasks, there are so many more important things I'd like to see added that it is hard to want to write a suggestion for fear of delaying main things. Also, I'd assume this sort of thing would be obvious to internal Mfd users that it would get done after more pressing items. On the other hand, perhaps internal Mfd users use it for testing and don't see the annoyances associated with repetitive use. Maybe all this is obvious and not worth the post :-)

tjhb
10,094 post(s)
#10-Jan-20 00:37

It’s always worth the post Mike! For example, by contrast, my worldview is that Manifold 9 is now practically complete. Continuous improvement is all that is needed from now on.

In my opinion, what we must concentrate on now as a community is not further product development but use.

Manifold could help here too, by providing a venue, and maybe structuring incentives, towards the cataloging, sharing and trading of task-specific add-ins.

As far as a default greyscale/single channel image style is concerned, all it takes is for one person to suggest exactly what that should be, using current Manifold style syntax (JSON), and for others to offer broad agreement. Any ideas?

vincent

1,972 post(s)
#10-Jan-20 18:34

my worldview is that Manifold 9 is now practically complete

Surprising. It is the worst GUI never seen in GIS history. Everything useful in M8 seems to be gone.

Powerful, yes. Enjoyable ? no. Intuitive : zero.

M8 was advertised for everybody and was intuitve. M9 requires a post-doc and a month of reading before using it.

I'm amazed at what it can do. But I'm greatly sad about the time needed to understand it. This amount of time is not available for most GIS Analyst having work to do everyday. So it will remains something for students and teachers I guess.

It like I waited 7 years for an electric car. Not it's there, but I don't think I will ever have the resources to buy it.

tjhb
10,094 post(s)
#10-Jan-20 21:41

That sounds a bit harsh. Some of it may be misplaced or overstated, but is written in the service of making a clear point, so is all fair.

You are probably not the only one who has this sort of feeling about 9.

If I were translating what you have said into one sentence, I would say:

Manifold 9 is genius technology but feels somewhat out of reach.

In reality it is not out of anyone's reach, but if 9 has a perception problem, or poses barriers to learning, then those things need fixing.

You Vincent among others are really well-placed to suggest how.

So, putting aside new features for a moment, what else is missing? What could help to make 9 more approachable, in your opinion?

Think about things like: a physical textbook, a cheatsheet, a diagram for the new object model, a categorized plan of SQL functions, a how-to guide or "cookbook".

Some of that is self-interested on my part, since I would like to produce one or more of these, though not alone--and I say this as someone who very much admires the existing online manual.

What would help?

Mike Pelletier

2,122 post(s)
#10-Jan-20 22:30

Well said Tim.

I think the GUI is very tough in the beginning because it relies on ctrl, alt, and shift greatly and is a big departure from other GIS. The former can make it fast in knowledgeable hands but a showstopper otherwise.

In depth reading will be needed but two suggestions to get people launched or refreshed. Perhaps a little bit of short paragraph instruction on concepts in the Context Help would be good. Secondly, what if Dimitri did a general video that was also linked under the Help drop down.

Not sure if this could be done succinct enough but here's a general outline with the idea of hitting the main tools in each:

open project > open map > navigate tools > context help (speak to alt, ctrl, shift) > alt-click to get info > drawing tools > layer pane > style vector > style image > transform > open table > filter > select > order > transform (sample table gymnastics) > mention SQL > open layout > print to PDF.

Dimitri


7,413 post(s)
#11-Jan-20 05:00

In the "great minds think alike" department, it's funny that before I visited the forum this morning the first thing I did was to jot down some suggestions for videos. For some reason, I woke up thinking "there needs to be a new series of videos called "Getting Started" or "Basics," a series of short videos that could be taken in order. Your synopsis "open project > ..." is exactly that idea.

Perhaps starting with an "Introducing" video that's an updated version of the "Introducing Viewer" video, subsequent videos could each take on something basic. It's best to keep videos under 5 minutes, given that human nature online seems to prefer watching five videos that are five minutes each instead of a single video that is fifteen minutes. :-) You could then also have videos that look at various items in a step up from a total beginner.

What I wrote down this morning was for things like "Drawings", a video that shows how drawings are just a viewport for data in a table, copy / pasting a drawing to create copies that can be styled differently (or just creating a new drawing from the same table), creating multiple drawings from the same table that use different fields in the table, and so on.

You could do that for "Tables" as well, showing how to use multiple geometry fields in tables, so you can have for each record an area object, borders, centroids, all in the same record so selecting one selects all, and re-cycling attributes. And then show how all that can be done on the fly with a simple query, from which drawings can be created, etc.

That gets me to a different theme, how the GUI is different. That's not a simple discussion and worth a post of its own.

tjhb
10,094 post(s)
#11-Jan-20 06:55

With respect, that assumes, wrongly, that videos (that is, screenshot-plus-audio) are a good learning and product-appraisment tool for the general case.

They are not.

tjhb
10,094 post(s)
#11-Jan-20 07:58

It's only fair to explain further what I mean.

Voice-over-screenshot videos make these assumptions (among others):

  • That the viewer is interested in anything and everything the presenter has to say
  • That the viewer trusts the presenter to say what the viewer wants to know
  • That the viewer has at least X minutes of time (now) to find out whether the time might be wasted
  • Patience
  • Headphones or speakers
  • A conducive listening environment
  • Enjoyment or acceptance of the sound of the presenter's voice
  • Enjoyment or acceptance of largely static shots of computer monitors
  • Competent aural understanding in the presented language, in the absence of text

That is a lot to assume.

I tend to fail on the first four points. Sometimes I persist, and fail or succeed on the others. (I always fail if there is also musak.)

By contrast, with well-written text, possibly with pictures, I can survey and skim. Flick, flick, yes, no, go back, go forward. Let's read this thoroughly, or perhaps let's try elsewhere first and come back if necessary. I am in control of my attention and my time. Lightning speed. And I can remember it.

If the text is printed then--bonus--I can also write on it, join in.

Dimitri


7,413 post(s)
#11-Jan-20 10:57

I personally don't like sites that promise to tell you how to do something and then launch a video. I much prefer written, step by step documentation.

With Manifold the commitment is to get everything down in writing in the documentation, at least as some sort of technical summary or guide initially, and then fleshed out with some examples and illustrations wherever possible. One advantage of that is the presence of online translation tools, so at least people who don't speak English as a native language can get the gist of the topic. Documentation is also far easier to update when small or big things change.

Videos are an optional extra. I think they can be useful for many people, but documentation has to be done first. Videos are way harder to update, because basically they have to get re-done. There are about 110 videos right now and I think 90% need to be redone, with probably around 40 way out of date.

ColinD

2,081 post(s)
#11-Jan-20 08:53

Keep the videos coming Dimitri, I am always time challenged and find them very helpful summaries. I like the idea of the introductory series as well. I'm fairly sympathetic to Vincent's original post.


Aussie Nature Shots

Mike Pelletier

2,122 post(s)
#11-Jan-20 16:54

Good to hear and Tim's cautions are good. I'd like to add that often trying to describe the action with words slows down the cursor and learning. Your doing great with videos and it is not at all easy to be clear and concise, especially given we all need to hear something a bit different. From my perspective, perhaps focus on keeping the cursor moving slow but consistent and minimize the temptation to sell the product. Never seen this done, but it would be nice if the video could have bookmarks somehow.

Also, would like to say Adam's post below describing what is being worked on is really good. He has probably said all that in various threads but to hear it repeated as an update (still in the works) is very encouraging.

A specific suggestion is that it is very common to change the border width of polygons but far less so to change the width of polygon internal line patterns. Would it not make sense to put the former on the initial style pane rather than the later?

Dimitri


7,413 post(s)
#12-Jan-20 08:54

Would it not make sense to put the former on the initial style pane rather than the later?

Can't speak for others, but I sure agree with the above. I think that's a classic example of a small tweak that makes for smoother and faster functionality. Wish I'd thought of it. :-)

Dimitri


7,413 post(s)
#13-Jan-20 07:31

Aha... well, here we go: This turns out to be a classic case of needing to be careful when tinkering with user interfaces, to avoid losing something valuable in the process of getting something you want. It also illustrates how because 9 has way more capabilities than 8, you have to think through how to deliver a simple user interface that retains the ability to leverage greater capabilities.

My initial reaction was, "This is great: just wire up the "size" button for Area style. You can then click the button and pick 1 or 2 or whatever and that's what you get in border size. But what I forgot in that snap judgement is that "size" in 9 is an abstract idea that powers very many other style parameter options. It's not just a single number in points all the time.

Making it a single number in points for just one parameter loses is the overall idea of an abstract "size" that can be interpreted by various style settings in the Style dialog in four different ways: as a number (automatically choosing points), as a number with a unit, as a % sign for percent, or as a relative size specification using the @ sign.

Having an abstract size number that automatically can be interpreted by very many different style parameters, like symbol, border, inner pattern, inner border, outer pattern, outer border, and the many settings (stroke, offsets, halo padding, etc.) is a very big and useful thing. It makes it possible to set up a sophisticated style that automatically scales all such relationships when you click the Size button for area style. Change it from 2 to 3 and over a dozen other parameters automatically scale correctly, using specification such as %.

If you hard wire that Style pane button to just a single parameter, the width of the stroke line for border, you lose not only the ability to specify stroke line for border in % and other options other than a single number, you also break how an entire constellation of style parameters automatically can scale from that single parameter.

Short term "solution" -

Here's how to get what you want right away, using a technique that uses such "constellation" capabilities to do what you want:

1. Open a new drawing. Click the "all styles" button for areas and then choose More... to launch the Area Style dialog.

2. Click on the Border tab, and then change the 1 in the box to 100%. Change the "size" parameter under the preview from 8 to 1. Press OK.

You've just defined a custom style for Areas where the Border stroke thickness is 100% of the default size parameter, and you've set that default size parameter to 1 instead of the default 8. It's normally at 8 since by default only those Symbol styles that are hatch patterns use it, and it is not used in the Solid symbol style.

Now, when you change the size number for areas in the top level dialog, the border stroke width will automatically change, just like it does for lines. If you want to use hatch patterns, when you choose that symbol you can set those up to react however you like to the size parameter.

Long term solution -

The short term "solution" sets up a drawing so that clicking the Area size button automatically changes border size. That facilitates subsequent tinkering where you might want to change the border width repeatedly, such as copying and pasting the drawing and using different formatting in the different drawings. That's great, and it does avoid having to drill into the Border tab of the Area Style dialog each time you want to make that change. But it does mean you have to drill down into that dialog at least once.

The long term solution is what's in progress anyway for other reasons: the ability to specify default styles for new components, both those which you create and those which are created automatically, for example, as a result of imports.

If you click on the "all parameters" big area button in the Style pane (the leftmost one), you'll see a list of prepacked style configurations you can choose. Manifold could add another one that used 100% for border stroke width and size 1 for Solid style.

Manifold could also add a Default Style dialog, or perhaps better, a Favorites dialog for styles, where one of those favorites can be designated as the default for new components. The default could create all new drawings using 100% for border stroke, etc.

You'd then have what you want: when creating a new drawing, it would automatically be created so border stroke for areas is 100% of the size parameter, set to 1 for solid style. No need to drill into the Area Style dialog, not even once, to get that.

Allowing customization of what styles are used by default could also do things like automatically assign some color from a chosen palette, for those who want all newly created drawings to use some colors other than the default gray.

Mike Pelletier

2,122 post(s)
#13-Jan-20 17:36

Not surprised that there was complication to this at it seems a pretty obvious desire. Thanks for the short-term fix.

The ability to set Favorites and default would be without a doubt the best. It would save a lot of time as each of us fine tune what styles work best with our printers and background images.

The capabilities of the Style Pane are super. One challenge though for new users I think is the concept that setting items like fill color via the "More" drilldown overrides fill color in the main top level dialog.

Here's a few thoughts that came to mind while messing with the style pane that might be helpful:

- The size and rotation buttons in the top dialog could be made smaller to open up space for another parameter.

- Perhaps make use of the blank space in the line top dialog for another parameter, maybe the halo padding width or left/right offset.

- Size parameter types/units do not always seem apparent and perhaps should be always shown.

Dimitri


7,413 post(s)
#11-Jan-20 10:50

This is a long post, but I think the importance of the topic and the opportunity for us all to work together to get what we want is worth more depth.

Before I discuss the GUI, I'd like to touch on what I meant by "infrastructure" in other postings: I meant the internal machinery that makes all else possible.

If the internal machinery can handle vectors but has no data structures or ability to handle rasters, then it doesn't matter what GUI or features you want to change things like hue or contrast. First, you have to have the infrastructure to store rasters, to get at them fast, and to compute how to render pixels on screen from what you get. If you have that and the supporting machinery to utilize those capabilities, then you can go on to decide how you want to offer GUI controls to manipulate contrast, hue, etc.

Three years ago the first betas of 9 were lacking infrastructure for some fundamental things. Since then the core internals and supporting machinery have grown to the point where pretty much anything people do in GIS or in related niche areas like remote sensing imagery manipulation can be done.

For example, there is infrastructure for Z coordinates in addition to XY, so all the infrastructure is in there to do things like 3D scene visualization. Anything that people want can now be implemented as features. That's what I meant by infrastructure being pretty much complete.

About that GUI.... let's launch from two observations:

if 9 has a perception problem, or poses barriers to learning, then those things need fixing.

I agree, of course, and

I think the GUI is very tough in the beginning because it relies on ctrl, alt, and shift greatly and is a big departure from other GIS.

...also something to consider carefully. When tuning the GUI to be what we want it to be, going forward it should not lose those parts where it works very well, better than alternatives.

For starters, sure, the GUI is different, as most truly new things are. It is different from other GIS for two reasons: First, because the capabilities are different, and second, because of how the product evolved.

I'll take that second point first. Most classic GIS packages are primarily viewers and display engines at heart. They don't have any DBMS within them, but are designed to display what's linked in from external files. If those external files or sources are a DBMS, well, then the GIS can be a client to that and do some DBMS things by calling on the external source. But they don't really have a way of bringing data into their own, internal world of full-power data storage and data engineering.

That is one reason why most classic GIS packages are full of viewing features such as formatting and presentation but are light on data engineering features. Arc, for example, can't even select on the fly based on, say, an expression computed on geometry. You first have to create a new attribute column, that literally creates for each record the results of the expression you would use for selection, and then you can select on that field. Arc has no clean, effective GUI for data engineering, because it lacks core infrastructure.

Unlike classic GIS packages, 9 was built from a data centric perspective. That was primarily for three reasons: first, because of the feeling that the value of the data is the main reason GIS has big value over things like Illustrator, second, because doing the data side of it is far harder than the viewer / display side of it, and third, because if you want quick response and speed with contemporary data sizes in GIS viewer/display functionality, you can't get that without basing the whole thing on a high-speed, parallel DBMS. There was no choice about that.

As soon as the Radian engine came to life enough to be useful for data engineering, the product was released so the community that could use it could get the benefit of it and, also, help guide it. Guidance from users who were expert at data engineering helped ensure Radian infrastructure was built out to world class levels. Weak spots were eliminated, that otherwise later on would have caused a mess when the thing would be applied in GIS.

Most of those expert voices came from outside classic GIS, understandably so, since most data engineering expertise comes from DBMS and not from classic GIS. It therefore makes sense that early users prioritized capabilities that may puzzle classic GIS users. Collations are a good example. If you don't have the infrastructure for those, you really don't have an ability to do data engineering in a high quality, contemporary way. But I'd bet not one person in 100 doing GIS with Arc even knows what a collation is.

The initial focus on data centric infrastructure and features is not a bad thing, because all else is based on that. Get the data side done right and you get speed, power, flexibility, and accuracy to do whatever else you want throughout data science and GIS.

As all the data centric infrastructure gets done, you can move on to using it for new and better GIS, but now you can do that GIS in a far more useful way. The key is to implement those GIS features in a way that GIS people will find them convenient, evolving into a GUI that provides visual/interactive GIS-style tools in addition to retaining interfaces that can make full use of very rich data engineering capabilities.

That brings me to how the GUI is different because the product is different.

It's important not to surrender to inertia and to just keep doing the same thing over and over because that 30 years ago was the best that could be done. If we did that, everybody still would be thumbing away on Blackberry devices with a zillion buttons, instead of using gestures on smart phones that have no buttons.

Classic GIS like Arc, QGIS, and 8 are "button heavy." If you want to do something, generally you click a button and have at it within a modal dialog that prevents you from doing anything else at the same time. There are no previews and no going back and forth to check something in the middle of, say, a transform session. It's always back and forth to yet another button.

Button heavy interfaces are similar to wizard interfaces, in that they can be easier to teach and to learn for simple things. They are also similar to wizard interfaces in that they usually are slower, more limiting, and way more annoying for intermediate or expert users who want to do more advanced things.

I don't think it is automatically necessary for a high end, modeless, light-on-buttons, interface to be hard to learn for users, or difficult for beginners. A good example is how 9 does navigation.

Navigation, how you pan and zoom to see what you want to see, is probably the most important part of a GUI for visual display packages like GIS, CAD, or graphics editors. It's the first thing you meet. For navigation, 9 is the best I've seen. Left click and drag to pan, and right click and drag to zoom box, plus scroll to zoom, are revolutionary: much faster and easier than how 8 does it.

With 8 if you want to pan the view you have to click a "hand" button and then you can pan. Want to zoom box? Got to click a zoom box button to do that. Most GIS packages are like that. 9 is far easier.

The downside? It's true that the very first time you lwant to pan and zoom in 9 you don't have a "hand" button and a "zoom box" button that you might think "oh, OK,... that'st the button to click." You have to learn about left mouse / right mouse as the path to dramatically faster, more convenient, and more intuitive panning and zooming. That's a microscopic amount to learn, for huge benefits in return, but it can be an obstacle if the package is either presented or approached in the wrong way.

That bring's me to Tim's comment, about perception.

I think the biggest issue with 9 in terms of perception is that many people who first jumped into 9 were 8 users, and few of those 8 users either heard or heeded the advice that 9 is totally different: instead, they just jumped into 9 thinking it must be just like 8, only faster. Pop it open and start punching buttons you expect should be there. As Mike correctly put it, that's a showstopper even in something as simple and so much better in 9 as panning and zooming.

Manifold didn't help by naming the new product "9." Given the numeric name, people expected it to be another version of 8. That possible effect was considered. In fact, at first there was talk of totally new branding. At one point it was thought the product would fork into two products, one for the data world called Radian and another for the GIS world called Polygon (both of which are registered trademarks Manifold owns, initially used as trade names for OEM products).

Launching two different products would have enabled a full commitment to the data side for advancing a product into the DBMS and IT world, and another for a purely GIS world. It also would have nipped in the bud any notion that 9 is just a newer version of 8.

That didn't happen because it was felt wrong to deny GIS people a full-power data engineering product, and wrong to deny DBMS people a full power GIS. The two just naturally go together. So, as GIS features were added there was no fork in the road, just a totally full power product for everyone going forward.

I think the key for perception issues now is a two part plan.

The first part is a new, initial presentation of the product in terms of a learning experience. I think that's best done with a new series of initial "Getting Started" and "Basics" topics, that work together with a new series of short, introductory videos.

The second part is what is going on right now in builds, where much attention to adding useful features and evolving key details to make command use smoother is happening in build after build. A good structure that minimizes clicks for maximum speed and power in interactive commands makes 9 more productive for experts and easier to learn for beginners.

But that will take some commitment from beginners. I think the best example of how a product tuned for regular users can be a spectacularly wonderful product while being famously difficult for beginners to learn is PhotoShop. If you want to be mean, take somebody who knows computers but who doesn't know PhotoShop and sit them in front of it and ask them to draw a straight line. People who don't RTFM can't figure that out. For all that, people who know graphics editors will tell you PhotoShop is still the gold standard. It really is highly effective and delivers wonderful productivity, in the hands of somebody who has learned to use it.

That's true of all the very best applications which do sophisticated, powerful things. They don't trade off power in the hands of regular users for interfaces that may be easier to learn but ultimately slower and more limiting.

I don't think there are that many features left to do to get to "completeness" in feature set: adding a couple of hundred features, with a mix of small things like a split tool, medium features like georegistration, and more advanced features like very elaborate and full featured legending, and you pretty much run out of "must do." In fact, I think most people would be hard pressed to come up with a list of 50 "9 really needs this..." items.

A couple of hundred features isn't a lot considering what's been done to date and the continuing issuance of builds every two weeks. That's something the community in this forum can help make happen.

There is also a continuing balance, with frequent builds an opportunity to tune the balance, between a more data centric approach and a more GIS approach to how a particular command or feature is implemented. Small changes are always easy, and often the path to a very smooth GUI is many small changes and details that work together. The community can help there.

How to Help

1. Keep sending in lists in priority order of Top 5, Top 10 and similar you feel 9 must have.

2. Never hesitate to send in small notes where an existing feature can be improved. This usually requires mastery of what is there today, to get a genuine improvement that doesn't throw away valuable aspects of how it is done today.

3. Send in short summaries of what would be helpful learning materials. For example, an outline of which ten topics you think would be a super "Getting Started" section and an outline of what each of those ten topics would contain. Likewise, what would be the analogous set of ten, five-minute videos?

It really helps to get a guide to a new introduction by documentation and video. Manifold documentation uses many illustrations, so small changes in the GUI can require re-shooting hundreds of illustrations in addition to changes in dozens or even hundreds of topics. I think there are now around 9000 illustrations and about 650 topics, so that's a lot to update for any changes.

I think a steady process of every build making small changes and adding more and more items from "must have" lists will fairly rapidly, in months, not years, bring 9 to a much higher level. Already there are synergistic effects as holes in the GUI are filled and connections for faster workflow are made possible, with each tweak or new addition having geometrically greater effect.

Presentation (initial learning materials) will catch up to that as the GUI settles down, and that will help people get to a quicker, easier start with 9, and with moving on to leveraging 9 to do their work faster and easier than is possible with alternatives.

We can then move on to greatly expanding the 9 feature set, steadily adding GUI features that leverage more and more of what the infrastructure makes possible, to do more and more things that are simply not possible in alternatives. There are many, many ideas for such things.

pslinder1
228 post(s)
#19-Jan-20 19:14

I love navigation in 9. The scrolling zoom and the click and drag for panning and zoom box are awesome. The awesomeness is amplified by the speed with which the display reacts (that is huge). Your point about a microscopic effort is maybe an understatement. Learning how to navigate in 9 took only about 15 seconds of frustration: first not being able to locate the "pan hand" and "magnifying glass" and second clicking around in a drawing layer until I figured out what was going on. After that 15 seconds I never want to go back. So well done. That learning speed is the proof and measure of intuitiveness. Other aspects of the UI in 9 I would say are less intuitive and cause longer periods of frustration.

One little niggling thing that keeps tripping me us as I use this program is the weird fact that in most screens the right and left mouse button clicks do nothing. They are such common actions with a mouse; learned and constantly reinforced from all software and web interfaces that it feels really jarring in Manifold 9 when nothing happens. Do you have some plans for these two common actions in the near future?

A much bigger source of frustration for me is converting a table into a drawing. One of the things that I often need to do (I mean once every two or three months) with Manifold is to import and view a table with lat/longs. Converting that table to a drawing in 8 was beautifully intuitive; Copy, Paste as Drawing, fill out the popup window. Simple and logical. In 9 you have to open the table schema, add a b_tree index (a little bit of arcana hardly anyone needs to know about), add a Geom field, save that (I often have to repeat this step because the OK button in the second screen does not actually save the fields), then open up transforms pick the Geom field and do "compose point" (the wizard almost never guesses the correct fields for lat or long; in 8 it almost always got it right). I suppose if I did this everyday it would become second nature but I don't and it hasn't. Is there a reason for this minor UI horror?

Manifold 9's best feature is its wicked fast speed. It is great in all things but the rendering speed really is the best thing from an every day use point of view. It is what makes 9 so much easier to use than 8 and a host of other GISs. I would say though that the UI is still a little hit or miss. On the whole it has improved on 8 but some things are just strange and in some places it has regressed.

adamw


10,447 post(s)
#20-Jan-20 09:40

Thanks!

We plan to add more context menus, yes. We agree that being able to right-click basically anywhere and get a list of commands appropriate for what you clicked is very useful. We have context menus in many places, but not everywhere we could have them - most notably we don't have them in many places in map window / layout window. We'll fix that.

Regarding converting a table into a drawing, first, we agree that Copy / Paste as Drawing in 8 was simple and the current workflow is more difficult. Now, let's see how we can improve the current workflow:

(a) open the table schema / add a btree index - you can just click the Add Identity button in the toolbar and then click OK, no? We are perhaps going to simplify this further and just prompt you to add a new identity index if the table is not editable (that's why we need the index in the first place). Also, since this mostly only appears for imports, maybe we'll add an option to auto-add identity data for imported tables, that way you won't have to bother with Add Identity at all.

(b) when you say that you have to repeat adding a geom field, could you provide some more details as to what happens? Adding a btree index and adding a geom field should complete in one step. If adding a geom field fails under certain circumstances, we'd like to fix that.

(c) the Compose Point transform does not guess the field names for X / Y - we can try to guess them similarly to 8, agreed. We'll see about adding it. Ultimately, the Compose Point transform could perhaps add a new Geom field automatically. We'll look into this as well, we have a couple of wishlist items that could benefit from transforms adding new fields as well.

Thanks again for a very helpful post.

pslinder1
228 post(s)
#21-Jan-20 18:36

Sorry for the late response. I was not at my computer much of yesterday.

I am chuffed to hear about your plans for extending the applicability of the single right mouse button click. How about for the left mouse button?

(a) Actually I did not notice the "add identity" before. That's a cool little shortcut. But the larger point is still why would anyone new to Manifold 9 even think this was required. Glad to hear you are considering implementing the identity automatically. Doing it at the import is fine. I still think you need to have a more obvious way of turning a table with lat longs into a drawing. Its such a common (and novice) thing to do that you should not make noobs (or anyone really) do a "compose points" in Transforms. I vote for a popup that automatically creates a Geom field when you choose the "Create drawing" option in the resulting context menu from a right mouse button click on a table in the project pane.

(b) The problem of the repetition is my own. You have the schema popup box open, you create a new field which opens a new popup box, you edit that and click "OK". What you are supposed to do then is click on "Save Changes" in the original schema pop-up; instead my lizard-brain often assumes that I have already saved things because I clicked "OK" in the second popup box and I accidentally click "Close" in first instead of "Save Changes". I know this is my "fault" but as I find myself doing it often enough I wonder if there is maybe something inherently un-intuitive about the process.

(c) That would be great. But again (see 'a' above) I would add this capability directly to the "Create Drawing" option for a Table instead of making the user go to the transform pane first to create a Geom and then going to "Create Drawing". Do it all at once.

I know this is all super minor but somehow I often find that little irritants sometimes can have an outsize effect on my perception and general satisfaction.

adamw


10,447 post(s)
#22-Jan-20 13:52

Regarding using the left mouse button more, what specific things you want to click and what this should do? We see where we are under-using the right mouse button, but we don't see where we are under-using the left one.

Regarding an option to create a drawing from XY / XYZ fields right in the New Drawing dialog - thanks for this suggestion, we think this is a very natural place to put that functionality indeed!

pslinder1
228 post(s)
#22-Jan-20 14:29

What does a single left mouse button click do in a drawing? Nothing as far as I can tell. How about in a table? All it does is put a little arrowhead in the grey box heading the row that you clicked on and puts a little grey dotted line around the cell. What is the utility of any of that?

In most other programs a single left mouse button click usually selects something.

Dimitri


7,413 post(s)
#22-Jan-20 15:10

What does a single left mouse button click do in a drawing? Nothing as far as I can tell.

A single click is a safe way to move focus to that window. Suppose you have a dozen undocked windows. Clicking into the window you want to use moves the focus to that window so that panes like the Layers Pane and whatever pane is up in the Contents pane now work on that window. By "safe" it doesn't trigger commands or, for example, destroy context for things like alt-picked objects. Say you have two drawings full of objects, both undocked for ease of use. You've alt-clicked an object in one drawing, and then to compare its coordinates to an object in another drawing you click into that other drawing and alt-click the object of interest. You can now change focus with a click into either drawing without losing the picked objects.

The left mouse is used to pan the view in a window by clicking and dragging. It's an extremely fundamental move, and using a single left click to trigger a command, as opposed to moving focus to that window, turns a safe panning command into a risky move.

Having fast and fluid panning and zoom box in Manifold is a very big deal. It's much faster than "click a button to enter panning mode" and then "click another button to enter zoom box mode".

In most other programs a single left mouse button click usually selects something.

No, not in PhotoShop for example, which for many of the same reasons as Manifold uses a single left mouse click as a "put focus here" command.

How about in a table? All it does is put a little arrowhead in the grey box heading the row that you clicked on and puts a little grey dotted line around the cell. What is the utility of any of that?

Same as with windows: moving the focus to a given cell and row and column, for example, for Edit - Find operations, which automatically load the context column. The "little arrowhead" is a big deal, because it marks a row for provisionary editing, which you need to be able to do to make edits involving multiple fields for constraints, for example. See the Editing Tables topic.

pslinder1
228 post(s)
#22-Jan-20 19:07

First of all I totally agree about the fast panning and zooming. That is a great and awesome thing. But the click and drag is a very different thing than a single click. I think you over state the potential harm of a "risky move".

No, not in PhotoShop for example, which for many of the same reasons as Manifold uses a single left mouse click as a "put focus here" command.

I don't use photoshop. But in Word a single click selects a cursor location. In Excel it selects a cell in a meaningful way that allows you to copy or paste or whatever. The cntrl+click that you guys do for selection is kind of breaking the MS paradigm at least in tables. A single click should act as a select only-this (if something else was previously selected it is unselected) and cntrl+click should add to or delete from the selection. Manifold has no select only-this function which is very weird in a table like setting. Not wrong necessarily but unusual and therefore uncomfortable. What pushes it into annoying territory is that there is really no good reason for this.

You've alt-clicked an object in one drawing, and then to compare its coordinates to an object in another drawing you click into that other drawing and alt-click the object of interest. You can now change focus with a click into either drawing without losing the picked objects.

This is not a problem in Excel. I often have two or more excel workbooks open at the same time. When you switch between them the first single click gives you the focus and any subsequent single click gives you selection. I think they do it this way because selection is done way more often than focus change.

It seems to me that Manifold is breaking a paradigm that almost a billion people (over a billion if you add in google sheets) are familiar and comfortable with for no real benefit.

Same as with windows: moving the focus to a given cell and row and column, for example, for Edit - Find operations, which automatically load the context column.

Sure but the cell could be selected as well as just having "focus". Ready for copying and pasting and everything else.

The "little arrowhead" is a big deal, because it marks a row for provisionary editing, which you need to be able to do to make edits involving multiple fields for constraints, for example.

provisionary editing. Doesn't that seem like something that would be better as a right mouse button click context menu option. You are basically breaking the paradigm of every spreadsheet program to provide provisionary editing. Is it so common that it is worth the prime mind share of a single left mouse click?

adamw


10,447 post(s)
#23-Jan-20 16:01

The reason we don't select data on a simple click like Word does is that we are dealing with many more objects than Word, that we can select by much more sophisticated criteria, and that we are frequently looking at the same data from multiple windows and the user is switching between them. We protect the selection more than Word does - by requiring Ctrl-click instead of a simple click to alter it - because establishing the selection you want in Manifold takes more time than in Word, and that happens because spatial data is richer and bigger than text.

If a simple click would replace the selection, every time you used a template in the Select pane to select the objects you want and would like to switch back to the window to check what objects got selected, you'd be afraid of clicking within the window rather than on its caption, because that would simply erase the selection you just created. Moreover, if you wanted to zoom around to check whether objects in that corner of the map got selected or not, how would you do that? It's because of multiple windows and bigger / more complex data that we keep simple clicks and drags (on anything that is not a button) completely safe.

We hear you on using repetitive clicks (click once to move focus, click again to do something) instead of clicks with modifiers - we agree this can frequently be cleaner and we do that in places already, see vector editing for example.

With provisionary editing and how often it happens - whenever a database contains a field that cannot be NULL and cannot auto-fill itself, adding a new record needs to supply the value for each such field. This is a common scenario, yes. There are other scenarios as well - eg, fields whose values depend on each other.

pslinder1
228 post(s)
#24-Jan-20 03:23

Fair enough. But I would ask: Are you absolutely sure that this selection fragility is a problem for most people? I ask that because in MapInfo a single click selects and item and I have never found that too problematic.

Moreover, if you wanted to zoom around to check whether objects in that corner of the map got selected or not, how would you do that?

In MapInfo I would do this with the zoom out on the scroll wheel. You could also do this by clicking on the hand icon and panning with out triggering the selection. Now I know you will say that in Manifold we use click and drag for panning so that option is not available. Maybe, but a click and drag is a different action than a single click so I do not see why a pan also has to equal a select necessarily.

Now if you after all of this you still think selection protection is critical in a drawing/map then I would suggest the following compromise. Use a single click to center the window on the place where you click and use a rapid double click to do a single select. Then ctrl+click would remain as it is to add to or subtract from a "selection collection". I think it is really important that the single and double click actually do something. They are the most common user actions in almost all Windows and web programs and it is uncomfortable and emasculating when they do nothing.

This is a common scenario, yes. There are other scenarios as well - eg, fields whose values depend on each other.

I will take your word for it. Is the way you implemented provisionary editing (you really should change that to "provisional") a common standard in the DBMS world? If not I have suggested another way of implementing it that might make more sense in my response to Dimitri's post below. The method would still give you the capability without breaking the common windows table selection paradigm for a left mouse button click.

adamw


10,447 post(s)
#24-Jan-20 07:59

I think making selections fragile would be bad, yes.

It boils down to how you are using the selection and that is partly dictated by how complex your maps are. If a drawing contains a hundred of objects, it's all simple - you select a couple of objects, you understand what is selected and what is not, so you run a command on the selection confident that the command is running on the objects you want. But if a drawing contains a hundred thousand of objects, it changes - after you select a couple of objects using either a visual tool or a select template, you frequently want to check whether your selection criteria was good enough to select everything you want (eg, to change with a command) and nothing you don't want. This means that you go and inspect the map looking into various regions, maybe click some of the selected objects and inspect their fields, maybe click some unselected objects next to the selected objects and inspect their fields as well, maybe zoom to the extent of the selection, maybe scroll along some rivers or roads in a different layer to check visually, etc. This all to avoid running a command and changing a wrong set of objects, because if you do that and notice the error later, it will be harder to fix. Running a command on a big set of objects can also take noticeable time, that's another reason you wouldn't want to run it on a wrong set.

So, it all depends on how often you find yourself in the scenario where you want to inspect the selection - even cursorily - prior to using it. We find that this is somewhat related to the size of the data set: the larger the data set, the more often you want to inspect prior to committing to the action. Now, when we are designing the UI, we don't know whether your data set is going to be big or small, the UI has to work similarly for both. We err on the side of making the selection less fragile because erring the other way is costlier to people with large data sets (if a simple click erases the selection, you have to use a special tool to pan and even that does not save you from accidental clicks - that's much more unpleasant than having to press Ctrl to select) and because data set sizes have been growing a lot (just look at LiDAR data where it is common to have millions and billions of points).

Realistically, if we were to re-purpose simple click to do something, we'd re-purpose it to do what Alt-click does = pick an object, not what Ctrl-click does = select an object. Because picking a wrong thing is less destructive than selecting it. Maybe that's a terminology / thinking issue all along - maybe we should think of Alt-clicking object as selecting it (intermittent), and think of selected objects as objects marked for further operations (somewhat more persistent). Maybe we should provide an option to choose whether to pick an object using a simple click or an Alt-click, or maybe we should just use a simple click for that with no option. We'll think about it.

On to the record handles and provisionary editing - a lot of database clients simply don't allow editing records in tables with complex requirements for fields at all, you have to write a query to alter the record. Those that do tend to always edit a record as a two-stage process - change the fields you want, then commit changes by clicking some button. We use two modes - a simple one where you change a value and this immediately gets sent to the database, and a more complex one where you first click a record handle, then edit one or more values, then commit changes.

The issue here seems to be not so much about using a simple click for provisionary editing (that can move to a click with a modifier, yes, or we could use a toolbar button to switch between editing a cell and editing a record, no problem), but rather not using a simple click for selection. And the reason not to use a simple click for selection is the same as above - this makes the selection fragile and this is not good. We allow selecting a record using Ctrl-click. If we ever decide that a simple click in a map should select objects, we'll make a simple click in a table do the same, too.

tjhb
10,094 post(s)
#24-Jan-20 08:23

I have got a massive amount out of Dimitri and Adam's explanations of design principles in this thread, especially regarding the reason why a single left click has only one function, to change or confirm focus.

I had barely thought about it before.

To change this interface element to do something else would, in my opinion, be disastrous. For my part I would lose valuable work and data, every day.

This left click is so intuitive that it feels like nothing. But actually it is the firm foundation for all other work.

(I am all in favour of helpful right-click menus though.)

pslinder1
228 post(s)
#26-Jan-20 02:55

I have gotten a lot out of it too.

I disagree that currently the left button is intuitive. I think its lack of real activity cuts the balls off the bull. It is the most fundamental interaction (the power move) in almost all software and 99% of the time in Manifold it simply does nothing. A dead man's click. It is actually disconcerting. If there is valid concern about "selection fragility" fine but make that click do something. A replacement of the Alt+click, I think makes a lot of sense.

pslinder1
228 post(s)
#26-Jan-20 02:47

Ok. It seems that you are pretty firm on the "selection fragility" issue.

Realistically, if we were to re-purpose simple click to do something, we'd re-purpose it to do what Alt-click does = pick an object, not what Ctrl-click does = select an object.

That is an excellent idea! Much better than my alternative of centering the map. I support this. Please have a double click do something as well in the drawing/map window. In google maps a double left click zooms in one zoom level.

The issue here seems to be not so much about using a simple click for provisionary editing (that can move to a click with a modifier, yes, or we could use a toolbar button to switch between editing a cell and editing a record, no problem),

If the way you currently do it by picking the row handle is not some sort of standard then I recommend changing it. There is nothing about the workflow here that makes it obvious that what is happening is "provisionary editing"; realistically you would have to read the manual to figure out what is happening. I think if you make it a context menu option that brings up a narrow pane with two columns, one for field names and the other with values, it will be much clearer that the row is editable and when a "Save Changes" button is hit the row will be updated. I think that is much clearer; it also would allow the user to get to that capability from any cell in the row (not just the row handle).

Dimitri thought it would be a little slower. He is probably right but I think that is worth it for clarity on a non standard interface capability. I think the clues this approach gives off would make the capability discoverable and understandable without hitting the manual. Although, a lot of people still might struggle with why it is needed, they are less likely to accidentally stumble on to this weird behavior and wonder what is going on.

Dimitri also feels like it a new pane would block out some of the screen. But if a user needs to see something in another row on the table to edit his row he can move the long narrow popup out of the way. Again it makes the workflow a little slower but again I don't badly so. And again its still worth it for clarity's sake in my opinion.

adamw


10,447 post(s)
#27-Jan-20 09:19

Just to recap, what you suggest for editing multiple fields for a single record (new or existing) is this:

Click a record handle -> open the Record pane -> edit the values in the Record pane -> click Save Changes.

We are fine with this and have been thinking about using the Record pane with the table window for other scenarios as well, but I'll just quickly say that if I understand what you suggest correctly, then this is not a big change from what we already have. Because:

(a) if a particular record is in the Record pane, this presumably is best indicated in the table window as well - eg, with an icon on a record handle (like now);

(b) the values you change in the Record pane are perhaps best shown in the record in the table window as well (like now);

(c) you should perhaps be able to edit the values in the table window as well (like now), once you see a value you want edited, there's little reason to make you go to the Record pane to do so.

So, in essence, the current behavior will stay, we'll just get the ability to use the Record pane to see / edit all field values (which is a good thing).

Correct?

pslinder1
228 post(s)
#27-Jan-20 23:20

Perfect but I would also add the following:

Right mouse button click anywhere in a row->Select context menu option "Provisionally Edit Row" (Provisionary if you insist)->Open the Record Pane -> edit the values in the Record pane -> Click Save Changes.

The advantage to this is that you can get to the feature without know ahead of time that you have to click on the record handle in the table. A right mouse button click makes the feature much more discoverable, I think.

adamw


10,447 post(s)
#28-Jan-20 07:27

Agree, this is a good idea. (The command is going to be named just "Edit Record" in all likelihood. :-))

Dimitri


7,413 post(s)
#23-Jan-20 18:07

I very much appreciate that you're taking the time to drill into this, as I enjoy healthy debates on anything that can make a UI faster or better. So in that spirit, let's drill deeper...

I don't use photoshop. But in Word a single click selects a cursor location. In Excel it selects a cell in a meaningful way that allows you to copy or paste or whatever. The cntrl+click that you guys do for selection is kind of breaking the MS paradigm at least in tables. A single click should act as a select only-this (if something else was previously selected it is unselected) and cntrl+click should add to or delete from the selection.

It seems to me you have to compare similar systems. Excel just doesn't do what PhotoShop or AutoCAD or Illustrator or Manifold do in visual windows, so what may make good sense for Excel's text grid could be way off in a vector or raster graphics system like PhotoShop. It helps to see how acknowledged masters of the art do it... "standing on the shoulders of giants," and all that, so looking at things like PhotoShop can be helpful.

Tables in Manifold aren't spreadsheets, either. They're tables, like in Access or any of very many other databases. Those systems, including Access, use a somewhat different paradigm than click / ctrl-click as you describe in Excel.

Open Access and pop open two tables (use the books.mdb example database if you like), resizing and positioning the tables so both are open at the same time. Clicking into those tables actually opens for editing the cell that was clicked. So... you can't just click to change context without risking an edit.

OK, you can get away with that in a purely personal, consumer setting but it's way too unprofessional for more serious organizational databases where changes to data shouldn't be risked just on the basis of single clicks.

OK. Click a table, and now Ctrl-click somewhere else. There's no "add to the selection," the ctrl-click just results in opening a different cell for editing, as if there were no Ctrl modifier.

The general paradigm when selecting things in windows, like choosing more than one file in Explorer, is that Ctrl-click chooses an item without un-choosing any previously highlighted items. That's how it works in Manifold. More precisely, Ctrl-click (in Manifold and in Windows explorer and many other Windows settings) toggles whether something is chosen, without altering the highlighted status of other items. That's true whether it is just one item or many.

provisionary editing. Doesn't that seem like something that would be better as a right mouse button click context menu option. You are basically breaking the paradigm of every spreadsheet program to provide provisionary editing.

Like Access tables, Manifold tables are database tables. They are not spreadsheet grids. The job is to build a UI that works well for what goes on in database tables, which is different than what goes on in spreadsheet grids. So you'd expect some significant differences in UI details between the two to support the very big differences between databases and spreadsheets.

Interactive selections in tables are a feature in Manifold that you don't have in Access. There, a "pick" of a row by clicking the row handle is sort of like a selection, but only for that one row. You can't go on to add more rows with ctrl-clicks of more rows. Selection in Manifold tables is an extremely powerful capability, not something to give up. It gives you what Access has (can always just select one row and the Copy that, just like Access), but a whole lot more as well.

I could be wrong about this, but I don't think there's any such thing as a provisional selection of a pattern of cells in a spreadsheet, where all formulae / contents can be simultaneously edited in order to meet a constraint, before commitment. I suppose if there was, Excel would have to evolve a UI feature to make that quick and easy (like Manifold does).

pslinder1
228 post(s)
#24-Jan-20 02:34

I love the drill down.

I just took a quick look at the behavior of the "drawing screen" of Google Maps, QGIS, and Mapinfo. A single left mouse button click all does something different in each of these programs. In Google Maps a single mouse button click lays down a marker and removes it on the next click. In QGIS a single click centers the map on the point clicked. In Mapinfo a single click selects an object (record) in the layer. Only in Manifold does it seem to do nothing. I know that you think getting the focus is something but that only happens once after that every button press is a dead man's click. I think the three programs I looked at above are at least as relevant as (maybe more so than) photoshop.

Open Access and pop open two tables (use the books.mdb example database if you like), resizing and positioning the tables so both are open at the same time. Clicking into those tables actually opens for editingthe cell that was clicked. So... you can't just click to change context without risking an edit.

Ok this seems to be making my point for me. Access's purpose is more like Manifold's but you still think Access is more dangerous than Excel; fine use the Excel method and make the first click a focus click and then future clicks make cell selections. The point is that in Excel, in Access, in Mapinfo tables and in QGIS tables a left mouse button click makes the cell editable. In Manifold it does nothing active for the user. Why reinvent how everyone works with all other tables and grids?

I think your concerns about security are overblown. I don't think anyone feels that their data is safer because of a slightly awkward UI. If a mistake gets made once in a while that is what an 'Undo' button is for. In my experience very few errors are due to editing a cell that you did not know was selected and editable (I cannot think of a single time where that has happened to me personally).

I know it is a small thing but it is really jarring when the most basic user action (a left mouse button click) appears to do nothing. You immediately think either you are doing something wrong or the software is not working right.

The general paradigm when selecting things in windows, like choosing more than one file in Explorer, is that Ctrl-click chooses an item without un-choosing any previously highlighted items. That's how it works in Manifold. More precisely, Ctrl-click (in Manifold and in Windows explorer and many other Windows settings) toggles whether something is chosen, without altering the highlighted status of other items. That's true whether it is just one item or many.

Manifold Ctrl-click acts almost exactly as you say in windows programs (like Explorer). Where Manifold does not act like a standard windows program is a single click. (in Explorer it selects a file, in Manifold it selects nothing). The reasons why I say it kind of breaks the paradigm is that Ctrl-click adds to the selection but is also the only way to select a single thing at time; which is unlike most Windows programs where that is done by the left mouse button click.

I could be wrong about this, but I don't think there's any such thing as a provisional selection of a pattern of cells in a spreadsheet, where all formulae / contents can be simultaneously edited in order to meet a constraint, before commitment. I suppose if there was, Excel would have to evolve a UI feature to make that quick and easy (like Manifold does).

I do not know if Excel has this capability. Do any other programs handle this provisionary (are you sure it should not be "provisional"?) capability in the same way? Database programs perhaps? I had never come across this capability before and it confused the hell out of me when I first noticed the behavior in tables. After about 5 minutes of fooling around I realized that you could change multiple cells in the same row but I could not figure out why I would want to do this. I then went to the manual and you had a very good explanation of the feature. But it was definitely not obvious to me from the little arrow head that what I was getting was a provisional row editing capability until I played around a bit and then RTFMed. To my mind it is a very confusing implementation of a feature that if necessary is probably uncommon.

If the way Manifold implements provisionary (sp? even the spell checker on this forum does not think it is a word) is the standard implementation of this feature in database interfaces then I guess it makes sense if you want Manifold to be chiefly used by database experts. If there is no standardized implementation or if you do not intend Manifold to be chiefly a maven-only tool I think there is a much simpler and clearer way of implementing that "provisional" editing capability.

Make it an option in a context menu that when selected a window pops up with pane showing all the data for the row that will be provisionally edited in two columns; the name for each field in the record (row) on one side and the value for that field in the record on the other side. You can then make all the changes you want to the values and when ready you can commit those changes with a save record button or hit a cancel button. That would make it clear to even a noob what was going on and not break the standard selection paradigm in almost all table editing.

I am truly impressed by Manifold software and have always loved playing around with it since it came to my attention some 16 years ago. The power and flexibility it brings has always astounded me and no other package comes close to matching its value, not even the open source ones. The one frustration I have had throughout these years is trying to evangelize the product. The learning curve just seems to be too steep. Unless someone in my field (mobile network design and optimization) has a hobby-like interest in GIS, Manifold always seems more complicated as a "go-to" tool for people than ArcGIS or QGIS. I think the reason is because the interface takes some getting used to.

I know you may believe that such a powerful package requires you to invest significant time to learn it. I find that engineers that work for me understand that when you are doing something complicated or unusual you have to put in some time to figure it out but most abandon Manifold long before they hit that point. There is something more simply inviting about the interfaces in QGIS and Arc that make them discount the flexibility, speed, and power of Manifold. Even when they have to do something complicated that Manifold might excel at they will take an inordinate amount of time to try to figure it out in the other programs before grudgingly going to Manifold. I am not absolutely certain why, but I strongly suspect it is the user interface. It is just slightly awkward, not catastrophic, but just peculiar enough to be uncomfortable.

Anyway that is my motivation for all of this.

PS: The reason why I bring up Excel so often is not because it is the non plus ultra of interfaces but because it is so ubiquitous and has so much mind share that you deviate from its paradigm at great peril when dealing with data in tables or grids. Over a billion people in this world use spreadsheets how many use a DBMS? I would guess less than 10 million. Am I close? I am also pretty sure almost no one has started using Excel by reading the manual.

adamw


10,447 post(s)
#24-Jan-20 08:39

... fine use the Excel method and make the first click a focus click and then future clicks make cell selections. The point is that in Excel, in Access, in Mapinfo tables and in QGIS tables a left mouse button click makes the cell editable. In Manifold it does nothing active for the user. Why reinvent how everyone works with all other tables and grids?

I think that's a good idea - click a cell to make it the active cell, click it again to start editing it. Thanks, we'll consider it.

In general, we are all for making the interface more friendly than in any other GIS software, we are ready to invest whatever time and resources we need into it. We think there are tons of things one could do to make the interface easy to figure out without losing the power / flexibility. We are trying to do this all the time, it isn't easy, but we believe we have a lot of ideas on that front. Posts like yours help us a great deal, thanks for them.

pslinder1
228 post(s)
#26-Jan-20 00:39

We think there are tons of things one could do to make the interface easy to figure out without losing the power / flexibility.

Exactly. I agree with both you and Dimitri that the power and flexibility (I would add speed too) are primary. But as long as that is not significantly compromised ruthlessly drive to make the interface maximally easy and intuitive.

Dimitri


7,413 post(s)
#24-Jan-20 13:10

I think there is a much simpler and clearer way of implementing that "provisional" editing capability.

Make it an option in a context menu that when selected a window pops up with pane showing all the data for the row that will be provisionally edited in two columns; the name for each field in the record (row) on one side and the value for that field in the record on the other side. You can then make all the changes you want to the values and when ready you can commit those changes with a save record button or hit a cancel button. That would make it clear to even a noob what was going on and not break the standard selection paradigm in almost all table editing.

Well, you could do that, but then it would be slower, and you'd have a pane pop open that might be covering some of the rest of the table. It's a lot quicker to just pick and do it in context. I disagree that it breaks any standard selection paradigm. Click a row handle to put the focus on that row and then have at it using the usual Editing tools.

Manifold always seems more complicated as a "go-to" tool for people than ArcGIS or QGIS. I think the reason is because the interface takes some getting used to.

You have to be careful what lessons you take based on the populations you're observing. The primary rule is that people think the easiest system is the one they already know very well. Once they learn something well, they tend to forget the learning process they had. But, by dang, they understandably want to avoid any learning process for anything new. That's only human nature.

The reality is that whether or not Arc or Q have easy or efficient user interfaces, people who already know those packages very well will find them easier than Manifold. For that matter, they'll find them easier than whatever ESRI offers them that is new and better. Look at all the criticism of ArcGIS Pro by die-hard users of previous Arc things, even though once people learn Pro they almost always grudgingly agree it really is better.

Q has a particularly awful interface - so many butttons it's like bringing the Blackberry back in a smartphone world - but once you learn it well you can work it just fine. But nutty stuff like a click in a map to center it is just crazy. Just try to have multiple windows open and change context between them... you end up centering the map when all you wanted to do was to change focus.

That particular goofy feature appears to be a legacy remnant from when QGIS had no ability to work with multiple undocked windows. But now that Q users have it wired into their nervous systems, it's there to stay.

Over a billion people in this world use spreadsheets how many use a DBMS?

That's both a fair point and also a red herring. How many use a DBMS? Everybody who does GIS, whether or not they realize they're really doing DBMS things. So if the target market is GIS, what's a better user interface? One that serves what people do in GIS or one that serves what people want to do in spreadsheets but not in GIS?

You encounter the hard parts in such decisions when you do not assume away the hard parts. The hard part is how to package the increasingly sophisticated capabilities that GIS people now need. One aspect of those hard parts is the increasing desire to have more powerful and more precise control over working with data. That's a database world, not an Excel world.

Another side of serving GIS in contrast to serving spreadsheets is that what is now a routine, small size of data in GIS absolutely crushes spreadsheets: it brings them to a halt. Why? Because the technology of spreadsheets is not built to handle larger data. Some (not all, but some) of how user interfaces are wired up in spreadsheets is making an implicit choice about the corner cutting that can be done with smaller data, but which breaks down with bigger data.

You can see that collide in places like just how risky is it to open for editing a cell with an initial click. You feel that can be handled with an Undo. But there's no Undo in significant DBMS: not in Oracle, SQL Server, MySQL, or PostgreSQL, to name just a few. Why not? It's not because the guys who wrote Oracle are stupid or have some religious thing against Undo. It's because those Oracle guys are very, very smart guys who want to provide the extremely rich and powerful capabilities people want to use in the bigger data settings such systems can handle. Not having an Undo is one of the tradeoffs that, so far, everyone who has implemented such powerful systems thinks is worth what you get in exchange.

People want that power in GIS too. It's not like it's an option any more, as with every year there are more and more use cases that Arc or Q simply cannot handle, because they don't have the infrastructure to handle the scale. I get it that somebody whose muscle memory is already wired to Arc or Q doesn't want to learn a new way. But if that's the case they have to reconcile themselves to living within the constraints of Arc or Q. Those constraints can be profound and not just something that can be wished away.

An example: I've been following some friends who are comparing 9 performance to Arc in various analytic tasks, Spatial Analyst and similar. They try something, like generating contours on a raster, and scale up the size of the raster to see how long the two take. What's surprising is how often Arc just drops dead and crashes long before Manifold even breaks a sweat. I heard a case today where they had to limit the size of the raster to only 8000 x 8000 because larger than that Arc just crashed too often. In that case Arc took over a minute for what 9 did in a second and a half.

So what's the Arc guy going to do when he has a few hundred 12000 x 12000 rasters to build contours for? That might be worth learning a few new user interface details, especially if those UI details are the key to opening instant response and fast action. Time is money you know, and nobody likes to sit staring at an Arc screen for minutes wondering if it's crashed again or just annoyingly slow. Speed is everything when it comes to really pleasant user interfaces.

A good bit of what is puzzling you I think comes from the slight differences that are required to support the greater capability that people want. Learn those once, and learn them well, and you won't want to go back to less efficient interfaces. I respect the idea that if a billion people know how to use Excel, may as well leverage that familiarity to help kick start a beginner. But not at the cost of failing to do the job that has to be done. Before you assume something is unnecessary or rarely used, learn about it first, and then suggest something that what might work better that does not result in losing important capability. I'm all for that! :-)

Mike Pelletier

2,122 post(s)
#24-Jan-20 16:30

Interesting conversation and I'm sympathetic to pslinder1's view. Some thoughts:

- Mfd is different than Excel but generally try to copy Excel UI whenever possible for the reasons discussed (muscle memory and highly vetted UI)

- Mfd is a sophisticated database like the big boys but try to implement Undo whenever possible. Consider local tables, small tables, single user, be creative :-) Undo is just too awesome to not strive for.

- In a table, left click and double left click to edit is the current. A second single left click to edit is better unless there's a good reason not to. Also, after the first left click in a cell, why not allow typing to start changing the entry like in Excel.

- In drawings, Alt-click to see info and edit geometry at first seems odd but I think it is good. Ctrl-click to select is good as well. Once in Alt-click mode, I think it would be nice to be able to Ctrl-click another object without ending the At-click. Exiting Alt-click mode is a bit of a pain. Ctrl-backspace does not seem to work as the Context Help says it should. Pressing Escape would be easier and more natural. If there are pending changes to the object's geometry or table info, then ask if the user wants to save or not before exiting.

Mike Pelletier

2,122 post(s)
#24-Jan-20 17:41

Also, it would be really nice if the UI could work for tablets in the field. That situation is less friendly to using Alt and Ctrl.

I'm thinking a pretty straight forward UI approach is initial left click to focus on window, then left click to get object info, and a double click to edit vertices, and right click to select.

Is there a way to add on the desired safety measures to this approach?

pslinder1
228 post(s)
#26-Jan-20 00:28

Very much agree.

pslinder1
228 post(s)
#26-Jan-20 03:59

I agree with everything wholeheartedly up until the last point.

Why not have the Alt+click functionality transferred to a single let mouse click in drawings? What's the downside? What if anything do you want a left mouse button click to do in a drawing?

Mike Pelletier

2,122 post(s)
#26-Jan-20 04:40

If a single click can do the job without causing trouble, I'd prefer it to Alt-click. Perhaps left click to focus on window, then left click to get object info, and a double click to edit vertices, double right click to select, single right click to get a dropdown menu.

pslinder1
228 post(s)
#27-Jan-20 23:04

I like it.

pslinder1
228 post(s)
#27-Jan-20 23:23

Actually I really like that. Adam and Dimitri, what do you think of Mike's suggestions?

adamw


10,447 post(s)
#28-Jan-20 07:41

We will discuss it internally.

The central issue here is whether simple clicks with no key modifiers should do anything at all in a map window. There are pros and cons both ways. The main pro: one of the common actions no longer needs a modifier. The main con: trying to pan the map by dragging is going to lose the picked object whenever you fail to drag and end up just clicking - this happens frequently enough to be annoying. (Losing the picked object is pretty destructive because you might have edited its fields or geometry and might not have pressed Save Changes yet, so picking a different object is going to lose your changes. We can ignore picking a new object if an existing object contains unsaved changes, but that has its own drawbacks.)

Click once to set focus, click a second time to pick an object has the same issue, once the focus is in the window trying to pan risks losing the picked object. Only picking an object if you clicked that exact object the second time does not work well in a map, because a table window has a focus rect showing the focused (= last clicked for our purposes) cell and a map window has nothing like that.

Maybe we'll find a good solution. If we don't, we'll leave things as they are and keep thinking about it.

pslinder1
228 post(s)
#30-Jan-20 01:50

The main con: trying to pan the map by dragging is going to lose the picked object whenever you fail to drag and end up just clicking - this happens frequently enough to be annoying.

When you were talking about creating large and complex selections I thought you might have a compelling argument even though I had not really experienced the "selection fragility" problem myself. But certainly that is much less of a concern when you are talking about a single object. Can it happen? Sure but I really don't think it happens all that often. I find that a click and drag is a very different action than just a click. And even if it does happen occasionally we are just talking about losing the unsaved data for only a single record. I think that might be a good trade-off for a useful single button click.

BTW I tried to get Manifold 9 to mistakenly select a record in a drawing by alt+click+drag. After 10 minutes of trying I gave up because no matter how I tried I could not get it to screw up as long as any infinitesimal amount of dragging was done. It seems that the only way you could really screw up is if you clicked then failed to drag at all before releasing. Of course anything that can happen will happen, but I think it will be very rare that a click+drag will cause a problem.

Now I bet that a much more common failure scenario is that someone forgets to Save Changes before picking, intentionally, the next object. But that is another issue.

We can ignore picking a new object if an existing object contains unsaved changes, but that has its own drawbacks.

That or just giving an "Are you sure?" alert might be a good trade off. But again that is chiefly addressing the problem of forgetting to Save before moving on.

Dimitri


7,413 post(s)
#30-Jan-20 08:09

When you were talking about creating large and complex selections I thought you might have a compelling argument even though I had not really experienced the "selection fragility" problem myself.

It's not about selection, it's about picking objects and editing them. Edits to complex objects can themselves be complex. You don't want to lose that by accident, and an "are you sure?" gives away the speed and fluidity that is the objective.

BTW I tried to get Manifold 9 to mistakenly select a record in a drawing by alt+click+drag.

In case you really meant selection and not picking, it's about picking, not selection. Also, when you try to simulate it using alt-modified moves and paying special attention that's not where the main effect of concern comes into play. It's the difference between a simple click and a click and drag when you're not paying so much attention but have your mind on other things. Even when you have your mind on other things there can be no worries involved, so there are no small hesitations or distractions to keep in the back of your mind.

Click and drag to pan is a very big deal when editing because that allows you to zoom in to where you can see and handle details well while quickly and easily panning to a slightly different location to see more. When you're thinking about editing, you don't want to have to think "OK... better do this click and drag cleanly so it's not just a click...". It's exactly when you're thinking about other things that it's easy to do a click when you intended a click and drag, and especially so if the mouse is older or not the highest quality.

If you never do just a click when you intended a click and drag, that's great, but that's not the experience of most people.

Don't get me wrong: I like the idea of a single click being useful beyond setting focus. It's worth pursuing all ramifications surrounding that idea, to see if and how it might be done without losing other things that are very valuable, like being able to pan in the middle of editing to better see what you're doing and what's surrounding what you're doing.

pslinder1
228 post(s)
#30-Jan-20 15:47

It's not about selection, it's about picking objects and editing them.

I was referring to an earlier set of posts on this thread where we were discussing the possibility of using a left mouse button click for selections. Adam made the point that if you have done a complex selection selection it would suck to lose it if you misfired on the click+drag and had to start again. I referred to that as the "Selection Fragility" problem.

and an "are you sure?" gives away the speed and fluidity that is the objective.

I am not sure that has to be true. If in the record panel you added a "Ignore Changes" button which allowed the user to intentionally not save his changes and move on, then you could be pretty confident that if a "pan" turned into an accidental "unpick" an "are you sure?" dialog would not be an unwelcome compromise on speed and fluidity. If either Save Changes or Ignore Changes button were pressed then no "are you sure?".

adamw


10,447 post(s)
#30-Jan-20 08:16

And even if it does happen occasionally we are just talking about losing the unsaved data for only a single record.

This will risk happening on every interactive edit of geometry. To edit geometry, you Alt-click it, then drag / click the coordinates and use various editing commands, then click Save Changes. If, during the editing, you need to pan the screen to better see the data / etc, you risk accidentally clicking instead of clicking and dragging and losing it all.

I am not saying this is an insurmountable problem. We can disable switching to a different object when there are unsaved changes, etc. I am just saying this is something that has to be thought about and perhaps addressed in some way. There are perhaps other things like that.

Regarding accidentally clicking when you meant to drag - we assure you that this happens often enough and we don't think much about it exactly because of efforts to make such errors non-destructive.

(By way of an anecdote, Windows once switched the default for Explorer to open files on single click instead of on double click, they had to switch back in the very next version partly because people were clicking instead of dragging a lot of the time. We have our internal stories as well. Dragging is scary as it is, a number of people plain hate to drag things and want alternative means to do what dragging does, because they cannot control the cursor well enough and find it irritating. No need to make the failure to drag even more punishing.)

We'll think about it. There are many options here. It is tempting to use a simple click to immediately get to an object, I agree, and it would play well with right-clicking the same object to show a context menu with commands that do something to it.

pslinder1
228 post(s)
#30-Jan-20 15:30

Ok. I have made my case about as exhaustively as I can. I think that the single left click is in its own small way kind of a big deal. Thank you and Dimitri for staying engaged for so long. If you do decide to make a change you will have at least one very strong supporter!

Mike Pelletier

2,122 post(s)
#30-Jan-20 17:30

Another big pro I think of not needing modifiers is use with tablets. For more tablet use, what if the user could add a toolbar along the bottom that had many or all the buttons found in the Context help.

Speaking of the left click and drag to pan, I think it slows down drawing objects. Not a lot but it is a compromise. Dimitri told be long ago that I would get used to it and I have :-) Nevertheless, perhaps the timing could be changed when editing geometry such that the drag kicks in after a longer hold with the left click.

I tend to agree with pslinder1 that the timing now is such that I don't seem to confuse click vs click + drag while holding down the Alt key. Nevertheless, if there are pending changes to the picked item perhaps the timing could change a tiny bit.

It would also be nice if the final solution had a way to allow users manually change between having to Save Changes vs assume save for each drawing object.

Double left click to pick (view records and edit geometry) and double right click to select is another potential compromise.

Thanks for re-contemplating this topic.

tjhb
10,094 post(s)
#25-Jan-20 03:38

Brilliant post Dimitri. Right on.

BTW it confuses me that anyone would hold out Excel as a good UI example for any purpose, let alone for GIS or DBMS purposes (fundamentally different, as you say).

Microsoft's Excel interface used to be pretty good for a spreadsheet, circa 15 years ago. LibreOffice has inherited the best of it, though that makes LibreOffice seem somewhat old-fashioned. (Still good, functionally the best.)

Excel was deeply vandalized by the "360" interface and before that the "ribbon" toolbar. We can tell that someone really high-up must have invented that toolbar, because Microsoft can still not admit that it was a mistake. Instead they copy it to Windows Explorer. All that means is that Explorer has no useful toolbar either.

Manifold does not make these design mistakes. Not least because it focusses on the user's needs and purposes, bearing firmly in mind the user's data.

Yes there is task-specific learning. That is ideal.

pslinder1
228 post(s)
#26-Jan-20 04:06

What Evs. Spreadsheets are the defacto standard on how everyone works with data in tables. It is simply indisputable. If Manifold wants to go mass market (or at least much massier market) hewing to that standard as much as possible is just eminently sensible.

tjhb
10,094 post(s)
#26-Jan-20 05:43

Spreadsheets are the defacto standard on how everyone works with data in tables.

I think that is where you show a fundamental misunderstanding.

Spreadsheets are not tables. Informally perhaps, but not at all tables in the sense that Manifold and other DBMS system use.

Both concepts present data in tabular form--they have rows and columns. But the similarity more or less ends there.

There is no conceptual similarity between a Manifold table and a spreadsheet. None.

The (obvious) resemblance is only superficial.

Is there any reason why they should have similar user interfaces? In my opinion, no.

Mike Pelletier

2,122 post(s)
#26-Jan-20 04:06

Agree with you Tim about LibreOffice UI being better than Excel. My statement was far too sweeping. I was thinking of the cell editing.

pslinder1
228 post(s)
#26-Jan-20 03:51

Well, you could do that, but then it would be slower, and you'd have a pane pop open that might be covering some of the rest of the table. It's a lot quicker to just pick and do it in context. I disagree that it breaks any standard selection paradigm. Click a row handle to put the focus on that row and then have at it using the usual Editing tools.

I agree that workflow would be slightly slower; you might have to move the pane out of the way occasionally. But I think that is well worth it for having clarity on what is going on. From what Adam has said above it UI method being used to enter "provisionary editing" mode is not some standard industry practice, actually even having that capability is somewhat controversial. The current method you are using is fast enough but you would have no idea of what was going on without reading the manual on the topic. A little bit of slow down for providing obvious clues for what is happening seems well worth the trade off.

You have to be careful what lessons you take based on the populations you're observing. The primary rule is that people think the easiest system is the one they already know very well. Once they learn something well, they tend to forget the learning process they had.

Very true. But the specific example I was thinking of was very different. I had started working at a startup about 4 years ago when a GIS project came up that I did not have time to deal with so I gave it to 2 smart but junior electrical engineers (one with a BS the other with a PhD both never having worked with a real GIS package before) and told them to buy and expense Manifold 8 to get it done. A week later when the came back with the results I found out that they had given up on Manifold after 2 days (they had bought Art's videos as well) and gotten part of it done with an ARC license that a colleague had on her computer and the rest with QGIS which they had downloaded. About a month afterwards when I had a little free time I spent a couple of hours one morning investigating what they had done and confirmed that they could have done the project (as I had suspected initially) quite easily with Manifold 8. Its just an anecdote but it is a pattern I have seen in the last 15 years in the telecoms industry.

I evangelize a lot about the product but I have never heard of anyone using it that I have not introduced it to. And of the 100 to 200 or so people with whom I have talked about it only 5 or so have actually stuck with it. And all 5 of those had some interest in GIS outside of a professional capacity.

Q has a particularly awful interface - so many butttons it's like bringing the Blackberry back

Totally agree. I find it annoying and fiddly.

But nutty stuff like a click in a map to center it is just crazy.

It may not be very useful but why is centering a map on a click crazy or nutty? it may be slightly annoying that the map gets centered when you change focus but it is hardly a major problem. I get that they should solve that problem and could do so easily. Manifold's approach I think is stranger. Using one click to change focus is fine but why should every subsequent click in the window do nothing? That is crazier and nuttier to me.

But there's no Undo in significant DBMS: not in Oracle, SQL Server, MySQL, or PostgreSQL, to name just a few.

This herring is glowing scarlet. I brought up the "undo" as a great way to solve the problem you described of accidentally wiping out data in a cell that you edited unwittingly when you selected it. I would guess a lot of these programs might not allow you to edit a single cell. I like the fact that Manifold does and I do not think that an accidental edit is such a big deal but if you want to protect against that then an Undo button makes a lot of sense.

What's surprising is how often Arc just drops dead and crashes long before Manifold even breaks a sweat.

No doubt. But I don't think any of that is because of how ARC or Q implemented a left mouse button click.

I respect the idea that if a billion people know how to use Excel, may as well leverage that familiarity to help kick start a beginner. But not at the cost of failing to do the job that has to be done. Before you assume something is unnecessary or rarely used, learn about it first, and then suggest something that what might work better that does not result in losing important capability. I'm all for that! :-)

What specific feature have I mentioned implementing that would result in losing important capabilities and at the cost of failing to do some critical job?

Dimitri


7,413 post(s)
#27-Jan-20 11:29

What specific feature have I mentioned implementing that would result in losing important capabilities and at the cost of failing to do some critical job?

Before assuming something is unnecessary is the issue. A good example is provisionary editing before a commit. That's a huge capability which is not something to discount lightly. It gives you Undo, for example, and it opens the door to seamless utilization of hundreds of different data sources as an equal citizen.

actually even having that capability is somewhat controversial.

No, there's nothing controversial about the need, an absolute necessity, really, to have provisionary editing in those many circumstances that require it when taking advantage of an organization's data stores.

If you don't have that capability, you can't participate as an equal citizen in the editing/updating of data in data stores using constraints, a very common and vital feature in the data stores of many organizations.

but if you want to protect against that then an Undo button makes a lot of sense.

Agree Undo makes a lot of sense. That's one reason you can have Undo when editing Manifold tables.

Undo is part of what you get with the provisionary editing apparatus, which you get by clicking a row handle. Among other capabilities, that marks the row where you want editing to have Undo capabilities: you can edit however you like, and then Undo.

Click the row handle and double-click into whatever cell in that row you want to edit. When you're done editing a cell, press Enter and you see the cell you edited in preview blue color. Right click and choose Undo changes if you like. You can go back and forth between different fields in the row, editing them as you like, changing your mind and doing Undos as you like.

You even get multistep Undo that way. Edit the cell for field A to change "harry" to "bob" and then edit the cell field B to change "jones" to "smith." Now you can right click on field A and choose Undo Changes and it snaps back to "bob" while field B stays at "smith." If you want, right click on field B and choose Undo Changes and field B changes back to "jones." You can do that with a hundred changes in that row and Undo whichever ones you want. Can't do that in Arc or Q.

It's pretty magical that works everywhere, even when you're editing tables that live inside data sources like Oracle that don't themselves support Undo.

But I don't think any of that is because of how ARC or Q implemented a left mouse button click.

It's related. Both Arc and Q have gone for relatively sloppy implementations of superficials while never making the effort to build good data infrastructure at their cores. That is one reason they are so wretchedly slow and so often crash, and it is also a reason why they don't have Undo like Manifold does, as in the example I gave above. So sure, you get a left mouse button click to open and edit a cell, but you don't get provisionary edits and an Undo like Manifold.

Could Manifold also do an instant open of a cell on a click, once the cell has been picked (as Mike Pelletier suggested)? Sure, and it might just do that. Manifold's always up for doing what people want, that overall will increase functionality and make things easier. But you have to be careful with making such changes, because Manifold is used in bigger data settings where Arc or Q just drop dead, so you can't assume the simple approach Arc or Q take is always a good idea. But if simple approaches can be added in a way that does not interfere with more sophisticated capabilities, well, why not? That's why we have conversations in the forum. :-)

Clicking a row to mark it for provisionary editing is an example of a sophisticated feature enabled with just one click. It's just one click, but it safely opens up an entire universe of compatibility with both big, sophisticated data sources as well as Undo for relatively simple things like linking in a shapefile. (No Undo in Q for the attribute table of your locally linked shapefile. )

Big DBMS and linked shapefiles are related because in Manifold you can write queries and otherwise simultaneously work with data that is entirely local, is coming in from DBMSs like Oracle, and is coming in from a linked shapefile. You can combine all those in a query and create drawings from the virtual results tables of such queries, and - astonishingly - even have full editing capability back into the original tables, be they in Oracle, in a shapefile, or in the local project. That little click on the row to declare you want Undo and the rest of the provisionary editing apparatus in that row is all it takes to make sure all that works together smoothly and seamlessly while still providing Undo. It's a good tradeoff for that one click to get all the rest of it.

Mike Pelletier

2,122 post(s)
#27-Jan-20 22:43

The provisional editing feature is very welcome. Better then undo in one way, since you can skip over good changes to go back to a mistake before committing and works with any table. It does seem to require a second extra click in the row handle to commit the changes.

In general though it is less useful then a typical undo (revert any change up to a certain number) since it is limited to a single record. Doing a Save As is sometimes useful but the incremental undo should be added when possible given all the constraints.

In particular, I'd like to see it with modifying geometry (not just the backspace option for vertices) and with applying Styles to vector layers.

Perhaps an option gets added that allows user to turn on undo with the associated trade-offs it creates. When initiated all the changes get stored to temporary tables until the user manually commits the changes.

Easier said then done I'm sure. A low priority item at this point too :-)

pslinder1
228 post(s)
#27-Jan-20 23:28

Hear Hear. But I would bump the priority a bit.

Dimitri


7,413 post(s)
#25-Jan-20 04:50

If the way Manifold implements provisionary (sp? even the spell checker on this forum does not think it is a word)

Oops! Forgot to answer the question. Sorry about that.

Yes, I mean "provisionary". That an online spell checker can't find it is just evidence of the superficiality of "free" things on Internet, not of the limitations of English.

Using the OED (Oxford English Dictionary), we see provisionary is defined as:

Arranged or existing for the present; provisional.

‘a provisionary party member’

pslinder1
228 post(s)
#25-Jan-20 19:46

True, its officially a word. Its not wrong, but it is an awkward usage in English. In the Manifold documentation if you go to "provisionary" in the Index you will see "provisionally" right below it. Its not "provisionarilly" for a reason. If you type in "provisionary editing" in Google you get "did you mean provisional editing?". The top search results that do come up are all Manifold. No one really uses the term.

tjhb
10,094 post(s)
#25-Jan-20 21:51

There is a subtle difference in usage.

Provisionary means something done or arranged for a temporary or incomplete purpose, or in case of a situation that might arise. The emphasis is on the act (such as editing).

Provisional means something which is done only temporarily or incompletely, awaiting a better or more lasting replacement. The emphasis is on the quality of the thing (such as an edit).

pslinder1
228 post(s)
#26-Jan-20 03:53

Its so subtle its a distinction without a difference. They are listed as synonyms of each other and can be used interchangeably but provisional is way more commonly used.

tjhb
10,094 post(s)
#26-Jan-20 06:01

There is a difference. Try this.

"The assessment is provisionary at this stage. We don't even know if we can get the land yet. In the tables, figures from last year's report are in black, while provisional figures for the current year are in red."

Both good. Near synonyms, usually interchangeable in practice.

Another, different example: provisionary spending, i.e. to allow for a possibility. That could also be provisional, in that the amount might be changed later, but the idea is different.

pslinder1
228 post(s)
#26-Jan-20 17:10

How about if the amount of provisional spending is 0? Wouldn't that also make it provisionary?

pslinder1
228 post(s)
#26-Jan-20 17:09

In point of fact there is not adverbial form of provisionary so it cannot actually modify an act. But I find no firm support for anywhere for your thesis. The two words are synonyms when provisional is used as an adjective. Any perceived difference is probably accidental or just mistaken.

Dimitri


7,413 post(s)
#26-Jan-20 08:02

Sigh... don't confuse Internet with English, and don't confuse posts in forums with Faulkner. Otherwise people might criticize you for failing to write it's instead of its when you mean the contraction "it is."

In the Manifold documentation if you go to "provisionary" in the Index you will see "provisionally" right below it. Its not "provisionarilly" for a reason.

Yes. The reason is that those are two different sequences of characters. The Index automatically finds different words, whether or not they are real words (computer terms famously not being real words), misspellings and the like.

If you type in "provisionary editing" in Google you get "did you mean provisional editing?". The top search results that do come up are all Manifold. No one really uses the term.

Enter "provisionary" and you get 103,000 hits. It's a fine word used in many settings where the nuances it implies are spot on. Like in this page.

It's true that the race to mediocrity and least common denominators that has been put on afterburner by Internet has reduced the vocabulary of many Internet participants. To the extent people use what still passes for English in technical discussions, they use a very simple lexicon, often recycling the same words even when the meaning is not quite right.

When you search for specific terms on Google, all you get is a view of what's most popular on Internet. You don't get a view of what is correct, or even how the dwindling minority of well-educated people use language. You simply get a snapshot of the long process by which so-called "civilization" is entering a new Dark Age, in which fewer and fewer people are genuinely literate, able to compose - on their own - more than a few sentences without gross errors of spelling and grammar.

Enter "provisionary <insert a technical term of art here>" and no, you're not going to get millions of hits. You'll get only those hits where some specific technical text uses that term next to the given technical term of art. If you intended to find whether provisionary is used as a word, you've misused Google's double quote interface to search for specific uses, failing to take advantage of Google AI filtering to find broader results.

Enter "provisionary edits" and you get many hits, 63000. Enter "provisionary changes" and you get fewer, but they tend to be spot on. For example, this link with provisionary changes to a database and another link with provisionary changes to a score of Turandot.

I think that having a good vocabulary is one of the joys of using English. I believe one of the ways to push back against the rising tide of idiocy that is powered by Internet is to avoid writing with the discourteous assumption that my readers require spoon feeding with simple words.

I also believe that "variety is the spice of life." Use only common words all the time and you miss out on the spectacular breadth and depth of English.

pslinder1
228 post(s)
#26-Jan-20 17:05

Now there is a good old-fashioned Dimitri rant. I have missed them sorely. Truly, no sarcasm.

I think that having a good vocabulary is one of the joys of using English.

I agree that is how I know these two words are exact synonyms (when provisonal is used as an adjective; dont want the Provas to be getting mad at me).

I believe one of the ways to push back against the rising tide of idiocy that is powered by Internet is to avoid writing with the discourteous assumption that my readers require spoon feeding with simple words.

That is a recipe for being misunderstood. It may be fun but its horrible for effective communications. If you want to talk to a professor, a professional, a confessor, or even a hair dresser pitch a middle school vocabulary.

The attached file is from Collins English Dictionary printed in 1990 I think. But the definition I am sure has not changed. I hope this satisfies you and tjhb that these two words are exact synonyms, one is just not commonly used. Any distinction between the two is purely aesthetic. When it comes to provisionary, its just not it's own thing!

I think this is a good metaphor for some frustrating parts of Manifold's interface. Awkward choices that are too stubbornly defended for no particularly compelling reasons.

Attachments:
IMG_20200126_113608.jpg

Dimitri


7,413 post(s)
#27-Jan-20 08:32

Awkward choices that are too stubbornly defended for no particularly compelling reasons.

An example?

pslinder1
228 post(s)
#27-Jan-20 23:27

Left mouse button click doing nothing except focus change.

I know you disagree because you are used to it but it is as disconcerting as pulling a trigger and not getting "Bang".

Dimitri


7,413 post(s)
#22-Jan-20 14:58

But the larger point is still why would anyone new to Manifold 9 even think this was required.

Somewhat tongue in cheek: Because they read the Tables topic and saw that as the first, prominent headline at the beginning?

Seriously, though, a beginner just starting with a new system full of zillions of features has no way to know what is required and what isn't if he doesn't take the time to learn it by reading documentation, watching videos and so on. That's true of PhotoShop, Oracle, Visual Studio and pretty darned much any highly capable package out there.

Guys who try to learn PhotoShop by guessing encounter lots of frustration that people who learn PhotoShop by reading documentation and doing tutorials avoid, and it takes them years sometimes to find key features that people who learned in a more structured manner knew about right from the beginning.

Most tables in GIS work in 9 automatically have indexes because that's how the dataports import them. No need to add an index when you import a shapefile, for example.

For those tables that get created without an index, the table is read-only and can't be edited. If somebody has skipped initial reading and wonders why,it's not unreasonable to think about checking out the Tables topic to clear up any mystery, or to ask in this forum.

As to why indices are not automatically added to imports from certain formats, like CSV files, that's a matter of taste and choice where different people like different defaults. Guys who come from a strong IT background usually hate it when fields or indices are added, because what they're shown as the imported table is not what was actually in the table. They'd add those themselves if they want, thank you.

In contrast, guys who just want to do stuff with tables right away and aren't so concerned about having a direct tie to the original form (I'm one of them) would prefer an identity field and index would be automatically added. I've had that in my wishlist for a long time for CSV files, for example. :-)

pslinder1
228 post(s)
#22-Jan-20 19:26

I suppose a lot depends on who you think the software is ultimately for. If it is chiefly for IT and database experts who know about why tables are either editable or not then it makes perfect sense for the default to not create an index if that is their preference. But you are limiting yourself to a tiny market if that is the case. I think the number of people who could use your software but are ignorant of the arcana of table editability far outweigh the former cohort. I would risk the mild annoyance of the smaller group.

However, as a reasonable compromise you could make the addition of the index (and choosing the lat/long columns for the georegistering) default behavior when creating a new drawing from a table that does not have an index. Actually you could make that a popup option whenever someone tries to edit an uneditable table.

On a general point about reading I would say this. The more reading needs to be done the less intuitive the interface probably is. Some confusion necessitating reading may be inevitable but as William of Occam once said:

Entia non sunt multiplicanda sine necessitatem.

Rakau110 post(s)
#11-Jan-20 00:36

I think it was Abraham Lincoln who said "if I had 8 hours to cut a tree, I would spend 6 sharpening my axe"

Personally I found sitting down for a few minutes each day reading a small part of manual and translating into a manual that I would use to teach someone with no knowledge increased my skill and knowledge tremendously.

I now refer back to my own manual when I strike an issue I am not comfortable with.

vincent

1,972 post(s)
#05-Feb-20 16:40

"what else is missing", Tim ?

A lot of Contextual menus is the first thing. I had to ask here how to rename a component ! And still, I come back here to get this answer this morning, because I cannot remember how, I cannot find it in the manual, I cannot find it in the Help Context Menu. This is a bad design. It have to be in the right-click-menu of a component in the Project pane. It was there in M8. Why is it gone ??? What is the economy of removing it ??? Someone wanted to win the award of the more fengshui GIS design ???

One thing is lacking at Manifold office : a GUI designer. Too much ctrl+alt+click and keyboard shortcuts and not-intuitive fonction access. It was bad in M8, it really sucks in M9.

I never use M8 to manually edit something or to make a map. It's a PITA. I wiil never do in M9 either.

I can read the whole manual to understand M9 with pleasure. I look forward to it. But the GUI is a complete turn-off.

Dimitri


7,413 post(s)
#06-Feb-20 02:41

It have to be in the right-click-menu of a component in the Project pane.

? Right-click on a component in the project pane, and you get a context menu:

See also the Show / Rename / Delete Related topic in the basics book. You can also rename an individual component with a slow double-click.

The user interface is community driven just like everything else. Whatever people want changed gets changed to how they want it. The process for that is very straightforward and laid out in the Suggestions page, which has great advice on how to influence the product with maximum effectiveness.

One side effect of that, I believe, is that 9 tends to be shaped by people who use it regularly. That's not a bad thing because a tool should be something that works well for people who depend upon it for their daily work. Those people tend to emphasize more that something is highly efficient and easy to use in daily workflow. Such characteristics and UI can be different than something easy to learn for beginners who are just starting. PhotoShop is like that, too.

Anyway, the way to guide the product is to make suggestions. Talk about ideas in the forum, but never fail to send in your vote!

Attachments:
mnu_project_context01_02.png

tjhb
10,094 post(s)
#06-Feb-20 05:35

The user interface is community driven...

One side effect of that, I believe, is that 9 tends to be shaped by people who use it regularly. That's not a bad thing because a tool should be something that works well for people who depend upon it for their daily work.

Right on. And...

Such characteristics and UI can be different than something easy to learn for beginners who are just starting.

I think there is a useful analogy available here.

It is the distinction--an optional one, built in--between performing analysis by packaged transforms, and performing analysis by SQL.

Both are in reality done by SQL, as we can immediately find out (via Edit Query--when we feel ready).

That was a great design decision, early on. Seems simple now... but that extra layer of redirection really makes a difference. It equally makes a big difference to usability for long-term, experienced users. It's truly a lever.

I think that is the way to continue.

So: a product with two skins, one that should be immediately approachable, by anyone prepared to put in a minimal level of familiarization work; and another that might well be approachable only to seasoned users (though in gradual steps by everyone in between).

Maybe the transform "skin" should be allowed to approach a wizard-like interface more than it has been, with extra prompts and F1 context help, even at the cost of some speed; while the underlying SQL interface is allowed to remain pure and pristine, and perhaps inherently faster (and always accessible by direct translation, with no approximations or fudges).

I don't know.

One more comment (hesitantly made but surely right!). If some users find it strange that a left mouse click does nothing massively obvious, and/or are disconcerted that the only actual action in some cases is to activate the clicked window, then I think a good answer may just be like this:

Make the actual action more obvious, less subtle, give more feedback. Flash the activated window in boysenberry, change its title bar to bold, permanently adjust tab colours, add a window name to the status bar--anything really, as long as it hits your face as soon as the button is pressed.

That might be all that is missing--in some cases anyway. Not madly different from suggested UI adjustments for the 50,000-record table limit, for similar reasons.

tjhb
10,094 post(s)
#06-Feb-20 05:52

extra layer of reindirection

Oops.

pslinder1
228 post(s)
#09-Feb-20 23:38

Maybe the transform "skin" should be allowed to approach a wizard-like interface more than it has been, with extra prompts and F1 context help, even at the cost of some speed; while the underlying SQL interface is allowed to remain pure and pristine, and perhaps inherently faster (and always accessible by direct translation, with no approximations or fudges).

I don't know.

I do know. It is an awesome idea. I agree with you completely. Make the user interface easier but leave the underlying SQL pristine and quickly accessible for the pros.

If some users find it strange that a left mouse click does nothing massively obvious, and/or are disconcerted that the only actual action in some cases is to activate the clicked window, then I think a good answer may just be like this:

Make the actual action more obvious, less subtle, give more feedback. Flash the activated window in boysenberry, change its title bar to bold, permanently adjust tab colours, add a window name to the status bar--anything really, as long as it hits your face as soon as the button is pressed.

I do not think that is enough. Most Windows programs that open multiple windows change window focus with a single click. So that is not what is confusing. The problem is that subsequent single clicks in the same window do nothing.

I still advocate very strongly for some real action on subsequent single clicks in a map window. Have it do something useful. Please. No Dead Clicks! I think i will get campaign buttons made up with that on them:)

vincent

1,972 post(s)
#06-Feb-20 13:30

Right-click on a component in the project pane, and you get a context menu

It wasn't there in 170.0, which I am using.

Still, it is complicated. It opens another box and if you don't know that it is possible to "slow-double-click", there is in fact no progress.

Who have the time to "slow-double-click" ? A GIS analyst in my mind knows how to work fast. I use a computer for more than 30 years, and "slow-double-click" wasn't a concept until I read it here a few months ago. Again, this is a poor design. Why not a simple click, a real double-click or a right-click. It have to be intuitive.

See also the Show / Rename / Delete Relatedtopic in the basics book

It's like finding a needle in a haystack. Who expect to read the manual to know how to rename a component, after years of computing ?

Simple things turn around customers. This is one.

rk
621 post(s)
#06-Feb-20 13:38

You don't know that it is possible to "slow-double-click"?

How do you rename files?

There is also F2

Dimitri


7,413 post(s)
#07-Feb-20 09:15

It's like finding a needle in a haystack

? Do it the same way you would in Windows Explorer... either slow double click to open it for renaming, or right click on it and choose the only command in the context menu that begins with "Rename".

I understand you were using an old version and not the latest version. When something is evolving rapidly in response to community requests, it's easy to fall behind. But, I respectfully suggest that is a good problem to have - keeping up with something that evolves rapidly in response to community demand.

The dialog is easy to use and greatly simplifies something that otherwise could be very complex. Manifold gives you very many capabilities that less capable systems do not have, so if you want to take advantage of that, it pays to read a bit.

In Manifold you can have dozens of drawings that show the same table's data, and you could have dozens of different labels that are based on each of those drawings. Hundreds of maps might use a particular drawing.

Suppose, for example, that you work in government where hundreds of different maps are created, and most of them use a "Transportation" drawing as a layer. One day the boss says "the name is too long. Rename it to Roads." Super. With Manifold, you can use the Rename dialog and it will not only rename that drawing, it will also update all the labels and maps and layouts that refer to it.

Suppose you have a table with a very long name, from which were created many different drawings, all using that same table. If you want to change the name of the table to something short, like "2019" data, you might want to change the names of some of the drawings but not others. The dialog will do that for you as well.

It's not perfect, of course, but it's a great first step that goes a long way to simplifying renaming a component in a rich system where there can be constellations of dozens or hundreds of components that are related. That you can take advantage of such constellations is a good thing, a very powerful thing, and the dialog helps make it easier to manage them.

pslinder1
228 post(s)
#09-Feb-20 23:46

One side effect of that, I believe, is that 9 tends to be shaped by people who use it regularly. That's not a bad thing because a tool should be something that works well for people who depend upon it for their daily work. Those people tend to emphasize more that something is highly efficient and easy to use in daily workflow.

I really like the community and the forum. I have gotten a lot of useful information and a lot of critical help from here over the years. But I am not sure you want this group of self selected experts guiding the development of what you hope (I think) will be a much more mass market product.

The top 1% of power users buy Manifold for the same price as the other 99%. And if you kept an eye out for the sporadic GIS needs of people that have never used Manifold you might grow your user base by a couple of orders of magnitude (actually I do not really know, but I am assuming Manifold 9 is not all that ubiquitous).

Dimitri


7,413 post(s)
#10-Feb-20 05:10

But I am not sure you want this group of self selected experts guiding the development of what you hope (I think) will be a much more mass market product.

Well, the key requirement is that the tool has to work well for the people who bought it and who depend upon it as a tool in their toolbox. If you do that well, everything else takes care of itself. PhotoShop is the classic example of how that works.

It's a bit like creating tools for people who work with wood. If you create, say, a plane, it has to feel good in the hand, be easy to adjust and hold the adjustment, cut true, maintain a keen edge, be easy to clean and so on.

A natural effect of tool users helping to steer the tool process is that those people who actually use the tools are usually the first to note inconveniences and missing needs, and they are often the first to suggest refinements that help make a better tool for everyone, beginners included. Experts are often the first to see ways to make workflow simpler and easier.

The workflow in the video (the root subject of this thread) is a good example of that. What's easy to do in Manifold in five minutes in just a few short steps requires much more intricate workflow in ESRI's Spatial Analyst. To create watershed lines, for example, the ESRI tools require four operations creating intermediate rasters, which Manifold does directly in one click, usually over fifty times faster.

And if you kept an eye out for the sporadic GIS needs of people that have never used Manifold you might grow your user base by a couple of orders of magnitude

100% agree. It's a great objective to design in a way that people who have never used Manifold can get started more easily - so long as that doesn't make workflow less effective or more annoying for people who depend on the tool as a working instrument. There's a lot of experience that's emerged over the last 40 years on that, which is worth leveraging.

Looking at a very wide cross section of software packages that have emerged over the last 40 years since the original personal computer revolution, there are two iron rules that seem to hold true across all of them. The first of those rules is: "Software that is trivially easy to learn can do only trivial things." For all the justifiable interest by everyone in "user-friendliness" and "intuitiveness" that rule has held true with rock-solid truth through all four decades of the personal computing era.

You have to remember that "user-friendliness" and "intuitiveness" doesn't exist in a vacuum. The terms are often used to mean "what I know how to do, after many years of trial and error / learning / struggle / slow osmosis / etc"

It's also true that small amounts of learning and education can dramatically expand how "user-friendly" or "intuitive" something is perceived to be. The classic case is when a colleague knows how to work something, and he or she sits down with a beginner to show a few opening moves. What would be impenetrable to the beginner if the beginner was unwilling to read anything on their own can come off as much easier when they sit down with a friend for a quick introduction.

These days, people often use videos for that, but it's amazing how many people won't even watch a ten minute video to get started, but they'll happily attend a training session or workshop that does the same thing.

But however a beginner learns, it's clear that you get the biggest bang for the buck, so to speak, in terms of rapid gains in functionality for beginners *as well as* dramatically more efficient, capable, and convenient workflow for regular users, if the interface is designed expecting that *some learning* will be required. That's the second iron rule that has emerged from 40 years of experience across virtually all software categories and across many generations of hardware: A little bit of learning makes for a whole lot of intuitiveness and user-friendliness.

If you respect the user enough to expect at least some willingness to study a bit in the beginning, you can design an interface that will make that user far more productive, far faster, with far greater breadth and depth of capabilities, than if you limit yourself to a design which can be puzzled out by somebody who is unwilling to undertaken any learning, but instead pokes at buttons in a trial-and-error way.

I guess the bottom line also is how easy something is to learn compared to alternatives. Arc-anything is infamously difficult to learn for people who are new to GIS. Q is also extremely difficult to learn for beginners totally new to GIS. Release 8 has a reputation for being slightly easier to learn for people who are new to GIS, but it still requires learning. The key question is: if someone totally new to GIS invests the same amount of effort into learning in the beginning with 9, is it harder or easier to learn than, say, ArcGIS Pro?

The answer is a mass of details where in some details the one will be easier and in other details the other will be easier. Our task as users and as a community is to look at those details that could be easier and to suggest *specific* tweaks to make them easier. That's what many people are doing now, with very many suggestions being sent in, and many quality of life improvements, both in small things and big things in every build.

170.4, for example, has some really nifty improvements for ease of use. My favorite is quick creation of maps with a base layer... use that all the time! That's a classic example of something that makes it easier both for beginners and also for regular users, so the more of that, the better. :-)

tjhb
10,094 post(s)
#10-Feb-20 07:41

What a great post. Worth several rereads.

I loved the plane analogy. I remember when I first tried to use Dad's plane. The first thing I did was to lower the blade so that it would shave a useful amount of wood per stroke. Obviously it shuddered and even if I pushed hard, it just stuck. What a stupid tool.

dchall8
1,008 post(s)
#10-Feb-20 21:36

I guess the bottom line also is how easy something is to learn compared to alternatives. Arc-anything is infamously difficult to learn for people who are new to GIS. Q is also extremely difficult to learn for beginners totally new to GIS. Release 8 has a reputation for being slightly easier to learn for people who are new to GIS, but it still requires learning.

I learned to make M4 work for me within a few days after watching Art Lembo's multi hour product demo. He sold it as training, but it was much more of a practical, quick-start demo. Within a week I printed a multi layer PDF, 56x38 inches, showing a 5-county map of my client's gas leases with land abstracts, competitor's leases, 1-mile lease buffer zones, labels, shale target zones (from a raster), legend, and roads. My previous experience had been futzing around with a non-GIS map making program (can't remember the name). Of course there was no complicated analysis involved, but as for getting started, it was exceptional.

Attachments:
5-county map.jpg

pslinder1
228 post(s)
#11-Feb-20 15:40

The key question is: if someone totally new to GIS invests the same amount of effort into learning in the beginning with 9, is it harder or easier to learn than, say, ArcGIS Pro?

That is the key question if you are trying to be the new standard in GIS software. But it is not the key question if you are trying to make a mass market realize how useful GIS/big data analysis could be in the regular professional lives. These occasional non-expert users are potential customers and they probably outnumber GIS experts by 4,5,6? orders of magnitude. But they share one thing in common with the experts, they will pay the same $500 for the software.

I think the clash between the expert and the novice is often a false one. Powerful software for the expert does not need to be hard to get started with for the novice. Manifold 9 already does a good job of that in certain respects. The expert has a command window that is pretty easy to find and gives them all the power they need. But everyone, including the novice, is more immediately presented with the easier to work with Templates and Expressions tabs in the Transform pane (actually Manifold should make the transform pane a little easier still to find).

It's also true that small amounts of learning and education can dramatically expand how "user-friendly" or "intuitive" something is perceived to be. The classic case is when a colleague knows how to work something, and he or she sits down with a beginner to show a few opening moves. What would be impenetrable to the beginner if the beginner was unwilling to read anything on their own can come off as much easier when they sit down with a friend for a quick introduction.

In my experience most people start to learn new software not by going to a video or reading a manual or going to an online forum. They tend to do those things only after they start and need to learn how to do more complicated and higher level things. Most start out by clicking around and seeing what happens. They will know the general purpose of the software and try to figure out the basic workflows on their own. If they can't get that in 20 minutes or so they give up trying forever. Not everyone, but most, in my experience.

Give people an easy way to do something right away (a hook) without experience and you increase your chances dramatically of keeping them as a permanent customers who will start then reading, watching, and foruming.

adamw


10,447 post(s)
#11-Jan-20 11:42

I didn't read the thread yet, will do this later, but let me just quickly say the following:

While we don't think the UI of 9 is all that bad, we agree that in places it is less intuitive than 8. Nearly all of this comes from 9 operating with much bigger data and being much more extensible. Now, we are constantly working on the UI, and over time we tend to find solutions which make the UI simpler - frequently as simple as 8 - without really trading off anything in performance or extensibility. This takes time, though.

(Many things in the UI actually got much simpler than they were in 8. Eg, to run a query in 8, you have to create a component. In 9, you just open a new command window. Or this - to look at image pixels in 8, you have to create a query or script. In 9, you just Alt-click the pixel and the Record pane shows its values, plus values of the surrounding pixels. Or anything related to linked components - in 8, you have to bring components through the Database Console and when you want to bring other components from the same database, you have to go through the Database Console again. In 9, you just see all components under the data source - and the MAP file does not even have to store component data = fast. Etc.)

Here's a short non-exhaustive list of areas related to the UI which we seek to improve, no priorities implied:

  • ordering and some other operations on tables bigger than 50,000 records - talked on the forum,
  • select / transform panes - should be streamlined significantly,
  • panes overall - should be made independent from each other (the thorn is control over the preview, but we know what to do there) and have more docking options.

We have tentative plans to improve things for each of the above items.

We also have a similar list of features which we consider to be missing - it's not terribly long, but it isn't empty either (eg, raster selections should be per-pixel, labels should bend around curves, need web interface, there're about 10 such items). Some items are relatively big, others less so, but they are all on the list because we want to rework and extend the relevant features significantly - similarly to how we added, say, vector styles which got extended in many directions since 8.

All in all, your feedback regarding what specifically feels unnecessarily complex could help here, too. It's completely fine if you don't feel like submitting it, but if you did, we could use it to make things better faster.

Dimitri


7,413 post(s)
#10-Jan-20 03:51

there are so many more important things I'd like to see added that it is hard to want to write a suggestion for fear of delaying main things.

Agree with Tim that it is always worth it to a) post for discussion and b) send in a quick email. The first expresses faith in the collective intelligence of our community. The second expresses faith in the workmanlike ability of engineering to balance priorities, a routine part of advancing constellations of code with a million moving parts.

A good way to express the balance you feel is right is when sending in a suggestion is to say what priority you assign to that, and also to repeat what you feel is your top five most important things.

I file a suggestion every day or two, usually on small details of GUI action. My prime focus right now is small things that get on my nerves during repetitive production work. Examples:

If I'm creating lines, I often forget to exit object creation mode to get back to the default cursor, so when I alt-click thinking I've just picked an existing object, instead I mark the first vertex of the next line. My suggestion: "When in object creation mode (add area, add line, etc) just after creating a new object or before starting to create an object, Alt-clicking another object exits creation mode and picks that object, just like Alt-clicking from the default cursor more."

I also want a faster way to repeatedly create a series of lines or areas. My suggestion: "During Insert Coordinates mode, a Shift-Ctrl-Click creates the last coordinate and also saves changes. This would make repetitive additions of new objects faster. It's OK that this loses shift as a means of creating multipoints: those can always be created from the context menu."

Another one: "With tracker in use, an Esc should be the same as Ctrl-Back / Clear Tracker."

In my current list of 30 small details, the above three are priority 1. Other items are priority 2, 3 and so on.

I agree with Tim that 9's infrastructure is practically complete. What is most necessary now is leveraging that infrastructure with very many small details that a) make existing operations smoother, and b) provide GUI elements for infrastructure that is there.

The above three examples I gave are examples of a). A b) example might be how there exists infrastructure (can do it in SQL) to do things like taking vector areas and adjusting pixel values in rasters beneath them, but what you really need is a transform that does that, not expert-level SQL.

Neither a) nor b) things are difficult, because the infrastructure make such things easy. It's just a question of, as Tim correctly puts it, of exactly stating what should be, and stating the priority for it. If you focus on what you really need, there's actually not that much, a hundred or two items, which Manifold engineering will steadily pump out in version after version. It's at the point where every month the package is significantly ratcheting forward.

Anyway, that's a long way around to asking:

there are so many more important things I'd like to see added

What is your list, say, in priority order, the top 10 items? top 20?

Mike Pelletier

2,122 post(s)
#10-Jan-20 15:40

Here's my latest list that I sent in to Mfd in December and I said that these are needed in order for myself (likely many others) to start using 9 day to day. They sent a nice reply saying they agreed.

1. A table relation tool similar to Version 8 but hopefully with explicit ways to deal with one to many relates. While SQL offers lots of flexibility in how the relate should occur, it lacks the convenience (or is harder to setup) of working on original data with the related table.

2. Tool to georegister images. A basic method would be fine at first. Working with big images in 9 is wonderful but it has to line up in space with other data to be truly useful.

3. The new vector tools are awesome! To get them into use they need a few more things. A way to split and combine features. The ability to edit or add to traverse entries via the record pane directly rather than via a text field. A scale factor option for the traverse entries is extremely important. It would be nice to set the amount of rounding for values in the traverse window for readability.

4. Curved labels would add greatly to the map display caliber.

5. Legend and scale bar.

To describe 9's infrastructure as practically complete I think would scare off new users. I'm confident the above plus more will be added but new users might not. Sure SQL can do much and the speed is excellent, but not really useful for the items above. The term "infrastructure" is a bit of a fuzzy concept here.

Also, to help users be patient with tough code writing and confident in the future, I think it is worthwhile to acknowledge certain delays. Last year around this time there was much talk about legends, scale bars, and labels. Then they dropped off the radar for other more important work. It would inspire more confidence to acknowledge delays rather leaving us to wonder.

Once the above 5 items are done, I think Mfd will be getting a lot more feedback on all the user friendly things (your a and b).

dchall8
1,008 post(s)
#13-Jan-20 06:42

Good list. I would add a few.

6. Ability to print high rez, oversized PDF files (42 inches by XX inches).

7. Ability to print layered PDF files.

8. Ability to print KML files with format coding of Google Earth bubbles.

9. Ability to easily change formatting based on selectedness of objects. For example setting labels to transparent unless the object is selected, when the label becomes a visible color.

10. Ability to see through selected objects.

11. Ability to select objects and have them retain some of their original color, instead of selected objects turning opaque and pink.

adamw


10,447 post(s)
#14-Jan-20 09:10

Great items from both you and Mike, thanks a lot!

I have a couple of questions about items on your list:

8 - do you mean importing formatting from KML?

9 / 10 / 11 - this might be best solved with saving the selection to a persistent field and basing formatting on that field. It's pretty easy to save the selection to a field using the Select pane - if you didn't use that before, try it, and if you did and find the process too laborious, tell us.

dchall8
1,008 post(s)
#15-Jan-20 18:57

Adam,

8 - I mean exporting information into the KML export which includes HTML and CSS formatting from Manifold. In M8 you can select three fields when you do a KML export as illustrated here.

When you do the selections and open the file in Google Earth (1 to 2), and then click on any parcel, a bubble opens with the information you selected visible in the bubble. But you can also build formatting into the selected export field to enhance the bubble. Going from 2 to 3 requires SQL for the query plus HTML and CSS to change bubble colors and links. What I did was make one field an active field which updates upon request. The updated field pulses the rest of the table to get the owner name, address, legal description, etc. but it also creates links for driving directions (Google Maps) and a link back to the appraisal district website for more information about that parcel. The next iteration of this active field will include links to the county website for tax information, county commissioner, fire department, and the name of the local constable serving that parcel. These KML exports have been very well received by Realtors, surveyors, city managers, zoning code enforcement, law enforcement, fire departments, homeowner associations, and the general public. I can refresh/update the file in seconds and post it to our website for download.

9 and 10 - I'll look into the saving to field thing.

11 - In M8, which I realize is a different program, you could select parcels as shown in this next image.

The original colors of the parcels are retained and shrouded with a reddish shadow. The colors show through thus retaining the valuable information in those colors. In M9 everything selected turns pink, so no color information is visible.

Attachments:
KML Export from Manifold 8.jpg
Soil Type Sample Cropped.jpg

adamw


10,447 post(s)
#17-Jan-20 08:34

Thanks, the screens help a lot!

We'll see what we can do.

Sloots

678 post(s)
#14-Jan-20 18:07

12. The ability to make a png or jpg from a layout using a script. Much of my map production relies on this. (Looping through the objects in a drawing, select each object, the layout changes view because it is set to show selection, generate the image, export it as png, show another layer, loop again.... )

13. The option to set the view of a map in a layout to "selection". Needed for 12. Just like M8 does.

14. A North arrow

15. Ability to use variables in layout. text components. Equivalent of [component] or [scale] etc. in M8

13.


http://www.mppng.nl/manifold/pointlabeler

antoniocarlos

609 post(s)
#14-Jan-20 22:41

What I would like to see in Manifold in order of personal priority.

I am used to M8 GUI and love it but do not expect it to be exactly the same in 9.

1. From M8 I miss the formatting, query and transform toolbars in particular. The way these are embedded in the contents tab right now feels cumbersome for me. it may be that i am used to the M8 interface.

2. I would like to be able to easily duplicate a drawing (to make backups). If I understand the current workflow now I have to duplicate the table and reconstruct the drawing from the geom column. or export the drawing to GPKG.

3. The current Panes system is a bit cumbersome. Perhaps if they worked undocked (not in tabs)? Also if I undoc them it would be ideal to have some semiautomatic way to organize them rather than fiddle with them.

4. Right now I only use M9 to prepare data to export to create maps in QGIS or in M8. A lot of effort has gone into the formatting and styling capabilities of M9 that is wasted if it can not create (easily).

5. Is there a way to easily project all drawings and images in a project (or a Map Component) to a specific projection. I'm sure this can be done via scripting but it is beyond my capabilities.

6. Linking tables in M9 (in M8 I think it is called Relations) is a pain. the SQL to do this is easy enough but the user interface to do this in 8 was great for beginners.

7. I would like to be able to globally apply a projection to all components of a map or a project (normalize projections?).

8. Legends.. both for Drawings and for Rasters. (I see that we are close to these being supported). Some discrete raster data like land use maps come with information that could/should be presented in a legend. (Pixel value 1 = Urban, Pixel Value 2 = Rural) and so on.

9. The Zoombox button. I know im in a loosing battle here. But the zoombox is nearly universal in all GIS packages. I find that switching to other GIS software the muscle memory acquired by using the M9 interface actually works against me while working with other GIS packages. But this is minor.


How soon?

tjhb
10,094 post(s)
#14-Jan-20 23:30

My wishes as well. Even though I still think the product is basically complete.

This is in addition to allowing geometry values to be written to image pixels!


1. Legends.

To make these active, incorporating elements from version 8's ViewBots feature.

So add a new type of legend feature, resembling a computed field, plus formatting. I would call this an active item, a live element, or something similar.

It would be backed by an expression in SQL syntax, and would operate on the active layer (or only layer), either on selected records or on all records.

Its display in the legend would be updated automatically (the default) or manually.


2. Layouts. (Longer term.)

Get rid of them!

Instead, extend the map component type to include every valuable feature currently offered by layouts. Strictly speaking, we don't need both things.

The means might be a new component called a map group. Yes, this could mainly be a rename of "layout", with changes. The changes could be modelled on Adobe Illustrator's multiple artboards.

A map group would include multiple maps (or the virtual maps of drawings or images) as members or elements.

Each such member would have a location property (similar to a Location component, no need to recreate the wheel) to define its centre, extent and scale. The scale could be either floating (render for the current display, exactly as individual map components do now) or fixed (use a given ratio of geographic unit to display unit).

A map group would have a layout property, defining page size, the list of its members and the upper left position of each. (If absolutely necessary then also a printer. But please not.) Minimal.

A map group could be rendered to TIFF, PDF or ECW.

All these properties and operations could please be scriptable.


3. Maps

Make these arbitrarily rotatable.

Relevant legend elements should respond accordingly.

(It would be OK to use the Oblique Mercator math under the hood. It is very fast.)

antoniocarlos

609 post(s)
#14-Jan-20 23:48

I think you need layouts to support 2 o more maps in the same page, no? I dont understand the difference between a map group and a layout.


How soon?

tjhb
10,094 post(s)
#15-Jan-20 00:00

Little difference--which is probably my main point.

Two concepts could be merged into one, and possibly some object-specific baggage could be dropped.

In other words a map group could and should be minimally different from a map.

A map would have one or more extents (one or more artboards, in Illustrator parlance). Every map needs at least one extent. Sometimes more extents are useful--that is a layout.

It's inherently one thing.

An example of why it is useful. The legend for a map group should (as far as possible) be the same thing (with the same JSON) as the legend for a map. Copy and paste should be possible as far as relevant.

tjhb
10,094 post(s)
#15-Jan-20 00:37

Every map needs at least one extent.

That's false or almost false.

Depending on the projection, maps can have an infinite or arbitrary extent.

But... any map can have an extent (which could be set to arbitrary/infinite).

That allows map <-> map group to work.


How is the scale bar for a map group to work?

It should show "1 : <unknown>", until one map element is selected, then the scale that would be displayed for the selected element (member).

North arrow similarly: a blank circle until we know what we mean.

antoniocarlos

609 post(s)
#15-Jan-20 01:07

I see were you are coming from (i think) but there is also the issue of workflow organization. The project pane needs work in my opinion to support visual cues to large amounts of different components. iI my opinion the filter feature is not sufficient. For example It does not support (visually) temporary maps. That is these are still considered drawings until they are saved as maps. Layouts (as a distinct) component are helpful just to organize a workflow using the project pane. The icons for images could also be different if for example the image has one (a DEM or Slope Surface) or many bands (Aerial photos).

Aside

I don't know why related components cant have lines between them (at least tables and drawings). I may be thinking in M8.

BTW larger icons for components in the project pane would be very welcome. Going blind here..


How soon?

adamw


10,447 post(s)
#15-Jan-20 11:40

BTW larger icons for components in the project pane would be very welcome. Going blind here..

We might be able to scale icons according to the system DPI setting. We'll see.

Just in case: if you aren't manipulating DPI to help your vision, try doing this, and if you do this and run into any issues using it with 9, let us know and we will fix these issues.

Dimitri


7,413 post(s)
#15-Jan-20 07:50

Well, the main difference between layouts and maps is that layouts use paper space while maps use abstract, coordinate system space.

Paper space is the Cartesian space of a physical sheet of paper, where borders and placement are based on whatever unit of measure your culture uses to define measurements on a sheet of paper, and where the design takes into account the dimensions of the actual sheet size intended (automatically adjusting, hopefully, if the sheet size is changed). Tools like Illustrator are great at enabling people to position different items on that sheet of paper, to size them and so on. To do that well requires tools to support workflow of composing items on a paper sheet.

The evolving trend, of generalizing paper composition tools to also do composition for web pages on monitors is similar: it's still an issue of positioning on the physical space of a monitor's screen, whether or not your composition engine is reactive to on-the-fly changes in display screen physical characteristics (big desk monitor or tiny telephone screen).

The ultimate evolution of layouts is to provide accessories like horizontal and vertical rulers on screen, tug-snaps to round-number border placement, and a variety of Illustrator-like tools. The evolution of maps is different, because they relate to representation of physical items in large dimensions on the surface of an ellipsoid, ultimately going 3D. Perhaps it would be best to let them be related where they can be, but at the same time be free to evolve to be the best they can be for their specific tasks.

tjhb
10,094 post(s)
#15-Jan-20 08:34

Well, the main difference between layouts and maps is that layouts use paper space while maps use abstract, coordinate system space.

Precisely.

They are both layouts, they are both maps. One can combine multiple viewports, each at a fixed scale; the other currently cannot.

The differences are too arbitrary (or too precise) to matter fundamentally. It is expensive and unnecessary to maintain two separate component types.

(But I still care about points 1 and 3 more,)

tjhb
10,094 post(s)
#15-Jan-20 00:57

(To me my items 1 and 3 are much more important than item 2, especially for now.)

adamw


10,447 post(s)
#15-Jan-20 11:28

We have been thinking about point 2 for a long time ourselves. :-) It's a complex topic and there are big pros and cons to both making maps and layouts the same thing and keeping them separate. For now, especially in view of us adding quite a bit of functionality to layouts, we would like to try to keep them separate. Later, if we find out that they are best joined, we might join them - or maybe separate them slightly differently. We'll see.

Legends we are working on. Rotating maps we are not currently doing, but they will come as a side effect of some other changes we planned.

Thanks for the post, it's nice to see someone dig this deep into the product!

tjhb
10,094 post(s)
#15-Jan-20 21:13

Thanks for your thoughts on that. I'm glad I made some limited sense--although I greatly overstated my case. The devil would be in the detail.

Perhaps a natural moment to rethink the map/layout separation would come, in Dimitri's forward thinking above, at the point where maps embrace 3D rendering. It's easy to suppose, now, that that would apply only to boundless geographic data, without inherent scale. But perhaps the demand for 3D printed terrain will be so prevalent by then, and 3D printers so commonplace, that even "paper space" will also need the same two modes. Who knows.

Very much looking forward to what you are designing for legends.

tjhb
10,094 post(s)
#15-Jan-20 21:44

[Can't help adding, thinking out loud...

One day it would be nice to see layouts given multiple sheets, a third axis so to speak, where each sheet (2D or possibly 3D) could address a different place, theme, scenario, proposal, version or time, limited only by our imagination. The same could be applied to maps--a third axis again showing multiple versions, scenarios, proposals, times etc.

Sheets could be visualized as a 3D isometric stack. Sheet tabs could run down the right-hand side of the screen, so that particular sheets could be activated, with perhaps the ability to "scroll" or roll up and down the vertical tabs to quickly visualize changes over time, between scenarios, etc.

To be a bit clearer, map layers would still have their tabs across the bottom, which would still control compositing of each sheet in XY or XYZ; tabs down the side would give access to each different sheet. (So they would be effectively cubes.)

Here again I think maps and layouts could in principle be combined.]

Dimitri


7,413 post(s)
#15-Jan-20 07:35

1. From M8 I miss the formatting, query and transform toolbars in particular. The way these are embedded in the contents tab right now feels cumbersome for me. it may be that i am used to the M8 interface.

The Style, Transform, and Select panes will all become independent, like the Project and Layers panes. As adamw has mentioned, those will be reworked as well.

If I understand the current workflow now I have to duplicate the table and reconstruct the drawing from the geom column. or export the drawing to GPKG.

No, not at all. Duplicating a drawing depends on whether you just want to make another drawing that points at the same table, so you can format it differently (a "theme" in 8), or if you want to make a completely standalone copy. Either way, it's a simple copy/paste.

To just duplicate the drawing, click on the drawing to highlight it, and do a Copy and then a Paste. Done. You can use the toolbar buttons for copy and paste, the menu selections or standard Windows Ctrl-C and Ctrl-V.

To duplicate both the drawing and the drawing's table, Ctrl-click both to highlight them (that's a standard Windows move), and then Copy / Paste. Manifold makes a copy of both, and it sets the properties correctly in the copy of the drawing to refer to the copy of the table. That's illustrated in the Rename Related video.

Perhaps if they worked undocked (not in tabs)?

They do work undocked. Shift-click on the tab to undock. Ctrl-click to arrange in a second horizontal row.

Is there a way to easily project all drawings and images in a project (or a Map Component) to a specific projection.

That's not something that is a good idea to do blindly, especially for images, since settings are often different depending upon the data. Images, for example, often use things like local scale for pixels and offsets in a way that is different from drawings. That is why there are so many controls for things like metrics, a very different reprojection dialog for images than drawings, etc. You *could* script that to force it, but except for limited circumstances where you know all the components are amenable to a "one size fits all" approach, it's probably better to reproject them individually as best makes sense.

As a practical matter, in Manifold you rarely need to reproject eveything because you can always show what you want in a Map window. Choose the projection you want "for everything" in that map window and even if there are a hundred different layers in the map window with each of the components for each layer using a different projection, the Map window will reproject them all into the map's projection for display.

the SQL to do this is easy enough but the user interface to do this in 8 was great for beginners.

Very true. There are plans to make relations easier for beginners, without requiring them to write SQL. Manifold does that a lot with many things, for example Transform templates. Transform templates provide point-and-click, fill in a box, dialogs to do something which is implemented behind the scenes in SQL (click the Edit Query button to see the SQL used by the transform).

Relations/Joins, by the way, are an example of the data centric origin of 9, where it was built out as a rigorously competent DBMS engine, without hacks or sloppy short cuts. One reason relations are easy in 8 is because they are taking short cuts, and do so in a way that is not really compatible with a bigger structure where everything has to be able to work with everything else without any short cuts.

Joins in 9 are the real deal: no matter how complex the use of them subsequently, they will always work, and they are compatible with very big structures and with all the different data sources you might use. That's not easy to do, as you can see by how many other GIS packages support Joins in a limited way, or in some cases (MapD being a famous example when it first came out) despite many millions invested not doing Joins at all.

With 9, the data centric approach required Joins to be done right first, in a fully articulated way for the many forms in which Joins are used, and fully usable within the query engine to allow people who know how to use them to be able to use them in as sophisticated a way as they want. Once you have the infrastructure to do anything at all with Joins, then you can package them with an interface to make them easier for people who don't know or don't want to use SQL (now planned). But because the Joins are done right in the first place, you can create a more flexible and more powerful interface that is still easy to use.

I find that switching to other GIS software the muscle memory acquired by using the M9 interface actually works against me while working with other GIS packages.

This is easy to fix. As 9 expands you'll find yourself working with other GIS packages less and less. :-)

In all seriousness, though, the need to click buttons for basic nav things like zoom box is one of the things that very rapidly makes it annoying to use other GIS packages. Using left drag to pan and right drag to zoom box is sooo much faster and easier than clicking and unclicking a zoom box button.

The best thing would be for other GIS packages to copy Manifold and make that speed and ease available to their users.

adamw


10,447 post(s)
#15-Jan-20 11:06

In addition to what Dimitri posted, and as a quick recap:

1, 3 - we will make all panes independent, the Contents pane will be split (back) into individual panes.

2 - you can duplicate drawings fairly easily now (copy / paste both drawing and table).

5, 7 - point taken, will think about it and we had a couple of ideas running close to this anyway (mass-projection / mass-overriding of a projection), if you aren't afraid to mess with MFD_META, you can mass-override right now by copying / pasting coordinate system properties.

6 - we are working on a tool to help make this easier with no SQL needed (Relations like in 8).

8 - being worked on (legends).

9 - we do have zoom box, it's right-click and drag, no special tool needed, works in all modes (eg, during vector editing).

Thanks a lot for the wonderful post!

adamw


10,447 post(s)
#15-Jan-20 08:59

Thanks!

A note on item 13:

What we are hesitant about using selections directly is that in 9, they are not persistent, they exist only within the session of Manifold. In 8, they were persistent. There are many benefits to having selections not be persistent - eg, one big benefit is that you can select read-only data, and another big benefit is that changing the selection does not trigger the "your data has changed, you have to save it now" flag. But there are some drawbacks as well, and one of the drawbacks is that if you base something persistent on the selection, it has to work with the fact that the selection might be gone. So, if we have a layout frame and we set it to display data within the extents of the selection, it has to do something sensible when you just open the MAP file and the selection is empty. Maybe we should let the layout frame display all data in that case and adopt the universal notion that tools referencing the selection should just use all data in case the selection is empty. But this has its drawbacks as well - eg, when you try to establish a selection and your criteria selects no objects, you can no longer tell whether the criteria failed to select anything or whether it selected all data.

That's what has been stopping us here. We'll keep thinking, maybe we'll find a sensible solution.

What is absolutely non-controversial though is having some button or command that would zoom a layout frame to the extents of the current selection. (And being able to invoke that from a script.)

HMS
185 post(s)
#17-Jan-20 14:47

Maybe I'm a little bit late for this topic, but given M9 amazing capabilities and that I'm really on the verge of making a full transition from M8 to M9, there are still a few steps lacking that prevent me to do so. Here's a list with a few tools/options that are present in M8 and that a similar capacity in M9 would allow me to make a full transition (I also sent these ones to suggestions):

1 Legends: these are fundamental for my workflow, without an automatic way to show them producing a layout is not workable (mainly when you have drawings with dozens of attributes that you have to make evident in the layout). Still on the legends topic, a way to hierarchize attributes in the menu would be awesome. Actually, to do this sort of hierarchization within M8 is also a headache, given that the "Format" pane is very limited and if you have more than around 16 items you have to constantly going between back and forth the side bar and the options on the left to change the settings for the "lower" records.

2 Georegister tool: it is essential as georegistering is such a common task;

3 Scale bar: a way to show an automatic scale in a view or layout is essential. I know we can draw one manually, but it's just a way to add more time to your workflow when you are already overflowed;

4 Layouts: showing grids associated with projections is fundamental to make them usefull in a professional way (at least in Portugal). Without this option the layouts are not acepted by the client/authorities. For instance, the way to choose between diferent grid spacings is fundamental, given that in a printed layout we can only have a maxim distance of 10cm between grid points/intervals);

5 Tranform bar: editing drawings is still a great part of my daily tasks, so a easy tool to split areas/lines , like provided by the M8 transform bar "Split with" is also essential. A suggestion would be to provide a tool with a "cutting" line that could abreviate the process of adding a line and than splitting the object;

6 Transform bar: the transform bar was very intuitive to make operations between table columns like the "Fill with..." options that, right now, are not so user friendly in M9;

7 Transfer selection: The "transfer selection" M8 option (mainly between drawings and images)

8 Layouts: adding vector elements to the layout (like lines and text) would be great. It seems that making everything out of a frame its just not as simple as it was in M8, given that you have to add more layers to the project pane for “entities” that are indeed not “geographical data layers”. The copy/paste between these elements in M8 worked fine.

9 Showing legends associated with wms links when those are provided by the server;

10 “Export image” in a similar way to what happened in M8 (for instance you can quickly click on an image with the right mouse button and make a georegistered image).

Also, I didn't explore this topic in the manual, but is there a way to import the Z value associatted with a dwg/dxf into M9 (like it happened with M8?

The hierarchization of these topics is highly changeable since I use all these features regularly in M8, so I hope this could be still helpful for this topic.

Dimitri


7,413 post(s)
#17-Jan-20 15:22

Also, I didn't explore this topic in the manual, but is there a way to import the Z value associatted with a dwg/dxf into M9 (like it happened with M8?

Don't know, (it's not mentioned in the DXF topic) but if Z were imported that would likely be brought in as a Z value for the Geom. 9 understands that objects might be 3D, in that each coordinate in the object might have a Z value as well as the usual X and Y values.

It's easy enough to try it and see:

1. Import a DXF where you know objects have a Z value to them.

2. Open the drawing's table.

3. In the Select pane, click the Geoms with Z template. Does the preview show rows highlighted? If so, those have a Z value in the Geom and that means the dataport brought in Z values and put them into the Geoms.

---

If you have Z values in your Geoms, you can put them into a field. With the table open, launch the Schema dialog. Add a new computed field called, say, Z, and create that field based on the expression:

VectorValue(GeomCoordXYZ([Geom],0),2)

That will generate Z numbers from the Z part of the Geom.

---

PS: That's a great list! Agree 100%.

HMS
185 post(s)
#17-Jan-20 15:59

Thanks Dimitri. I'll try following these steps for the dxf import and I'll give some feedback asap.

HMS
185 post(s)
#20-Jan-20 14:03

It worked perfectly with the dxf files and that's a great way to confirm the Z value was correctly set.

adamw


10,447 post(s)
#17-Jan-20 15:40

A superb list, thanks a lot!

Regarding item 6 - could you elaborate what specific transforms you find less friendly than in 8? Is it that we are missing some transforms or that it takes more clicks to run a specific transform than before or ...?

HMS
185 post(s)
#17-Jan-20 16:10

Thanks Adam.

In particular one the most common transforms that I'm used to apply in M8 is "Fill with sum" or "average" like filling "Column A" with sum (or average) of "Column B" as this is was really easy and intuitive in M8. Also arithmetic series was a really easy way to populate a table with an index.

I also miss some right mouse button table options, like the fast way to copy and paste a column without having to go through the "Edit schema" procedure. Nevertheles, I understand the infrastructure is entirely different and you have to compromise in some points. By now I'm getting used to "edit schema" options fairly well.

danb

2,064 post(s)
#17-Jan-20 19:33

I know it has been mentioned before and a SQL solution proffered, but from the selection pane, I really miss duplicates, duplicates except first which I use all the time when quick checking data.

The panes are really well designed an work well for transforms and when extending them with expressions, but personally I do find them a little cumbersome and also sometimes confusing. For example, when performing topology overlays, I am never really sure which is the Target and which is the Source/Argument. This is probably my own unfamiliarity coming through however (RTFM)

I loved the simplicity of the transform toolbar and while I understand it is not suited to all transforms, perhaps something similar may be suitable for a subset?

While we are on lists I will add my two top items, the first of which is mentioned in the recently revived 'M9 Modifying pixels in a raster using a vector features as analysis masks' thread.

It is just that, to have the facility to either modify pixels directly in an image that are covered by a vector feature, or to impose a selection from irregularly shaped vector features onto an image for subsequent modification by SQL, transforms, code. For me this would largely complete the round trip between vector and raster types.

My second would be the addition of tools akin to Viewbots which are so very useful for weighing up checking and validating data as it is built.

I am also so glad to see the quality of life improvements starting to come. These make the product so much more approachable and easier to use. I am really enjoying being able to sort on the column header and Best Fit / Best Fit Title commands, but the one I really used to use was Best Fit Titles which did all headers. Is it possible to add this?

All of the tireless work that goes into the product is very much appreciated and I feel like a critical mass for full acceptance of 9 is coming. Every build there seems to be something groundbreaking in terms of speed or productivity which is just great.


Landsystems Ltd ... Know your land | www.landsystems.co.nz

Dimitri


7,413 post(s)
#18-Jan-20 05:01

to impose a selection from irregularly shaped vector features onto an image for subsequent modification by SQL, transforms, code

I agree with the above. I think why it's not here already is that it is a logical part of adding a collection of per-pixel-selection raster tools, which begin with adding the ability to select sets of pixels, as opposed to sets of tiles. If you're going to process an entire raster, which is what tools like watersheds do, it makes sense to do it by tiles. But if you want to select some pixels and knock out their values to make them NULLs or otherwise edit only their values, then you need per-pixel selection and processing.

The moment you add per-pixel selection, especially in a transfer selection way from a selection in a vector layers, well, then immediately you need to add some per-pixel raster modification tools. Otherwise you get a situation where it's easy to select exactly the pixels you want to modify but then you don't have point-and-click ways to modify them. So it makes sense to try to do that in a clump, with at least some point-and-click raster tools appearing along with point-and-click transfer vector selection to raster per-pixel selection. That and way more additional raster tools will come, because 9's infrastructure can support much larger rasters than 8 can.

That's one of the big desires planned for not too far out, but I think closing the loop on the editing cycle that got started - "split" is on my list too, also "join" and "extend," etc... :-) - and doing legends should get done first.

On a personal note, I have to say it warms the heart to see praise for Viewbots. When they came out I was slightly bummed not to see them talked about very much. 9 will have something like that: the people who invented Viewbots are still big fans and these days such "control panel readouts" or widgets are rightfully popular.

tjhb
10,094 post(s)
#18-Jan-20 05:12

On ViewBots specifically, I would have been interested to hear your thoughts Dimitri (and Dan's) on my suggestion that they be integrated with legends.

That is their natural home, isn't it.

pslinder1
228 post(s)
#22-Jan-20 01:51

I second the need for Viewbots. They are a sorely missed capability from Manifold 8. I used them all the time. I would also like to upvote, the selection and transform toolbars. These were very useful and very simple to understand. Its good to have that immediacy always available without having to go to the transform pane. The georegistration tool is also one I very much want. It should be better implemented than in 8 though. I think you should allow placing the control points on an image and then place the same points on a map so you do not have to copy and paste lat/longs if you don’t want to.

My top three list of desired features:

1) Fully integrated 3D viewer. By fully integrated I mean you should have all the layering and viewing capabilities that you currently have in a map and drawing with the addition of being able to adjust your height and viewing angle. All layers with a Z dimension in a map should be 3D-izable. Rendering needs to be fast. A 3D viewer would not be very useful unless it was just as (or at least almost as) fast as panning and zooming in a 2D drawing or map.

2) Improved usability of Transforms. When using the transform pane there should be more help immediately available for each function or template. There should be an explanation of the function’s purpose and variables. There should also be two or three examples. All of this needs to be viewable without obstructing the view of the transform pane or hindering the user’s ability to simultaneously work in the pane.

Another big area for usability improvement in the Transform Pane is more help in filling out an expression. I always hate that when I pick an expression such as:

Trunc(<value>)

and then edit it by double clicking on <value> and it selects (<value>). Now I have to retype the parentheses when I just want to put in a value or a field name. Why do it that way? Why not allow the selection of just <value>? Also almost every time I have to input a value it is a field (column) name of a table I am working on. Why not have a drop down list option of all field names that filters down as you type? I think it could even work in a map with multiple tables. It would make expressions much easier to use.

3) Improved usability of the Tables. I love the Manifold 9 concept of everything is a table. But I think working in them could be improved. I would love the ability to have the option of adding or deleting a column directly in the table that I am working with. I see no inherent virtue in being always forced to go through the schema menu option instead. Also it would be good to be able to enter a math function or expression directly into a cell a-la “Excel”. When building a function, you should be able to reference other columns by clicking any cell in that column. This would be very useful for simple calculations or functions that the user is very familiar with and does not need the handholding of the Transform pane.

I also think you should provide "Fill" options, like in Excel, to quickly copy info (or create series) in columns of cells. This is particularly helpful after having filtered a number of rows. (BTW you need a more prominent way to show that a table is filtered. That little funnel at the top left corner is too subtle) Also, what happened to the ability to reorder columns that was there in Manifold 8?

The Transform pane is a great thing in Manifold 9 but it is too rigid of a method for quickly working with a table. In general, everyone knows how to use a spreadsheet (thanks to Excel and Google Sheets) why don’t you let them transfer more of those skills to Manifold 9 Tables?

tjhb
10,094 post(s)
#22-Jan-20 06:25

I can't help feeling that some or most of that is caused by not having put the effort into getting to know Manifold 9 as a product, an environment, separately from Manifold 8.

They are different products not just partly but entirely.

What drives Manifold 9? What makes it so fast and so reliable? How is it built on its data?

If you can answer those questions, then try to see how the user interface must be built bearing those advantages in mind. (If you can't, I can give pointers for reading.)

In words of one syllable: the huge wins are not all for free. Some need new eyes, and new skills.

pslinder1
228 post(s)
#22-Jan-20 14:43

I would not say they are entirely different products. I mean they are both GIS tools and ultimately have the same purpose.

I am not sure how Manifold 9's speed and reliability would be compromised by a 3D viewer or making <value> directly selectable.

I do however sincerely appreciate the offer of the reading list and the simple lucidity of your closing argument.

Dimitri


7,413 post(s)
#22-Jan-20 15:27

I am not sure how Manifold 9's speed and reliability would be compromised by a 3D viewer or making <value> directly selectable.

[Edit:] My post crossed with adamw's. I see you meant something different than what I understood you meant for how double-clicking on <value> should work. My apologies.

As for 3D viewing, nothing about that would compromise either speed or reliability in 9. But, it would tremendously annoy all those people who have been diligently lobbying for their priorities by sending in suggestions if Manifold put a 3D viewer ahead of, for example, legends, filling in the roster of vector editing tools, improvements in layouts, image georeferencing, etc.

I agree 3D viewing is a natural for 9, especially given all the advancements in rasters and rapid evolution of LiDAR capability. But I don't think it should be put ahead of, say, either vector editing or a super easy built-in web server. It's just a question of priorities and when something happens in the sequence of new features that come out, and not "if."

pslinder1
228 post(s)
#22-Jan-20 19:35

Fair enough on the priorities. I just said it was my priority in the spirit of this thread. Did not mean to imply it should be first on Manifold's list.

tjhb
10,094 post(s)
#24-Jan-20 06:56

For a reading list, try starting with this very short post.

tjhb
10,094 post(s)
#24-Jan-20 07:13

(I think you must understand that, to help you post other helpful ideas.)

pslinder1
228 post(s)
#26-Jan-20 04:36

This was a very interesting thread and it got me thinking.

Why not just let the user (from an interface point of view) change the field type and then in the background manifold just creates a computed field copies the result and pastes it into the current field and then deletes the computed field? or copies it into a new field and deletes the original field and renames the new field the same as the original field?

adamw


10,447 post(s)
#27-Jan-20 09:51

We are considering adding a command to change the type of an existing field which will do more or less that (create new field -> copy data -> remove old field -> rename new field to old), yes. There will be some restrictions (ie, no changing type of a field that is referenced by a computed field), but nothing too bad.

This is a classic example of a feature which (a) appears simple but is only simple when an application is working with a single data format (like 8 or many others, and unlike 9), so (b) we dropped it for 9 at first, and now (c) we are going to re-add it and in the new form it will work for many data formats.

pslinder1
228 post(s)
#27-Jan-20 22:50

Makes sense. I like it.

Dimitri


7,413 post(s)
#22-Jan-20 08:11

Some comments on how to get what you are asking for today...

When using the transform pane there should be more help immediately available for each function or template. There should be an explanation of the function’s purpose and variables. There should also be two or three examples. All of this needs to be viewable without obstructing the view of the transform pane or hindering the user’s ability to simultaneously work in the pane.

That's a lot of screen real estate to use up, which would hinder the view of other things (like the map or table being manipulated, for previews, etc.). Experience shows that once people learn a system they get really annoyed about having lots of screen real estate used up to show them help and examples they don't want. Nice to have in the very beginning, but thereafter turned off.

But if you want the above it is easy to do: launch a browser and visit the User Manual's transform topic for drawings or whatever. A quick Ctrl-F (for Find) on the name of the transform takes you instantly to the write up for that transform. If you want help constantly in view, it's a good idea to use at least two monitors so you have plenty of room on your desktop both for working windows, undocked panes and browser windows displaying help. You can even have multiple browser windows open to different topics at the same time.

and then edit it by double clicking on <value> and it selects (<value>). Now I have to retype the parentheses when I just want to put in a value or a field name. Why do it that way? Why not allow the selection of just <value>? Also almost every time I have to input a value it is a field (column) name of a table I am working on. Why not have a drop down list option of all field names that filters down as you type?

You're making it harder by not using the facilities that are there. Don't double-click <value> in the expression itself. Instead, highlight it and then double-click the field you want to add from the Fields portion of the list in the pane below. See the discussion under Expression Tab in the Transform Pane topic, as well as examples in the Query Builder topic.

It's done that way because it's more or less a universal paradigm in Windows text editing that you highlight text to do something to it. For example, if you want to replace some text with what you've copied to the Clipboard, you highlight that text and then press Ctrl-V... the paste replaces what was highlighted. It works that way in Notepad, in all the Word editors, and in Windows style text panes in applications, like the expression tab's text pane.

The advantages of that are the usual Windows moves continue to work when you're writing something, be it a poem or a snippet of SQL in an expression tab, while at the same time you can take advantage of auxiliary assistance, like just double-clicking a field, function, operator, etc., to add it without having to keyboard it. I think if you added different facilities to the text window, like double-clicking some text in <> brackets to pop open a pull down menu... what would that pull-down menu be? It would end up having to duplicate everything that already is in the list immediately below the expression pane.

And at the same time, it would lose functionality, in that the list would not contain values you might have copied to the Clipboard. Keep in mind that <value> is not limited to being just a single number or the name of a field. It can be an entire SQL expression that evaluates to whatever is a legal value. It could be an SQL expression you've copied from some web page somewhere, from a forum thread, from a help topic, etc. It's extremely convenient to be able to put that into <value> in the usual text way, by highlighting <value> and then pressing Ctrl-V to Paste.

Just saying, there is lots of value in how it is done - way more value than in 8. It's always good to come up with improvements, but a genuine improvement adds more value without a net loss to value by losing useful capabilities already there.

I see no inherent virtue in being always forced to go through the schema menu option instead.

It's that way because there are many capabilities related to such things (adding and deleting parts of tables, such as fields, constraints, indices, etc) which the Schema dialog provides that would be cumbersome or misleading to try to provide in the form of a context menu that pops open with a right click on a column head.

Besides providing faster, less error-prone and more comprehensive access to such capabilities, the Schema dialog provides a framework that people doing advanced work with such capabilities feel more comfortable with. In general, when you have a system with a wide range of sophisticated capabilities, you don't want to reduce functionality when crafting interfaces that are more efficient or easier to use.

In the case of tables, it's easy to forget that many people use 9 for very sophisticated manipulation of tables in big databases. If you are a database administrator for a DBMS that has tables with billions of records that are used in hundreds of queries by many people, you take any addition or deletion of a field very seriously, as rebuilding indices in tables or shooting some index in the head by deleting a field it uses can be a very big deal. They like doing such things in the context of seeing the entire Schema of the table.

It's a very different situation if it's just an individual user and a table with a few hundred records where you prefer to just do what you want. I feel the same way. There's good arguments to be made that if you just want to right-click a column head and press Delete it should be vaporized immediately. But there's also a good argument to pop open the Schema dialog to show that field in context with the schema so you get a look of how that field fits into the overall schema of the table, with the extra step being an "are you sure?" sort of safety step.

As for adding a field, you'll still need all the infrastructure for adding a field that the Schema dialog gives. You just skip launching that stuff from the Schema dialog and do it directly from a right click on the column head. But then you'll either need more context menu choices (for adding an index, a constraint, quick add of identity + index, etc.) that are top level conveniences in the Schema dialog, or you'll need to create some intermediate dialog that does what Schema does. It seems more efficient to just use Schema instead of forcing people to learn some alternative thing.

As much as I like "just do it" modes for when I'm doing simple, sloppy stuff, I don't think it's a big deal to pop open the Schema dialog: Ctrl-E does the trick, quicker than right-clicking a column head and choosing "Add Field" or "Add Index" or "Add constraint" from a context menu, especially when some things you want to add aren't really tied to that column head or just that column head. I guess for those who don't want to remember Ctrl-E, adding "Schema" as a command to context menus on column heads would be helpful.

Also it would be good to be able to enter a math function or expression directly into a cell a-la “Excel”.

If you want expressions in fields, use computed fields. If you want individual cells, that is, for only one record, to be computed from expressions, use a spreadsheet application like Excel.

There are tradeoffs to using Excel: it is absolutely wretched at doing the database type things people most often want to do in GIS, and it drops dead with more than just very small numbers of records. Why? Because structuring the thing to enable each individual cell to be computed from a unique expression works against the infrastructure and user interfaces that make for efficient and effective database capabilities.

Databases, in contrast, can compute an entire field, but generally limit it to that so they can retain user interfaces and internal machinery that support effective databases.

The user interface issue is part of that picture: spreadsheet programs like Excel have huge investment into user interfaces that support the use of individual expressions within each individual cell. You could do that with databases too (if you were willing to take the performance hit), but I think an application that combined all the learning needed to handle a DBMS plus also all the learning required for Excel would be very hard for most people to learn.

Also, what happened to the ability to reorder columns that was there in Manifold 8?

9 has much better ways to reorder columns. See the Tables topic, the Sorting Columns topic, and the Filters topic.

pslinder1
228 post(s)
#25-Jan-20 22:47

A quick general comment. I am not really asking for help here. I mostly know how Manifold works. I have done a lot more reading than I want to. I am sincerely trying to improve the usability and comfortableness of the program. I am kind of a slow writer but I am putting in the time because I really like Manifold and I think it could and should be a much more widely adopted program. I know this forum caters mostly to GIS professionals but Manifold should really have an audience that is at least a couple of orders of magnitude larger. After almost 20 years of using the program I have come to the conclusion that one main reason that it has not been more widely adopted is because the UI is problematic.

That's a lot of screen real estate to use up, which would hinder the view of other things (like the map or table being manipulated, for previews, etc.). Experience shows that once people learn a system they get really annoyed about having lots of screen real estate used up to show them help and examples they don't want. Nice to have in the very beginning, but thereafter turned off.

You are right. I was not specific enough. By available I mean that there should be some kind of help button for the function, expression or template that brings up all of the help info I mentioned about in a way that is persistent (does not disappear as you continue to work in the transform or select panes) and does not block visual access to the pane you are working in.

It's a very different situation if it's just an individual user and a table with a few hundred records where you prefer to just do what you want. I feel the same way. There's good arguments to be made that if you just want to right-click a column head and press Delete it should be vaporized immediately. But there's also a good argument to pop open the Schema dialog to show that field in context with the schema so you get a look of how that field fits into the overall schema of the table, with the extra step being an "are you sure?" sort of safety step.

I am like you; let the vaporizing begin. But even if you choose to be more careful you can still right button click delete then have an extra popup ask "Are you sure?". I think that even with that extra step I would like to have that capability rather than first going through the schema.

As for adding a field, you'll still need all the infrastructure for adding a field that the Schema dialog gives. You just skip launching that stuff from the Schema dialog and do it directly from a right click on the column head. But then you'll either need more context menu choices (for adding an index, a constraint, quick add of identity + index, etc.) that are top level conveniences in the Schema dialog, or you'll need to create some intermediate dialog that does what Schema does. It seems more efficient to just use Schema instead of forcing people to learn some alternative thing.

I disagree. I am usually just going to want to add field so I do not think you also have to clutter the context menu with add constraints, index and identity. You can still have the Schema screen for those things. After you choose "add field" or better yet "add column" and you bring up the add field popup you could add an "Options..." button that would bring up a secondary screen that would allow you to add an index or a constraint for only that field you are creating.

In general I think being able to do things directly in the table as opposed to going through the schema gives the user a sense of power and immediacy that pays some weird, but real enough, psychic dividends.

There are tradeoffs to using Excel: it is absolutely wretched at doing the database type things people most often want to do in GIS, and it drops dead with more than just very small numbers of records. Why? Because structuring the thing to enable each individual cell to be computed from a unique expression works against the infrastructure and user interfaces that make for efficient and effective database capabilities.

Yeah I get this to a certain degree. But Manifold already provides capabilities to edit individual cells directly, it also can do multiple cells through the copy to selection function. You actually can edit any subset of cells through your comprehensive Selection and Transform capability. So if that capability already exists I am just talking about altering the interface slightly. For instance you have a selection but instead of copy to selection you type in a simple function and the result gest copied to into the cells. You could also filter on the selection and type in a function directly in a cell and do a drag and copy fill down to the entire filtered selection.

I am not saying you have to change the field, just that how you get info into a subset of the cells could be more excel like. But as you rightly point out because this is a database you could not have a function remain active in only a subset of the cells. It would be a temporary method for updating a field for a subset of rows using a familiar excel type interface to accomplish it by typing in a function and doing a fill copy.

This is just an interface change not a change to the underlying infrastructure of Manifold's 'Database'. I am confident of this because you already allow all of the functional capabilities I am talking about.

adamw


10,447 post(s)
#22-Jan-20 15:15

Thanks a ton for a lot of great feedback.

We do hear you and we think that the post contains a lot of good ideas.

A couple of things to add to replies from others:

Double-clicking 'a' in 'Trunc(<value>)' in expression builders should select '<value>' instead of '(<value>)' - agree, we will do this. (We agree on the auto-complete lists as well. But they are trickier in that a dumb implementation ends up with a huge list that isn't useful and smarter implementations that filter the list sensibly take time to implement well, so we have several relevant items on the wishlist for the future, but are currently busy with other things.)

We agree on variants of the Fill transforms, they were requested by others as well. We do have a basic Fill transform, but we need variants for various statistical measures and series.

On reordering columns, I think you mean changing the visual order of columns in the window from left to right. You can currently do this using the Layers pane (yes, I know, it might feel strange to use the Layers pane to control fields in a table window, but fields in a table work very similarly to layers in a map - maybe we should be changing the caption of the Layers pane to Fields when a table window becomes active). We will re-add re-ordering via drag and drop as well.

We agree that we need to show when a table window does not list all records in a table more prominently. There were several related discussions on the forum regarding table windows, we are considering making several changes / additions based on them.

Thanks again for the wonderful post!

pslinder1
228 post(s)
#24-Jan-20 03:37

(We agree on the auto-complete lists as well. But they are trickier in that a dumb implementation ends up with a huge list that isn't useful and smarter implementations that filter the list sensibly take time to implement well,

To start with I think it would be enough just to have the field names in the list. The majority of the time that is what people will be replacing a value for I think. At the very least the field names should be at the top of the list as it auto-completes (maybe functions come second).

On reordering columns, I think you mean changing the visual order of columns in the window from left to right.

Just so. I look forward to getting the drag and drop capability back; thanks.

You can currently do this using the Layers pane

It looks like reordering is also currently theoretically available in the schema dialog but the arrows always seem to be grayed out.

Dimitri


7,413 post(s)
#24-Jan-20 07:21

It looks like reordering is also currently theoretically available in the schema dialog but the arrows always seem to be grayed out.

If something is puzzling, don't guess. Instead, read the relevant topic for insight. From the Schema topic:

Enabled only when adding more than one new field, to allow setting the order of newly-added fields relative to each other.

When adding new fields, you can manipulate their order: that's a convenience for working with the schema, where it is often convenient to see like fields near each other in terms of confirming the pending changes are what you want in terms of data type, etc., and precedence in the dialog is important for computed fields and priority of indexes. But that's not about display order.

The order in which columns appear in a table is somewhat like how there is no order for records: order of columns logically is not something that is a structural aspect of a database but, instead, is purely a matter of display choice. Order them however you want in the Layers pane, including choices of which should be hidden or displayed, and that's how they will be displayed.

The abstraction of the structure of data, for example, the data types, collations, whether indices or constraints apply, and so on, from the display of data is a very powerful and useful thing. For example, it makes it possible for many different queries to create custom views of desired data from the same tables without having to duplicate those tables for every such view. It's like how you can have many different drawings all showing the same field in the same table, but styled in many different ways, to suite different needs.

The place to set display characteristics, whether they be the left-right order, size of, hidden/displayed status of columns or (coming soon to a 9 near you...) the format of how data types like dates or numbers are displayed, is not in the definition and management of the fundamental infrastructure, it is in those dialogs that control the display. For drawings, that is primarily the Style pane. For tables, so far, it is primarily the Layers pane, but I can see how some time in the not too distant future we might be using the Style pane to do things like specify the style with which the contents of cells might be displayed.

pslinder1
228 post(s)
#25-Jan-20 21:51

Thanks that is a good explanation. And the Schema topic is complete. I am always impressed when I look at the Manifold documentation, it is so thorough and very clearly written.

However, I have to say that I feel sometimes as if Manifold relies a little too much on their documentation. Although, there will always be situations for every user where they will need to eventually hit the documentation, the goal should be to try to keep them from having to go there as much as possible. Every time a user goes to the documentation Manifold should feel a stab failure. I often get the impression that Manifold takes RTFM as a point of pride. It really should not. The more you need the manual the less intuitive the program.

As far as ordering in the schema goes I think you have to admit the feature is confusing. You allow reordering when you create a new field but not if you edit a preexisting one. I take your point that there is a difference between the real order and how they appear. If you have not set an order in "Layers" do the columns appear in the default order of the schema?

Currently if someone adds a field that refers to a field that comes after it in the order you do not allow the user to save that field to the schema. Why not allow users to change the order of existing fields in the schema and if they make a mistake don't allow them to save the changes and throw up a warning?

You offer a capability in one case (when you add field(s)) but not in another case (when you are editing the schema) and there is no clue in the interface as to why. The reason you gave does not seem that compelling; if a user wants to change the order for some reason I say let him. If he falls foul of the order rule tell him he has and don't let him save the changes, same as for new fields.

But if you insist on keeping the capability for only new fields then I would say instead of graying out the arrows at the top of the Schema popup have them disappeared until the user creates a new field and then they can appear. That would give the user a clue that there is a capability related to only new fields. That's beats making him go the manual .

BTW the selection in Layers pane and the Schema window work the exact same way as in drawings/maps and tables even though there is no real risk of Adam's "selection fragility" or Dimitri's "accidental editing". I guess it makes sense to keep all parts of the interface consistent but it still feels weird. I still find myself forgetting sometimes when I am not in the main windows that I have to select with the ctrl+click.

tjhb
10,094 post(s)
#25-Jan-20 22:15

However, I have to say that I feel sometimes as if Manifold relies a little too much on their documentation. Although, there will always be situations for every user where they will need to eventually hit the documentation, the goal should be to try to keep them from having to go there as much as possible. Every time a user goes to the documentation Manifold should feel a stab failure. I often get the impression that Manifold takes RTFM as a point of pride. It really should not. The more you need the manual the less intuitive the program.

That is partly backwards in my opinion.

The time when every user needs to hit the documentation is on day one, not "eventually". Every new user must read the introductory topics, which are clearly signalled, and kept to a minimum.

After that, yes, it is a matter of discretion and intuition and personal preference, and available time, and efficiency, and sometimes bloodymindedness, and even gender. "If all else fails, read the instructions" applies a bit to everyone.

Yes, it can only be a point of pride for Manifold engineers and UI designers when recourse to the manual is unnecessary. So I wholeheartedly agree with the last sentence above. It's the same with the law and legislation. If the law is a surprise, or needs to be read, then it's probably not very just.

But recourse to the manual is not a signal of failure. Sometimes it is necessary, and more often than that, it can save a lot of time. This is not a smartphone.

You offer a capability in one case (when you add field(s)) but not in another case (when you are editing the schema) and there is no clue in the interface as to why.

The reason for that is simple. In the second case, the change means making a trivial but expensive change to every existing record in the table (possibly billions or records). Do you want to wait while the system fetches everything and writes it all back? In the first case, no existing data needs to be touched, since the change only affects future data. No waiting.

pslinder1
228 post(s)
#26-Jan-20 04:19

In the second case, the change means making a trivial but expensive change to every existing record in the table (possibly billions or records). Do you want to wait while the system fetches everything and writes it all back?

I am not sure that is necessarily true but lets assume it. My main point here was the interface does not clue you in. Wouldn't it make more sense to not even show the arrows until the user creates a new field? That way the user would have a clue that the ordering capability is only possible on the new fields.

adamw


10,447 post(s)
#27-Jan-20 10:18

We disable buttons that re-order fields when the selected fields cannot be re-ordered (the way a particular button does it). That's a clue-in. What you are talking about is that we don't make it apparent as to why the fields cannot be re-ordered. But this "why" is hard to express visually in the UI (how do you express "the field cannot be re-ordered because it already exists and existing fields cannot be re-ordered because the physical order of fields is mostly of no interest other than for computed fields which use it to avoid creating circular dependencies"?), so we leave it for the documentation to explain.

This is just a remark on this particular issue, we agree with many other points you made (and yes, the less often you have to go into the documentation, the better, that's why we are doing things like organizing keyboard shortcuts so that they are tied to commands that also appear in some menus, so that you can invoke the menu to recall the shortcut, etc, etc, etc, we are constantly trying to bring the documentation into the product so that you have to go to the actual help topics less often).

pslinder1
228 post(s)
#27-Jan-20 22:32

we are constantly trying to bring the documentation into the product so that you have to go to the actual help topics less often

I think that this is an excellent approach, keep driving it forward. I think this is especially important when working with functions (expressions) and to a lesser extent templates. Having the help in the program and view-able while still working is a huge win. Having to leave the program and go to a browser and then look up the function is just annoying enough to discourage exploration.

But this "why" is hard to express visually in the UI (how do you express "the field cannot be re-ordered because it already exists and existing fields cannot be re-ordered because the physical order of fields is mostly of no interest other than for computed fields which use it to avoid creating circular dependencies"?)

You have the little arrows grayed out now and they do not become black until there is a new field and that field has been selected. My thinking was that you do not show any arrows until the user creates a new field. So instead of grayed out they are simply not there. Then as soon as a new field is created they appear grayed out and when you select the new field they turn black. It is not perfect but it does supply a pretty strong clue that this capability is only for new fields.

I actually think though that the better thing to do is to let people reorder even existing columns. If they fall afoul of the circular reference rule issue a warning and don't let them complete the reorder (you basically do that now if you have an illegal order with a series of new fields). I assume that one can currently reorder a table using SQL; if you already 'offer' that capability why not let users do it more simply in the schema window? If it takes time, warn the user and so be it.

Adam, a few questions:

Does referring to a column that is later in the table order inherently create a circular reference? Or is it just that it is impossible to have circular reference if the column being referenced is earlier in the order? Would it be possible (without much effort) to quickly check a user's desired reordering and see if an actual circular reference has been created?

Is it really true that if you were to reorder the columns of an existing table it could take a really long time depending on the number of records? Does the program really have to fetch all the data and write it back before the user can work with the table again?

adamw


10,447 post(s)
#28-Jan-20 08:20

Does referring to a column that is later in the table order inherently create a circular reference? Or is it just that it is impossible to have circular reference if the column being referenced is earlier in the order? Would it be possible (without much effort) to quickly check a user's desired reordering and see if an actual circular reference has been created?

No, referencing a column that is later in the table order does not necessarily create a circular reference - as you say it is simply that following the rule of only referencing columns that are earlier does not create such references. Yes, it would be possible to check whether computed fields have circular dependencies without the rule, however circular references can get pretty complex especially if we ever allow setting / removing / altering expressions for existing fields - which can be pretty useful for some scenarios - and it feels simpler just to have a single rule to deal with potentially circular dependencies once and for all.

Is it really true that if you were to reorder the columns of an existing table it could take a really long time depending on the number of records? Does the program really have to fetch all the data and write it back before the user can work with the table again?

The physical order is mostly a chimera that does not actually exist, it all gets virtualized pretty fast in most databases. There is little reason to be dictating to the data source what the physical order should be - there are some niche uses for that with legacy data formats (to place fields that are unlikely to be read by some third-party software last so that fields before them are still read, or to match the exact physical order of an existing data set so that records can be binary-compared, things like that), but they are just that, niche. The data source (a MAP file, or, say, a SQL Server database) is in perfect position to decide how the data is stored best. What's best is pretty nuanced, it changes between software versions, etc.

With that, yes, changing the physical order involves going through the entire table and recomposing each record. This takes time proportional to the size of the table = potentially long. It's actually worse than that, because no data source offers commands to change the physical order of fields of an existing table, all you can do is just to try and get the physical order you want by creating a new table with the desired order, copying all data from the old table to the new one, then dropping the old table (and renaming the new table to old). And in the end the data source will just virtualize all that anyway and you might absolutely get the exact same storage layout as before, with just the fields reported in a different order.

In 99.9% of the cases, the physical order of fields should just be ignored. We have a logical order of fields in the Layers pane, let's just use that. If needs be, we might have multiple different logical orders, etc.

pslinder1
228 post(s)
#29-Jan-20 00:43

Thanks for the explanation Adam. I am fine with the layers pane but if the order is virtual anyway why not let people do it in the Schema window, especially if it looks like they should be able to? If you do keep it restricted to the layers pane, I still think the reordering capability should be added in the table windows by grabbing column headers and dragging them to where you want them.

What do you think of the magically appearing grayed out arrows when a new field is created?

tjhb
10,094 post(s)
#29-Jan-20 05:18

One exculpatory aspect to this thread is that the latest post is only accessible to the most deeply obsessed members.

And to Adam, who has written a notification widget. (That makes him not deeply obsessed.)

Dimitri


7,413 post(s)
#29-Jan-20 07:35

I am fine with the layers pane but if the order is virtual anyway why not let people do it in the Schema window, especially if it looks like they should be able to?

Two reasons: first, because the Schema dialog and the Layers pane do fundamentally different things, and second, because user interfaces in complex settings are easier to learn when they are modularized.

It's ultimately easier to learn a UI if there are not many special cases that must be kept in mind. It can be confusing when a dialog takes on some roles another dialog does, but then doesn't do other parts of it. That's especially true when the non-native roles it takes on conflict with core functions it is supposed to do.

The Layers pane is all about presentation. The Schema dialog is about structure, including structure that is programming-like in essence when computed panes and computed relationships (like indices) are involved.

The moving up/down controls in the Schema dialog when new fields are created are part of that programming interface. It's not an option that can be discarded, but a key part of how computed fields work. If you used arrows to duplicate the purely presentation ordering the Layers pane does, just what kind of UI complexity are you going to introduce to also provide the programming ordering that's essential for computing fields? For that matter, how are you going to do the ordering well like the Layers pane does, when the list includes things like constraints and indices that don't appear as columns in the table?

Those are real questions that have to be answered, and answering them is not so easy if the objective is to help educate someone who does not understand the difference between the Schema dialog and the Layers pane. It's not so easy to convey that "oh, this dialog is all about structure and that pane is all about presentation... except for when they are not, and except not for here, and here, and here..." Modularizing functionality within dialogs makes it easier to keep clear the differences. It's easier to learn.

What do you think of the magically appearing grayed out arrows when a new field is created?

Not sure what you mean. I just launched the Schema dialog and greyed out ("disabled") arrows are there when it launches and after you add a new field. They are only enabled when you select a new field. That's the way it should be: quiet cockpit until a control is needed.

By the way, the term "quiet cockpit" is sometimes misunderstood to mean "no controls that are not needed." That's not what quiet cockpit design is about, not in aircraft and not in software. In aircraft, the source of the term, it means that instruments don't blink or shine lights at you when they don't need attention. But it doesn't mean that they disappear. They're just disabled, as it were, when they don't need attention or are in play.

That's a difference that matters, because it costs cognitive energy to adapt to varying cockpit configurations. It's important for mental and muscle memory that the layout of the cockpit stays the same, so the pilot always knows the location of an instrument, can immediately put an eye or finger on it, and there's no confusion about what it is when it comes to life.

Why are "disabled" controls so popular? Because they're effective for the same reasons. It's not a lot of cognitive energy, but it is some to have to adapt to control layouts that shift all the time. Much better to leave controls in a familiar place and to enable them when they are in play. There's also a slight teaching element to it as well, where a totally new user can see "OK, there are some controls here but they don't apply in this situation. What are those about? Maybe it's time to read the Schema topic to find out..."

While we're on the subject of arrows, I think the best way for both beginners and more advanced users is to simply get rid of them. They're there as programming constructs, a way to order relationships for computed fields. But quite likely the graph relationships which a quick hint by way of ordering instantly solves could also be solved through graph theoretic analysis of references within computed field expressions, to ensure all graphs are disambiguated, that is, are not looped or self-referential (I believe the term is "acyclic", could be wrong).

If you don't need those hints, you don't need arrows at all for the programming task. Adios, arrows! And then both beginners who are still in the process of learning what different modules do and also those more advanced users who would like to just write expressions without considering precedence would have one less thing to understand.

Now, it could well be that there is no hermetic proof an acyclic solution can be computed for all possible referential knots. In that case, it might be necessary to pop open some warning dialog "Whoa, cowboy! Can't ride a horse that ain't there!" that reference is being made to a quantity that as yet does not exist. But you could argue that such failures to resolve references will likely be very rare, and sorting those out manually when they do happen will be quicker than thinking about precedence every time multiple computed fields are added at the same time.

But then you get to the bottom line question: people who write expressions in computed fields have already taken a few steps up the skill sets ladder, and they know to read relevant topics. They're not baffled by putting a computed field that uses references after the references it uses are created. For them, that's a nit.

But eliminating that nit is a non-trivial investment of resources that could better be used, I would suggest, on more important things of broader applicability. For example, I'd rather see a New Map dialog that automatically adds a frequently used image server base layer to the newly created map (coming in the next build). That's fun for the whole family. :-)

I'm all for anything that saves time and makes life easier. But I sure don't want any effort diverted away from more important priorities to tinkering with minor UI matters which function very effectively and are trivially easy to learn for those who have read the advice on how to use a dialog.

By the way, I'm sympathetic to why someone might see a list of fields in the Layers pane and also see a list of fields in the Schema pane and why they might want to do similar things in both. People aren't born with an understanding of what a schema is, after all, and it's not something they teach in elementary schools, so it's OK not to know what a schema is.

That's always going to be an issue that pops up with applications like Manifold, which are sold to and used by an extremely wide range of people, from folks who are still learning the difference between mouse and keyboard to those who are highly experienced masters of the IT universe and amuse themselves by conjuring puzzles and puns in spatial SQL.

That's OK. I think in general it's perfectly possible to modularize the system so people can use it for very simple things with less learning required. But I do think it is unwise to assume away any need for learning, and I also think that it is disrespectful to assume that less technical people are unable or unwilling to invest a bit of time into learning.

On the contrary, my personal experience is that less technical people often have an easier time getting to that magic "Oh, it's easy!" moment with new technology, because they don't know any better than to follow the advice they are given: read the introductory topics, and that will help you get a running start. If anything seems confusing, drill into the documentation and read the topics that cover what you found confusing. They'll actually do that, and like magic, problems melt away.

One last thing: if I were to tinker with Schema dialog controls, I'd get rid of arrows entirely and instead put sort options on the columns, allowing people to see them sorted by name, by type (field, etc), by data type and so on. :-)

pslinder1
228 post(s)
#30-Jan-20 01:18

As always well written and compelling.

and I also think that it is disrespectful to assume that less technical people are unable or unwilling to invest a bit of time into learning.

I have my doubts here. If it is absolutely critical of course they will do it. But a lot of cases are not like that. I think there is a potentially large market of people that need GIS occasionally. In my industry we absolutely need it but each engineer might only use it 10 to 15 times a year; and maybe only 1 or 2 times intensely (by intensely, I mean for more than a day at a time). These users are often in the position of being beginners for a long time because they use the program infrequently enough to not remember its quirks.

For these people intuitiveness is critical. You undoubtedly have your own market research but I suspect that this market is huge and probably under-served given the prices of Mapinfo and Arc (and the complexity of Q, GRASS and maybe Manifold).

Its not that I think that non techies or non GIS people do not have the capacity to figure it out. I just don't think they have the time.

adamw


10,447 post(s)
#29-Jan-20 07:42

Changing the order of fields in the Schema dialog since it is commonly virtualized anyway - this is probably not a good idea. We already have a place where we can control the logical order = the Layers pane. As I said, if needed, we may allow having more than one logical order / overrides / whatnot, these are all fine as UI features. But there should be a place where you can view, if not a physical, then a "natural" order, meaning the default order that the data source reports to clients - which is going to be used by default by clients other than Manifold. We can let Edit Schema reorder fields, sure, and we can write the order the fields should be shown in Edit Schema back into the data source. But applications other than Manifold won't care and will show fields in the order the data source returns them. There should be a place in Manifold to see that default order, and that's Edit Schema. It's a different thing that *altering* that default order is not terribly useful, but seeing it is important.

Reordering fields in a table using drag and drop - agree completely, we will add it.

Hiding toolbar buttons that move fields in the Edit Schema dialog until there are new fields that can be moved - maybe, we'll think about it. Very roughly -- according to the UI guidelines -- showing a command in a disabled state is better than not showing it, but as the number of command grows, not showing becomes better and eventually wins. We'll see.

tjhb
10,094 post(s)
#25-Jan-20 22:37

I often get the impression that Manifold takes RTFM as a point of pride. It really should not.

I understand what you mean here, but it is not quite so simple.

Part of Manifold DNA is that it is decidedly not a collection of wizards. There are no prepackaged "solutions"--to specific "problems" which the engineers have imagined you might come across in the course of your work.

It does not treat you like a baby, or even hold your hand. I think this is indeed a point of pride, quite rightly. It likewise makes me proud to be a Manifold user.

Manifold products (8 as much as 9) are designed so that the user, and only the user, is in charge of the data and the purpose to which the software and the data will be put.

If you want to use Manifold for standard GIS work, go ahead. If you want to use it as a base for synthetic protein folding, go ahead. Building a dictionary of subsequences in "junk" DNA? Why not. It's your data, and your imagination.

This level of freedom means that, to a greater degree than with products that do attempt hold your hand, you need to put in some effort to get to know it.

The product is not going to presume that it knows you.

pslinder1
228 post(s)
#26-Jan-20 04:28

I like the flexibility too but I disagree that making the interface more intuitive has to destroy the flexibility. The improved panning and zooming is a perfect example; it is intuitive and fast yet compromised none of the flexibility, power or capability of the program. I think there are a lot more big painless wins like that which Manifold can still reap before compromise is necessary.

And even at the point where they may have to compromise, a lot of those trade offs will not reduce the flexibility it will just make some things easier than others. Making all things equally difficult is not sign of virtue.

Dimitri


7,413 post(s)
#28-Jan-20 18:32

Making all things equally difficult is not sign of virtue.

Absolutely agree. We're lucky Manifold doesn't do that. That's why tasks which are brutally complex in other systems are often dramatically easier in Manifold. What can be done effortlessly in five minutes in Manifold, as in the Closest Rasters video can often be profoundly more complex and harder to learn in other systems.

The test of the ease of use of a system isn't whether someone who doesn't learn the basics finds it difficult. It is how well it performs year after year in the hands of people who use it as a working tool to quickly and effectively cut through difficult tasks.

Manifold's interfaces and design make it much easier to accomplish significant tasks, often reducing what is a complex process of setup and other intermediate steps in Arc to far simpler and more direct workflow. The tradeoff is more of a learning curve in the beginning, but the big payoff is that after that initial learning you get a lifetime of easier and more convenient workflow.

The goal is to make it effortless, smooth, and intuitive in the hands of people who have learned it. UI improvements which contribute to that are good, even if they require a bit more learning initially.

I disagree that making the interface more intuitive has to destroy the flexibility.

The key question is for whom is it more intuitive. What's "intuitive" for a beginner who won't read advice before launching into a something new is usually very different than what is "intuitive" for a person who has invested into learning the basics. Ideally you want to cover both, but if that's not possible you have to go for making easier what people will be doing day-in and day-out for years with a bit of knowledge under their belts.

That's also what PhotoShop does, building an interface that is superlatively effective in the hands of someone who has learned how to use it, and that's one reason why PhotoShop to this day is hands down the superstar of the graphics editing world, despite being totally incomprehensible for beginners who won't invest in learning before diving in.

One of the ways well-designed tools support the growth curve of users from rank beginner to seasoned master is by crafting orthogonal interfaces. An orthogonal interface is one where similar interfaces apply in different settings in the software, ideally scaling so the same interface can work for simple things and also for very sophisticated things. That's important because it greatly reduces what you have to learn as your work scales up in sophistication.

Non-orthogonal interfaces tend to build out different, special case interfaces for various settings, even if those interfaces really are doing similar things. Software often uses non-orthogonal interfaces, like special case wizards, to make it easier for beginners to do something. But such special cases are often a trap that limits growth.

Non-orthogonal interfaces are not so good for personal growth because as you get into more sophisticated uses you end up with too much complexity in how all those special cases interact. It all becomes much more difficult to learn and to use. Arc and Q are classic examples of too many non-orthogonal interfaces.

The Schema dialog is a remarkable example of UI orthogonality: that same dialog works exactly the same way for any table in any data source, even those as varied as a table in a linked shapefile, a table in the project, or a table in a full-up, enterprise class super-DBMS data source like Oracle. You can actually use that Schema dialog to modify table schemas in your Oracle database sitting way off over some VPN in your corporate headquarters. You only need to learn a few simple things, set forth in the Schema topic, to do wonders with any of those tables. It's not a lot of learning, but it is a bit, and that bit is well worth it to have knowledge that works in the beginning and grows to whatever level you want to go.

The commitment to building interfaces that can scale from beginning days all the way up to experienced sophistication is exactly a commitment to a more intuitive, more fluid, and easier to use interface. It's just acknowledging that you can get a far better and more effective user interface for the working life of most users if you invest into a bit of learning at the beginning.

If controls can be made orthogonal and effective in a working tool for people who know what they are doing, and *also* be made self-documenting for total beginners who won't learn the basics in a more orderly way, well, sure that's a great idea. Who would be against that? It's a great objective and sometimes, like Manifold's navigation controls, it can be done. There are more improvements like that in upcoming builds, which work both for total beginners and for users with experience as well.

Anyway, I'm just saying that don't be too quick to discount the value of tradeoffs that may require a bit more book learning during the early days of learning. They could be the key to much easier and quicker workflow in a working life that stretches for years after the initial learning days are long past. Master what's there, and then whatever you don't like suggest how you think it could be better. That's how the product improves for everyone.

joebocop
514 post(s)
#17-Jan-20 17:46

I've been meaning to send in this suggestion.

At present the majority of the transform templates require results output to a new component. This can be tedious when trying to edit features in-place. My opinion is that, where possible, the result of a transform should be to UPDATE existing features, with the optional ability to create a new component containing the resulting features instead.

Thanks!

adamw


10,447 post(s)
#18-Jan-20 12:12

Agree for images, we'll do that.

As regards drawings, we are already doing it - transforms that can write the result back into the table without losing any data unrelated to the transform have an option to Update Field, transforms that will lose data by doing so do not have that option.

An illustration might help: Convex Hull does not have an Update Field option, because the transform takes multiple records, computes a convex hull of all geometry, then produces a single record. If Update Field was an option, the original records would be removed and replaced with a single record for the convex hull, and there would be no easy way to get the original records back. Hence we only leave an option to Add Component.

If you have a particular transform or transforms in mind that currently do not do Update Field and you'd like them to - tell us what they are, maybe we missed some opportunities here.

We may allow transforms like Convex Hull to Add Record to an existing drawing. This would, however, produce drawings with mixes of objects playing different roles - such mixes are only navigable while they stay small.

Two different things that we are planning that will perhaps help here is (a) being able to specify the name of the resulting component right in the Transform pane, without going into an extra dialog, and (b) an option to add the resulting component as a layer into the map.

Having many components isn't bad if they are organized coherently.

joebocop
514 post(s)
#20-Jan-20 19:21

Thank you for this.

I was talking about vectors, but reading your description makes sense. I had a digitizing contract a few months back where I was dearly missing the "split with" transform from 8, where existing features can be in-place split into arbitrary shapes using other geometries.

Thank you.

Dimitri


7,413 post(s)
#22-Jan-20 08:21

I was dearly missing the "split with" transform from 8, where existing features can be in-place split into arbitrary shapes using other geometries.

Splitting objects with other objects is in 9. For instant relief, learn about and use topology overlays.

joebocop
514 post(s)
#22-Jan-20 16:40

Dimitri that's exactly my point; none of the topology overlays presents the option to edit-in-place.

Dimitri


7,413 post(s)
#22-Jan-20 16:47

In your particular use case, why is "in place" a big need? Why not just use the new drawing that is created?

After all, splits using one drawing to cut objects in a different drawing can create millions of new objects... isn't it less risky to just say, "OK, that's a new drawing... glad I have the old one to fall back to in case of need..."?

dchall8
1,008 post(s)
#22-Jan-20 19:16

I have a layer with 40,000 parcels and want to split one parcel from say, 360 acres, to one that is 300 and one that is 60. You're saying it's better to create a new drawing with 40,001 parcels than to just split the one and update the old drawing?? Normally I would do this transform 10 times a day. It seems like creating 10 new drawings is creating unnecessary clutter in the project pane.

This was my process: I had the 40,000-parcel layer and a separate layer I called Cutting Tools. I would take the description of the cut to be made and draw it into the Cutting Tools layer. Usually that was one line. Then select the parcel and select the pertinent cutting tool and use the transform toolbar to make the split. If there was a mistake, I would Undo the transform and start over.

Dimitri


7,413 post(s)
#23-Jan-20 06:59

You're saying it's better to create a new drawing with 40,001 parcels than to just split the one and update the old drawing??

No, not at all. I meant in cases where many objects in one layer are being overlaid/modified by many objects in another layer.

If there's only one object, or just a few, you can use slightly different workflow. For example, select the object to be cut, and then do the overlay with the Restrict to selection box checked. You then create a results drawing with just two objects. Those can be copied and pasted into your original drawing (easy to delete the one selected drawing first).

I use the Restrict to selection checkbox all the time when editing drawings with transforms. For example, if I want to move just some objects a precise distance I select those I want to move and then I use Shift, with that "selected only" box checked.

I definitely agree it's more convenient when editing just one object to do it in place. For that I'd recommend a split tool that's an editing tool. Using a transform works, of course, but I'd prefer to have an editing tool.

ColinD

2,081 post(s)
#18-Jan-20 05:21

That HMS list is a close mirror of my own needs. Here are a few others that I regularly use.

11. Instant Data;

12. Shared Edit;

13. In the context of 12, to be able to add snapped coordinates to both borders of adjoining polygons (this one's been around for a while);

14. GPS Console;

15. COGO tools;

16. Side docked Panes which was one of the most useful additions to M8;

17. A bit more on layouts for printed maps. Attached is an example created in Arc something (not by me). This is a vegetation community map that I had prepared using 11, 12, and 14 plus a lot of Split With. Note the use of different font sizes, superscripts, callouts, text within color swatches, grid coordinates. The ability to format an individual word in a description is needed, particularly for italicizing scientific botanical names.

Attachments:
ExampleLayout.jpg


Aussie Nature Shots

adamw


10,447 post(s)
#18-Jan-20 12:36

Thanks!

Two questions:

For 11 (Instant Data) - you can currently specify field values for a new object using the Record pane. Is there anything we should add here?

For 15 (COGO tools) - we recently extended our vector editing tools and as part of that we added support for traverses. While we aren't done there yet and are working on features for both traverses and for vector editing in general, it would help to know which specific COGO features that we currently don't have you'd like to see the most.

Thanks again, we love these lists, they are wonderful.

joebocop
514 post(s)
#20-Jan-20 17:10

11 - Add the ability to auto-default to the previously-entered value, and add the ability to present a drop-down (or listbox) of constrained values to choose from.

ColinD

2,081 post(s)
#21-Jan-20 00:14

I think that the Record pane would do the job, just need to get accustomed to the different arrangement. A particular use for instant data is when driving around a large area identifying several hundred tree species using GPS Console with a hires aerial photo in the project. Placing a point object on the target tree and entering the species name. It does appear that subsequent points retain the data in the last fields entered.

I'll explore traverses, but it sounds like it should do what is needed.

18. Go To (selection) which I have an Addin for in M8 and use frequently. This is because I do a lot of spatial data exploration in the process of compiling reports as well as finding small area fragments that need editing in some way. So clicking on a table entry with no data and finding where the object is and its context.

Thanks Adam.


Aussie Nature Shots

Dimitri


7,413 post(s)
#21-Jan-20 05:49

Go To (selection)

Great suggestions! For the above, you can do that now: right click on the layer tab where there's a selection and choose Zoom to Selection.

StanNWT
196 post(s)
#28-Feb-20 21:20

I tried running this on a very large DEM and resulting slope data set with Manifold 9.0.170.5

The DEM and resulting slope data set is 2,306,560 records at a 128x128 tile size, image size is 327,603 x 115,217 pixels. It's the big DEM that Tim is familiar with that I sent him a while back, he may still have it, I'm not sure. In any event, I have a set of points that I tried to process using the "Closest Slope Path Distance" transform. Even though the last iteration took 3555 seconds, the resulting image has 0 records.

When generating the slope image, I used a distance of 5. When generating the closest path distance I used the defaults, with the Wells drawing as the point source.

The three attached images shows the contents of each image component, specifically the image tile size and number of records.

Is there a DEM and thus slope size that is too big to work on?

Attachments:
Closest_Path_Manifold_Issues_USGS_DEM_Feb28_2020.jpg
Closest_Path_Manifold_Issues_USGS_DEM_Slope_Closest_Path_Feb28_2020.jpg
Closest_Path_Manifold_Issues_USGS_DEM_Slope_Feb28_2020.jpg

adamw


10,447 post(s)
#29-Feb-20 11:04

It's strange that there's no result and there's no error report either.

The distance transforms / functions put no additional limits onto the image size, anything that fits into a MAP file is fine. I guess what might have happened is that the transform needed a lot of temp space to store intermediate results (raster transforms frequently store intermediate data for each pixel, for a ~320k x 120k pixels some transforms might use 600 GB of temp space or so) and it might not have been available. If temp space is not an issue, I guess there's no escaping performing a test run with the image that you have - we'll try with a synthesized image of the same size first.

Manifold User Community Use Agreement Copyright (C) 2007-2021 Manifold Software Limited. All rights reserved.