Subscribe to this thread
Home - General / All posts - Deep cancellation
tjhb
10,094 post(s)
#09-Nov-18 23:10

One thing that drives me nuts in Manifold 9, is that it often takes just as long to cancel an operation as to wait for it to complete. This is something we have discussed before (here and here).

If I write a statement, execute it, then recognise that it is inefficient, I can press Cancel ... ... but then I am stuck. I have to find something else to do until Manifold has finished "cancelling" the operation. It may take half an hour (or longer if I have done a really bad job of the query). The "cancellation" is only carried out after the task completes--even to the point of uselessly writing the (first 50000 records of) the result table. Infuriating!

So, test with less data? Well yes. But it is very inconvenient for a simple mistake to make such a huge dent in productivity. I hope that deep cancellation might be implemented soon, everywhere.

Dimitri


7,413 post(s)
#10-Nov-18 15:58

Something to send in as a suggestion, along with where you see the priority.

Quick cancellation is at times (not always, but more often than you might think) bound closely to speed. Consider the classic case, of recursion... how often do you want the work to surface from deep recursion to check if a cancellation is required? Do it often enough for quick cancellation and you eliminate the speed benefit of the technique.

There are also costs to cancellation in cases where large temp files are constructed. You can see an analogous situation when cancelling big copies in Windows: it takes the system time to delete what has been created to clean up once a cancel has been commanded. If gigabytes are involved, it could take more time to cancel than to continue to the finish. It's like flying from London to New York and then commanding a "cancel" of the flight when on final approach to the destination airport... turn around and fly back to London?

It sounds like with queries you are encountering the first effect more than the second. No doubt there is room for improvement. But at what priority over other things?

Given a mention in the forum is opening the door to opinions, I'd rank it as one of those "nice to do on an ongoing basis" things that, in specific cases where many people want effort would rank a higher priority, but otherwise not something to take significant time away from, say, interactive editing, filling GUI holes, raster editing and similar.

By the way, taking a totally primitive approach to this... I always make backup copies and then if I want to stop doing something I have the option of using Task Manager to do a "super cancel." :-) Brutal, perhaps, but effective when I have no patience.

adamw


10,447 post(s)
#12-Nov-18 10:02

Canceling and progress tracking are implemented individually by each operation. If a particular operation is not very responsive to cancel, it is usually not too difficult to make it more responsive, but we have to know what that operation is - and sometimes how it is being used, ie, what kind of arguments are being passed to it (lots of small objects or a few large objects, that kind of thing).

The posts you link are talking about topology overlays - fair enough. The reason we didn't spend much time on making them responsive to cancel is that we are planning to rework them. But maybe it makes sense to make the old operations responsive to cancel until we have new ones.

What other operations do not cancel well?

tjhb
10,094 post(s)
#12-Nov-18 20:51

What other operations do not cancel well?

I don't have a great record of operations for which I have had to "super cancel" (in Dimitri's words). I sometimes take a screenshot (the best I can do without a debug build?). I'll try to keep better records.

Here's one example screenshot. (The part greyed out is non-working draft code and not important.) From memory this took more than an hour to (not) cancel--I had to kill the process. (I can test this again, and supply data if that helps.)

The cause is always my dumb coding decisions. Asking too much of the engine, or asking a question in the wrong way. But these are usually simple mistakes--and ideally, learning experiences. They should never ruin someone's morning.

Likewise, they shouldn't sink a software demo. An interminable "Cancelling operation" dialog is a marketing issue. Manifold 9 should never seem to hang the system.

Last point: Cancel should mean Cancel (now). If quick cancellation is not possible (say within 30s), please grey the Cancel button out. (Can the software not judge this?) That would be somewhat useful.

In response to Dimitri, I would rank this before any new features. It really matters.

Attachments:
example.png

tjhb
10,094 post(s)
#13-Nov-18 01:46

I did try the same operation again. (Same as in the screenshot above.)

It has now been "cancelling" for 3 hours 15 minutes. I'll softly kill it.

adamw


10,447 post(s)
#13-Nov-18 06:33

Thanks, that helps. We'll try to make GeomClip more responsive to cancel (and will look at similar operations).

tjhb
10,094 post(s)
#13-Nov-18 07:02

Thank you Adam! If I find more examples (that seem to matter) I'll take a screenshot again and be as specific as possible.

For good measure I will also post some code that works brilliantly fast, as a worked example. I'll do that tomorrow.

gjsa100 post(s)
#15-Nov-18 10:05

And if possible, the same for Normalize Topology until that is reworked.

I just started a Normalize and almost immediately regretted it. I really can't wait 3,4 or 5 hours for this to cancel and I can't super-cancel because I didn't save the project before hand :(

This is without a doubt the biggest drawback to M9 presently: an un-optimized process is stuck cancelling and the rest of the app is locked out, including fundamental operations like save.

Terminating the app has always worked when I have recently saved a project file and have nothing to lose. I have never corrupted a project file doing it (although that doesn't mean its not possible, of course.) So this does make me wonder, is it really that difficult to implement a brute force cancel?

The funny side to this is that in 2018, the GIS world is reminiscent of 1980s photocopiers which loved to pump out reams of paper despite all manner of cancelling requests until someone pulled the plug.

lionel

995 post(s)
#24-Nov-18 14:19

Hi

why not an implicit save map content before launch any script sql ?


Book about Science , cosmological model , Interstellar travels

Boyle surface fr ,en

gjsa100 post(s)
#15-Jan-19 23:38

Great idea Lionel! Or at least an option in the Preferences to have that turned on. For me, that would allow doing a supercancel until such time as the core engine allows for immediately cancelling single core/sequential functions.

Saving/updating queries and scripts is probably more important to me than updating layers in a m9 project. If we had access to the text in the command window while a command was running any unsaved edits could be copied and pasted elsewhere, but alas, "Running Command" is a dependent process in Windows and the entire M9 GUI is locked until it completes.

Manifold User Community Use Agreement Copyright (C) 2007-2021 Manifold Software Limited. All rights reserved.