Subscribe to this thread
Home - General / All posts - Deep cancellation
tjhb

8,410 post(s)
online
#09-Nov-18 23:10

One thing that drives me nuts in Manifold 9, is that it often takes just as long to cancel an operation as to wait for it to complete. This is something we have discussed before (here and here).

If I write a statement, execute it, then recognise that it is inefficient, I can press Cancel ... ... but then I am stuck. I have to find something else to do until Manifold has finished "cancelling" the operation. It may take half an hour (or longer if I have done a really bad job of the query). The "cancellation" is only carried out after the task completes--even to the point of uselessly writing the (first 50000 records of) the result table. Infuriating!

So, test with less data? Well yes. But it is very inconvenient for a simple mistake to make such a huge dent in productivity. I hope that deep cancellation might be implemented soon, everywhere.

Dimitri

5,119 post(s)
#10-Nov-18 15:58

Something to send in as a suggestion, along with where you see the priority.

Quick cancellation is at times (not always, but more often than you might think) bound closely to speed. Consider the classic case, of recursion... how often do you want the work to surface from deep recursion to check if a cancellation is required? Do it often enough for quick cancellation and you eliminate the speed benefit of the technique.

There are also costs to cancellation in cases where large temp files are constructed. You can see an analogous situation when cancelling big copies in Windows: it takes the system time to delete what has been created to clean up once a cancel has been commanded. If gigabytes are involved, it could take more time to cancel than to continue to the finish. It's like flying from London to New York and then commanding a "cancel" of the flight when on final approach to the destination airport... turn around and fly back to London?

It sounds like with queries you are encountering the first effect more than the second. No doubt there is room for improvement. But at what priority over other things?

Given a mention in the forum is opening the door to opinions, I'd rank it as one of those "nice to do on an ongoing basis" things that, in specific cases where many people want effort would rank a higher priority, but otherwise not something to take significant time away from, say, interactive editing, filling GUI holes, raster editing and similar.

By the way, taking a totally primitive approach to this... I always make backup copies and then if I want to stop doing something I have the option of using Task Manager to do a "super cancel." :-) Brutal, perhaps, but effective when I have no patience.

adamw


8,259 post(s)
#12-Nov-18 10:02

Canceling and progress tracking are implemented individually by each operation. If a particular operation is not very responsive to cancel, it is usually not too difficult to make it more responsive, but we have to know what that operation is - and sometimes how it is being used, ie, what kind of arguments are being passed to it (lots of small objects or a few large objects, that kind of thing).

The posts you link are talking about topology overlays - fair enough. The reason we didn't spend much time on making them responsive to cancel is that we are planning to rework them. But maybe it makes sense to make the old operations responsive to cancel until we have new ones.

What other operations do not cancel well?

tjhb

8,410 post(s)
online
#12-Nov-18 20:51

What other operations do not cancel well?

I don't have a great record of operations for which I have had to "super cancel" (in Dimitri's words). I sometimes take a screenshot (the best I can do without a debug build?). I'll try to keep better records.

Here's one example screenshot. (The part greyed out is non-working draft code and not important.) From memory this took more than an hour to (not) cancel--I had to kill the process. (I can test this again, and supply data if that helps.)

The cause is always my dumb coding decisions. Asking too much of the engine, or asking a question in the wrong way. But these are usually simple mistakes--and ideally, learning experiences. They should never ruin someone's morning.

Likewise, they shouldn't sink a software demo. An interminable "Cancelling operation" dialog is a marketing issue. Manifold 9 should never seem to hang the system.

Last point: Cancel should mean Cancel (now). If quick cancellation is not possible (say within 30s), please grey the Cancel button out. (Can the software not judge this?) That would be somewhat useful.

In response to Dimitri, I would rank this before any new features. It really matters.

Attachments:
example.png

tjhb

8,410 post(s)
online
#13-Nov-18 01:46

I did try the same operation again. (Same as in the screenshot above.)

It has now been "cancelling" for 3 hours 15 minutes. I'll softly kill it.

adamw


8,259 post(s)
#13-Nov-18 06:33

Thanks, that helps. We'll try to make GeomClip more responsive to cancel (and will look at similar operations).

tjhb

8,410 post(s)
online
#13-Nov-18 07:02

Thank you Adam! If I find more examples (that seem to matter) I'll take a screenshot again and be as specific as possible.

For good measure I will also post some code that works brilliantly fast, as a worked example. I'll do that tomorrow.

gjsa42 post(s)
#15-Nov-18 10:05

And if possible, the same for Normalize Topology until that is reworked.

I just started a Normalize and almost immediately regretted it. I really can't wait 3,4 or 5 hours for this to cancel and I can't super-cancel because I didn't save the project before hand :(

This is without a doubt the biggest drawback to M9 presently: an un-optimized process is stuck cancelling and the rest of the app is locked out, including fundamental operations like save.

Terminating the app has always worked when I have recently saved a project file and have nothing to lose. I have never corrupted a project file doing it (although that doesn't mean its not possible, of course.) So this does make me wonder, is it really that difficult to implement a brute force cancel?

The funny side to this is that in 2018, the GIS world is reminiscent of 1980s photocopiers which loved to pump out reams of paper despite all manner of cancelling requests until someone pulled the plug.

lionel

519 post(s)
#24-Nov-18 14:19

Hi

why not an implicit save map content before launch any script sql ?


join image "Because my dad promised me" interstellar from Manifold: Time by Stephen Baxter. power Math destruction

Manifold User Community Use Agreement Copyright (C) 2007-2017 Manifold Software Limited. All rights reserved.