Subscribe to this thread
Home - General / All posts - manifold 8 on win 10
brewer25 post(s)
#05-Mar-18 16:37

anyone running manifold 8 32 bit on win10? I got a new computer and cant afford to upgrade version of manifold

Dimitri

5,119 post(s)
online
#05-Mar-18 19:12

I run Release 8 on Windows 10 with no problems.

hgmonaro45 post(s)
#06-Mar-18 00:37

yep

mikedufty

809 post(s)
#07-Mar-18 03:07

I'm running it. Subjectively it seems much more crash prone than on Windows 7. I can't quantify it, and it might be the new computer rather than the change in OS, but have lost quite a bit of work since upgrading. Usually freezing while refreshing linked ecw on network drives.

tjhb

8,410 post(s)
#07-Mar-18 04:21

Mike,

(a) Is that specifically, or only, for 32-bit Manifold 8?

(b) Have you had any problems *at all*, when using data on a local physical drive?

mikedufty

809 post(s)
#07-Mar-18 05:02

32 bit is what I have installed. I haven't tested it with 64 bit.

All our file are on network drives, I only occasionally save locally to work offline, so I don't think I have had any issues with local files, but haven't tried it enough to come to any conclusion. People trying to work over a VPN have had the same issue (and have been instructed not to work on Manifold files over the VPN).

It is likely primarily a network issue, but Manifold is not handling it well. It is possible it has nothing to do with Windows 10, but the timing seems to match.

tjhb

8,410 post(s)
#07-Mar-18 05:12

There it is.

(a)

32 bit is what I have installed. I haven't tested it with 64 bit.

That's nuts.

(b)

All our file are on network drives, I only occasionally save locally to work offline

That's nuts too.

You should sack whoever suggested you should work that way, then switch to 64-bit Manifold and local data.

It is not worth anyone's hourly rate to work like that.

mikedufty

809 post(s)
#13-Mar-18 14:09

That's nuts too.

You should sack whoever suggested you should work that way, then switch to 64-bit Manifold and local data.

It is not worth anyone's hourly rate to work like that.

We have a couple of 64 bit licences but was not able to detect any difference other than not talking to excel. Mostly we are just making maps for reports, no heavy number crunching.

How would you manage everyone having their own local copies of data? It's hard enough keeping one copy documented and backed up.

( it is me that I should sack by the way)

Dimitri

5,119 post(s)
online
#13-Mar-18 15:16

How would you manage everyone having their own local copies of data?

I 100% agree that is a serious issue and a good reason to keep standard files in a central location. The question is how to work with them in a way that does not expose the data to the fragility of networks.

Keeping data in ECW, as adamw has noted, imposes some limitations in that if you must use the ECW drivers to work with it you are at the mercy of what those drivers do when the network lapses. In that case, one decision, to use ECW, might conflict with another decision, to centralize data to avoid version skew from uncontrolled local copies.

A possible solution if all of the software you use is compatible with it is to centralize using a DBMS and have things like Manifold connect to the data on that DBMS. That's usually much more reliable over networks.

We are working hard to provide solutions for what you describe, but they won't be here next week or next month. It is also not so easy to figure out a centralized solution that is very fast and does not put you into a Manifold-only silo. But we have some ideas.

mikedufty

809 post(s)
#14-Mar-18 01:12

Anyway, my main point is that this model, linked ECWs on a network drive, seems to have worked great for many years with Windows 7, but is iffy with Windows 10, so something for people to watch out for, which was the original question here.

It may not be an issue for anyone else, but that is my experience. I used to enjoy telling new staff how great it was that Manifold almost never crashes, but can't do that anymore.

Dimitri

5,119 post(s)
online
#14-Mar-18 09:49

can't do that anymore.

Well, perhaps it would be more accurate to say that if once you could tell new staff your network never fails, you can't do that any more. :-)

That's not a totally humorous statement, either, since, after all, if somebody pulled the power plug on your computer and thus crashed Manifold, you would not take that as a reason for not being able to say any more that "Manifold never crashes."

The problem here, it seems, is the inherent risk of running software over networks that have flaws. One approach to reducing such risks is to increase network reliability.

this model, linked ECWs on a network drive, seems to have worked great for many years with Windows 7, but is iffy with Windows 10,

I hear you, but a connection to Windows 10 may not be right either. Network problems can be extremely difficult to identify. It might well be that some of your hardware has started misbehaving in a highly transient way, and those problems became noticed about when you switched to 10.

We had a problem in one of our facilities with an occasionally unreliable network. Massive amounts of effort were spent to diagnose the issues, with no progress. In "trying everything" mode, we swapped out an older generation of switches/hubs with a newer, higher quality generation and the problems instantly went away. As far as we know, nothing else changed, so it seems likely that one of those older switches was misbehaving.

You could say that "well, OK, maybe my network is imperfect, but the software should be hardened against that." A fair point, of course, but there too it is an open question what effort software reasonably should make to fend off problems in the network, whether that is cost-effective, and whether that is even possible.

As adamw noted below, when you get crashes with ECW those crashes are coming out of the ERDAS ECW SDK. If those crashes are caused by a network issue that shoots the ECW code in the head, while I grant you might ask that the ECW be hardened against such network issues, that is not something Manifold can add to ERDAS. I don't call it a "fix" because hardening against system failures is not a flaw in ERDAS's product.

Manifold has never said that the third party stuff in Manifold doesn't crash. Manifold has always been careful to say that if you use Manifold and stick to Manifold's native features, it is almost impossible to crash the system. That's because Manifold cannot control what is in the third party code, immediately fixing any problems.

ECW is a great format and the ERDAS SDK code is very good too, but in this case it might be vulnerable to network defects. I don't want to say more than "might" because without taking an analyzer to your network it is difficult to characterize what the issues are with the network, and whether it is reasonable to expect any software can defend against them.

Depending on the issues in your network it is also not clear that even if you used Manifold .map format to store your images, linking those .map files to whatever is your active project, that Manifold would be sufficiently hardened against whatever problems seem to be crashing the ERDAS code. We just don't know.

Like adamw recommends, I wouldn't count on perfection in networks. But if you are using a network, for more reliable network operation, you might get better results quicker by focusing on the reliability of the network itself instead of on the software that is using the network. Small measures, like a few hardware upgrades, sometimes help a lot.

mikedufty

809 post(s)
#15-Mar-18 03:19

Yes, it is certainly not a problem with Manifold, as the software hasn't changed. But the reality is we can only use it on our network. All we can do is try to make the network as good as possible and save more often.

Attempting to stick with Windows 7 is becoming quite difficult. I thought I would revert to Windows 7 last time I upgraded my home computer and had to do all sorts of workarounds and hacks to get it working (not officially supported on current gen intel processors) and no usb3 support for installing.

tjhb

8,410 post(s)
#15-Mar-18 04:57

I have hesitated to say anything about network reliability for the good reason that I know nothing about how to make a corporate network reliable.

But are you in a position to use--or at least to trial--a wifi network, rather than fixed wire ethernet?

As I understand it (see first para), wifi is now considerably more resilient than ethernet--because it has to be, since signals bounce all over the place. Resilience is what you are looking for I think.

And good wifi is now cheaper than fixed wire.

Could it be worth a try?

adamw


8,259 post(s)
#15-Mar-18 07:10

This might not help if the issues are on the level higher than the driver (ie, the operating system itself resetting the network periodically - ironically, to try and make sure none of the involved services / components gets stuck after a fatal issue for too long and the network auto-repairs after some time).

It might still be worth trying, if it's not too difficult to try.

mikedufty

809 post(s)
#16-Mar-18 04:20

Currently it works really badly over wifi, but that is probably because our wifi setup is very basic. Occasionally someone claims manifold has been running really badly and we trace it to the fact their network cable has come unplugged and they haven't noticed because they have wifi on.

I had moved some of the bigger ecw files to a network drive that is a step removed from the main server and I think that is part of the problem, so am trialling moving all the frequently used files back to the main server. Hard to see why that makes it worse on Windows 10 than Windows 7, but I suspect there is a combination of things going on.

Dimitri

5,119 post(s)
online
#07-Mar-18 09:14

It is likely primarily a network issue, but Manifold is not handling it well. It is possible it has nothing to do with Windows 10, but the timing seems to match.

This raises a few very important technical issues that need to be kept clear when working with networks. So, with no intent other than technical clarity...

1. There are limits to what an application can "handle" in areas the operating system manages. Otherwise, the operating system would have failed at its most fundamental job of abstracting system resources so applications can run without leaking outside their sandboxes. In key ways, Manifold has to trust that Windows functions correctly. There is no "oh, let Manifold take over in cases where Windows gets it wrong" option.

2. Part of an operating system's job is handling system level resources like networking resources. Windows cannot do that perfectly in cases where networking resources are uncontrollable because they are totally outside of Windows. Windows also has to trust that network resources will function correctly.

3. Neither the operating system nor an application can "handle" destructive events in hardware or third party software they do not control. Unplug a physical Ethernet cable between two computers, one of which hosts the network drive being used by the other, and there is no way for Windows or the application to plug that cable back in. Having an unreliable link between you and your network data store is an example as well.

When a sophisticated application uses data it is like the brain being connected to the retina in eyeballs via the optic nerve. That is such a tight and close connection many neurologists consider the retina an extension of the brain. For sight to work as it does the connection has to be very tight, and the brain has to be able to count on that optic nerve and retina to function.

A "destructive event" might be taking a knife to the optic nerve to cut it. No more sight. Trying to design a brain/retina system that can "handle" such a destructive event would not produce the fast and elegant vision system humans now enjoy. There are ways around that, like very many, compound eyes, but they do not have the features and benefits of the vision system we now have.

When you save your data on a networked device, it is as if you took the eyeballs out of your eye sockets, stretched out the optic nerves, and started playing ping-pong with them. No surprise if you get more "destructive events" that way.

There is not much Windows or any other operating system can do to guard against destructive events in network connections, or, at least not much that can be done which will not dramatically reduce performance when lots of data is involved.

A further effect is that what can be done to increase network reliability is very expensive compared to the basically consumer-level, non-fault tolerant systems that everyone uses. Are you using weapons-grade, embedded control networks like those that are used to connect the many fly-by-wire computers within a modern fighter aircraft? Probably not.

You're probably connecting over garden-variety Ethernet, TCP/IP local area and wide area networks, wiring up systems that are running the usual hodge-podge of Windows and Linux systems, all operating on motherboards that use network chips built out in some Chinese fab where the drivers were written by the lowest bid group of third world programmers. In most cases, I/O chip vendors have never met the people who do work for them and often they don't even know who they are.

At the same time the motherboard runs a BIOS and disk drivers that, under massive pressure to be politically correct, are wired up to automatically shut down both disk access and network function to save power, of course at whatever is considered a good time by some other, anonymous group of programmers, here today and gone tomorrow in whatever Eastern European or Asian country they live. Add to that all of the fun you get with routers, where router hardware vendors toss together outsourced hardware with whatever layers their one programmer who reads quasi-English can find on the web to get their router functional, without having the slightest idea who wrote those layers or how they function. Stir and mix well with endless updates and revisions.

That all that works together at all, given zillions of different motherboards, network chips, bridge chipsets, I/O chipsets, disk controller chipsets, drivers, operating systems, network layers, routers and switches and all of their layers is amazing. But expecting it to work perfectly for every byte over trillions of bytes all the time, well, that's expecting perfection in a system where perfection does not exist.

The bottom line is that if you want to run complex applications with storage outsourced to a network resource you usually end up trading off cost, performance and reliability. To get higher reliability you often have to spend more and/or accept lower performance. No surprises there.

I don't recommend trusting imperfect networks, but if you must, at least invest in super-high quality components every step of the way. That can be very expensive, as you spend more money everywhere. Highly fault tolerant servers for storage are significantly more expensive than ordinary gear most people use, and highly reliable switches, routers and network wiring also tend to be much more expensive. The top end of Cisco's product line, for example, is way more expensive than buying the cheapest D-Link stuff you can find on Amazon. When you connect over the web, like through VPN, you have no control over the quality of many unknown intermediaries.

There are ways of throwing money at that problem as well, to create fault tolerant pipes even through public networks, but such methods often have such performance-reducing side effects, in addition to their high cost, that you might not want to use them. For a fraction of the cost you can get a terabyte SSD and run faster anyway.

adamw


8,259 post(s)
#07-Mar-18 12:10

Mike, working *with files* over the network is indeed very dangerous. It could also explain why you are getting more issues under Windows 10 than under Windows 7 - the drivers have changed.

Working over the network is a recurring topic, primarily because there is no easy way for us to prevent damage from it. There is a (very brief) explanation of the issues here, for example. If you can avoid working with files over the network, please do so. If you need a central storage, use a database. Applications that have no issues working with files over the network are mostly just copying the data over locally and write it back during a save overwriting the entire file. We cannot do that, even in Manifold 8, because our data is big. (Or, rather, we can, but then we will just have a new problem: opens are slow / require a lot of free disk space.)

mikedufty

809 post(s)
#12-Mar-18 05:06

It seems to be linked ECWs that are the problem. Can that functionality be implemented with a database? When I have added linked ECWs to our server console setup on postgres it seems to just save a link there, not the data.

It is not really practical for everyone in the office to have their own copy of all the aerial imagery. Generally linked ecws from a network seem to have worked really well in Manifold previously.

adamw


8,259 post(s)
#13-Mar-18 09:18

You can set up a web server to serve image data.

There are multiple options: the most direct one is a server that produces ECWP (ERDAS Apollo, etc), this can be used directly in both 8 or 9, instead of providing a path to the ECW image, you provide a URL. Another option is to use WMS or something similar. 8 can both serve and read WMS. There are open source products that can serve WMS. 9 can read WMS (does this way better than 8), but cannot yet serve it.

In terms of solving network issues directly, in the case of ECW this is especially difficult for us, because we access ECW files using ERDAS ECW/JP2 SDK. We *can* patch the older version of the SDK used by 8, but it does not support newer versions of ECW files, so that's of limited use.

In general, I suppose we can add an option to both 9 and 8 to copy files accessed over the network locally into a temporary location. Copying will obviously take time and disk space, however.

KlausDE

6,247 post(s)
#12-Mar-18 06:32

ECWs are read-only resources in the described scenarios and it's typical, that the static background is stored centrally in a company.

Shouldn't it be technically possible to automatically reestablish a connection that Windows or an interrupted network connection broke? There is no danger of data corruption with read-only resources. And for Mfd9 I think of all types of read-only data including map-files. In the worst case just refetch the hole object.

It's simply no option to have all the background stuff stored on every workstation in a company. It's not only disk space. It's about data integrity in a work group which is essential.

So a read-only branch of all dataports should be more failure tolerant.

adamw


8,259 post(s)
#13-Mar-18 09:26

It is true that read-only connections don't risk losing persistent data, but this isn't about losing data, this is about the code failing to recover after a network issue and crashing the process. There are complications specific to ECW in that the code that does this is in the SDK which isn't ours, and in the case of 9 in particular, we cannot modify it. Maybe we can do something for 8, we'll take a look, but this is a long shot.

(Added: actually, maybe there is a way to try to tackle it for both 8 and 9. We'll investigate.)

Manifold User Community Use Agreement Copyright (C) 2007-2017 Manifold Software Limited. All rights reserved.