Subscribe to this thread
Home - General / All posts - Need help for importing ESRI geodatabase
orerockon

392 post(s)
#21-Aug-10 08:29

Hello, I need help importing the attached ESRI geodatabase (it may be a personal geodatabase but it does not contain any .mdb files). From reading the help topics in Professional 8.00 it seems that this may only be possible in 8.00 Enterprise edition. If so, can someone do this using Enterprise and export to map files? I do have the means to pay for consulting services.

If not, then can I use Professional 8.00 to do this for me? The help topics relating to ESRI geodatabases are pretty much beyond my level of understanding.

I hope this is an appropriate post for this forum; if not, then please delete it.

Attachments:
BLM or_wa_aquatic_restoration projects.zip

adamw


10,447 post(s)
#21-Aug-10 08:38

That's a 'file geodatabase', not 'personal geodatabase'. Manifold does not support file geodatabases.

joebocop
514 post(s)
#21-Aug-10 16:02

This may be the wrong forum for this type of question, but am I alone in thinking that the ability to read ESRI's "file geodatabase" format (alive since v 9.2) is a desirable one for the next Manifold 8x update?

Are there a series of really good reasons of which I'm not aware that would make supporting file geodatabases (read-only at least) a bad idea, or not possible even, for Manifold GIS?

Thanks for your insight.

Joe

danb

2,064 post(s)
#21-Aug-10 16:26

I think it is because it is still a closed format. The only thing that I know which can work with the file geodatabase outside ESRI is Safe Software's FME. As I understand however, this requires an Arc licence to be installed on the same machine for it to work.


Landsystems Ltd ... Know your land | www.landsystems.co.nz

joebocop
514 post(s)
#21-Aug-10 16:52

So closed, I'm guessing, means that the fine folks at Manifold would have to pay a royalty of some sorts to ESRI in order to include this functionality?

Data held hostage is a real pain, eh!

Joe

tjhb
10,094 post(s)
#21-Aug-10 18:56

So closed that there is no SDK available, at any price.

Some software developers nevertheless manage to enable import from file geodatabase format, somewhat obliquely, by making calls to ESRI's own .dlls—provided that an ESRI product that includes them is already installed on the user's system.

An instance of ArcGIS Explorer is enough. Which is a free product.

This is how Safe Software (FME) and Avenza (MAPublisher) do it. Whether they have to pay ESRI a license fee for hooking into the ESRI .dlls I don't know.

Manifold could do the same, in theory. But the problem really is with ESRI. They've made a "principled" decision to pitch a proprietary database format as an industry standard, and many large organisations have fallen for it. Not without some good reasons. File geodatabase storage does have good things going for it, in particular being able to define customised topology rules, and enforce them over all of an organisation's spatial data.

But the "principle" behind the format looks very like greed. That's why I doubt very much that Manifold will bend over backwards to enable access to their file geodatabases. ("You want me to say 'please' to that? You're kidding.")

What's more likely is that Manifold's own development will change the equation for ESRI, because it will change the equation for its customers.

teconner7396 post(s)
#22-Aug-10 12:51

"Manifold could do the same, in theory. But the problem really is with ESRI. They've made a "principled" decision to pitch a proprietary database format as an industry standard, and many large organisations have fallen for it. Not without some good reasons." tjhb

I do not see the problem as being with ESRI. ESRI does not have to open up its proprietary formats any more than Manifold has to open up its .map format to the public. ESRI is a business and they are going to do whatever it takes to maximize their profits. If there is a problem, it is with government organizations using taxpayer funds to create data and provide it to the public only in such proprietary formats.

How can anyone blame ESRI for such a situation?If one is referring to private organizations taking such an approach, then they have the right to do so.If another vendor, including Manifold has issues with this; the only way to remedy them would be to produce a product that far exceeds the capabilities of ESRI’s products.Considering the fact that most people that perform GIS do so with ESRI products would suggest that currently there is no product currently on the market that can meet the holistic needs of the GIS community better than those offered by ESRI.Despite what some have argued, GIS organization are not filled with ill informed ESRI lap dogs, most organizations choose ESRI products because they offer the best solutions available to fulfill that organizations needs.Obviously, this situation is not unique to GIS.The proliferation of Microsoft’s word format is very similar in nature.

tjhb
10,094 post(s)
#22-Aug-10 18:54

I do not see the problem as being with ESRI. ESRI does not have to open up its proprietary formats any more than Manifold has to open up its .map format to the public.

You might be right that ESRI does not have to open up its proprietary formats (though the legal arguments in the other direction are not hopeless). That's not the question.

The question is whether its not doing so is a problem. You seem to tacitly accept that it is ("I do not see the problem as being with ESRI").

ESRI might be playing entirely fairly by peddling a closed format as a public standard, or if not fairly then within its rights. But it is causing a problem for its customers—and the great data-owning public—by doing so.

Ultimately it will see that that is not in its own interests, either.

Manifold doesn't make its .map format public. On the other hand, Manifold makes sure its product can talk to every major DBMS format, in the DBMS's own language, without imposing needless restrictions, and in such a way that the data can also be accessed by other products. We'll have to wait and see how much further version 9 takes this, and how if at all it changes the market.

Your very last sentence is right, and that's why the Office formats were eventually opened up.

teconner7396 post(s)
#23-Aug-10 10:18

The question is whether its not doing so is a problem. You seem to tacitly accept that it is ("I do not see the problem as being with ESRI").

ESRI might be playing entirely fairly by peddling a closed format as a public standard, or if not fairly then within its rights. But it is causing a problem for its customers—and the great data-owning public—by doing so.

Ultimately it will see that that is not in its own interests, either.

Obviously there is a problem, orerockon cannot get the data he/she needs because it is in a file geodatabase.If this is BLM data as the title of the attachment suggests, then I would say the problem was caused by the BLM.I am sure ESRI is very happy about such situations because it forces people to use their software.ESRI has very little if anything to gain by opening up their format.Why would they willingly give up such an unfair competitive advantage, especially if they fear that it might lead to people using products other than their own? This situation could only change if enough people demanded organizations such as BLM to publish their data in truly open formats, as they are required to do.

mdsumner


4,260 post(s)
#13-Dec-10 23:10

A minor follow up, ESRI announces the long-awaited API:

http://blogs.esri.com/Dev/blogs/geodatabase/archive/2010/12/13/File-Geodatabase-API-details.aspx

Will depend on ESRI software being available, but would allow 3rd party access (AFAIK).


https://github.com/mdsumner

tjhb
10,094 post(s)
#13-Dec-10 23:32

That's massive news. There are limitations of course but it significantly widens the scope for selling Manifold as a "must have" tool even to organisations devoted passionately to ESRI products and standards. (At least, to those who've migrated to ArcGIS 10, since the API won't support anything before 10.)

Will Manifold graciously adopt the API and allow read access to GDB format? I doubt it. There are bound to be good and noble reasons why they shouldn't.

It might fall to the Manifold community to write an interface—and to make it free. I'd be keen to be on the team for that.

(When the API is released, in beta, in January.)

Thanks Mike.

BCowper


1,275 post(s)
#14-Dec-10 04:08

This is great news for all of us who work with Arc file geodatabases, thanks for the tip off Mike!

Tim - hasn't Manifold.net shown in the past that they are willing to add support of data formats if there are enough hands going up? Get those suggestions fired in - vote early, vote often!

tjhb
10,094 post(s)
#14-Dec-10 11:35

Brian, I do think you're right, so long as the ESRI code doesn't fall too far short of Manifold's high standards. They're not going to expose themselves to misery if the API is buggy. But if it's well written and well documented, then let's hope it comes down to voting in the usual way, even if Manifold have to hold their noses, as ever with shapefiles.

mdsumner


4,260 post(s)
#14-Dec-10 12:00

There'll be the GDAL route too no doubt. If GDAL / OGR was available as a data source in Manifold then a future driver for this API would be too. I'm still yet to ask for this, but I have some notes - I think it would provide a lot of good options, especially for anyone interpreting other data sets via GDAL already, but also for testing different routes.


https://github.com/mdsumner

danb

2,064 post(s)
#14-Dec-10 12:10

Nice Idea Mike, get it in. GDAL works very nicely in QGIS for imports.


Landsystems Ltd ... Know your land | www.landsystems.co.nz

Mike Fisher

899 post(s)
#14-Dec-10 14:54

Until the API is released and the terms and conditions published there is no way to know if there will be constraints (license or technical) that would make it difficult for an ESRI competitor to utilize it.

Given that ESRI's strategy for technologies like file geodatabases is to lock ESRI users into an ESRI silo, it is unlikely they would themselves subvert many years' of strategy by releasing an API that a competitor could use to enable free interchange.

Forest
625 post(s)
#22-Dec-10 21:18

I have heard from a professional Arc user that the gdb format has been opened up since the publication of Arc GIS 10. If all the Arc users switch to gdb format, it would make it very hard to switch to Manifold or use Manifold as well as Arc. I think that that is what is happening.

tjhb
10,094 post(s)
#22-Dec-10 22:21

I'm a bit lost by the reasoning.

If the GDB format is opened up for releases ≥ 10, and if many Arc users switch to 10 and to GDB format, and if the "opening up" of the GDB format means a reasonably priced SDK with no silly exclusions, then that should make it easier to use Manifold alongside ArcGIS and ArcServer, shouldn't it?

Dimitri


7,413 post(s)
#22-Dec-10 23:03

[edit - tim beat me with a concise comment... that will teach me not to be verbose! :-)]

Forest, I don't follow that. It seems to me that if gdb has been opened it would be easy for Manifold and anybody else to use it, true? Why would anybody hesitate?

If it were really opened that would make it easy for Manifold or any other GIS using the "open" thing to be procured and installed side-by-side with Arc, which is enough for Arc to lose more share (no need to force a switch). If on the other hand it has not been opened, that argues against it for people who wish to avoid being trapped in a "silo."

There is also the fundamental concern that gdb is not a particularly efficient or reliable format. Setting aside the fragility of embedding SDE-like structures or shapefiles (both of which have significant problems from a modern perspective) into a DBMS format, if one was to pick a DBMS format MDB is not a good choice because it is really old and fragile.

Microsoft's mdb (as Microsoft is the first to say) is a notoriously fragile format, especially for multi-user access or multi-process access as with a GIS-enabled web site. Microsoft has plenty of blood-curdling knowledge base articles about how to attempt to revive a corrupted .mdb, what best practices to use to try to keep a .mdb from being corrupted and so on. That's why many years ago they introduced MSDE as a more robust option and now have introduced SQL Server Express as an alternative.

Last but not least, isn't a .gdb limited to only 2 GB because of the 2 GB .mdb limitation? It's difficult to believe individual consultants doing significant work will settle for that, let alone government departments or enterprises. In the year 2010 they usually want more, and expect to need yet more again in 2011 and onward.

I don't know. I get the impression ESRI has been trying to push GDB adoption for years and not getting enough headway. With a few customers, yes, but to the degree necessary to replace even shapefiles, no.

It could be they've finally gotten some traction with some users, but that would be unfortunate for those users if that traction finally comes at a time when expectations have moved on. 2GB max is way too small, fragility in multiprocess or multiuser ops is no good for workgroups, government departments and enterprises, and the low performance of mdb compared to more modern technologies (heck, even early 2000's MSDE is far faster) is altogether looking back before the year 2000 planning for the past instead of standing in the year 2011 looking ahead planning for the present and the future. The more time goes on the more dated and less attractive gdb looks, so I'm not surprised if they are talking about opening it to try to get some traction before the only people left who know what it is start thinking more about Medicare than GIS.

It could be a moot point anyway, since if it ever got real traction Manifold would just open it anyway and if it doesn't get traction it is a don't care. After a zillion formats it's just one more that either will go somewhere or it won't.

volker

1,086 post(s)
#23-Dec-10 03:54

From ESRI Help-site:

  1. File Geodatabases—Stored as folders in a file system. Each dataset is held as a file that can scale up to 1 TB in size. This option is recommended over personal geodatabases.

  • Personal Geodatabases—All datasets are stored within a Microsoft Access data file, which is limited in size to 2 GB.

  • ArcSDE Geodatabases—Stored in a relational database using Oracle, Microsoft SQL Server, IBM DB2, or IBM Informix. These multiuser geodatabases require the use of ArcSDE and can be unlimited in size and numbers of users.

    Manifold doesn`t have a problem to read the Personal Geodatabase (based on *.mdb), but it`s not possible to read a file based geodatabase (and this is up to 1TB).


    http://www.thegisservicesector.de

  • orerockon

    392 post(s)
    #23-Dec-10 08:28

    I composed a lengthy and rambling reply that was subsequently hosed by IE (thanks again Bill) So to cut out the fluff, I will simplify my issue:

    Fed and state governments have been conned by ESRI to such an extent that virtually all of the new/updated coverages I receive or find posted to the net are now GDBs. I am sick and tired of having to beg/plead/cajole them into exporting them to usable format. Which doesn't always work - some of them have taken to ignoring my requests to do so (ref the "Supersize Me" filmmaker's repeated requests to interview someone at McDonalds LOL).

    The feds are notorious for standardizing and forcing standards on individual agencies (blame the NIST and the GAO for that). Their huge inertia makes it 1) extremely difficult to get them to switch to a new standard and 2) almost impossible to revert to a previous standard just because it made more sense int he first place.

    I expect that many more users are going to need the ability to import GDB coverages without relying on $$$ARC software (not just us vocal geeks either. Imagine the number of "lurkers" who never post on this forum only). Given that, doesn't it make sense for Manifold to try to accommodate users if/when it becomes possible to do so (assuming the API thingie is released next month, if ever)?

    Mike Fisher

    899 post(s)
    #23-Dec-10 09:08

    Fed and state governments have been conned by ESRI to such an extent that virtually all of the new/updated coverages I receive or find posted to the net are now GDBs.

    and

    Given that, doesn't it make sense for Manifold to try to accommodate users if/when it becomes possible to do so

    Manifold makes an effort to implement formats many users request, especially if large amounts of data are published in that format. Tips for recommending a new format are given in http://www.manifold.net/info/suggestions.shtml in the "Suggestions for New Formats" section.

    If there are many files extent in a suggested format and significant numbers of users write in as recommended by http://www.manifold.net/info/suggestions.shtml, that helps overcome implementation obstacles - if the format is difficult or not documented. For example, Manifold supports .e00 even though that format is not documented.

    As Klaus notes, "GDB" is three different formats: ESRI has used "GDB" to refer to three significantly different technologies (slide 6 of the presentation at http://proceedings.esri.com/library/userconf/pug07/papers/workshops/file-gdb.pdf): "Personal GDB," "File GDB" and "ArcSDE GDB." Therefore, should anyone choose to send in suggestions requesting "support for GDB format" please be specific about which you want. State which GDB is used for any examples of available data cited to increase priority of the request.

    Has anyone in this thread sent in a request for GDB? If not, and this is something you want, do not hesitate to follow the tips in http://www.manifold.net/info/suggestions.shtml to give your request the maximum influence you want it to have.

    Forest
    625 post(s)
    #23-Dec-10 23:11

    Sorry guys, I didn't put in enough detail. I was talking about file geodatabases. I work in an ESRI shop with over 100 Arc licences and I am the sole manifold user. When all the other GIS users start using file geodatabases and I can't read them, that is going to cause me serious grief. In fact it could be the killer argument to get manifold permanently out the door. I think that manifold is far better suited to the type of work that we do than Arc and is far more productive. That is the perspective of someone who has the current versions of Arc, Manifold and MapInfo on the same machine. Yes I could use Arc to export shape files from the geodatabase but that makes the process very long. File geodatabases are also painful as when someone else is using it, you can't just copy it. File geodatabases do not work over a medium speed wide area network so one has to make a local copy of the database. If you make a local copy and the file path has changed, it is quite painful to reconnect all the layers in a map document to see what maps the others have already created. For comparison, manifold map files from another office can be opened over a medium speed network and there is no issue with reconnecting to layers. Either manifold needs to learn to read geodatabase layers or serve layers that can be read by Arc.

    Mike Fisher

    899 post(s)
    #24-Dec-10 11:34

    When all the other GIS users start using file geodatabases and I can't read them, that is going to cause me serious grief.

    Have you sent in a suggestion on this, perhaps with an offer to upload sample data? You have real world experience and probably some strong opinions based on that experience for what you want done. See http://www.manifold.net/info/suggestions.shtml

    artlembo


    3,400 post(s)
    #21-Aug-10 08:46

    orerockon,

    I can try converting this for you into a .map file. Contact me offline so we can discuss what you need: artlembo<at>gmail<dot>com.

    volker

    1,086 post(s)
    #24-Aug-10 11:39

    i have converted your file geodatabse to shp-file.

    It was 3 shp (line, point & area, 4 tables and 4 saved relations,

    but i don`t know how to work with this relations in ArcGis and an auto

    loading this to Arc don`t work.... (in Manifold it`s no problem for me,

    but working with ArcGis it`s more complicated for me than with manifold...)

    Send me an email to cryjackone@aol.com so i send it to you...


    http://www.thegisservicesector.de

    teconner7396 post(s)
    #24-Aug-10 14:20

    Here are few problems converting geodatabases to shapefiles.

    1. You lose geometry, no true curves.

    2. Attribute names are truncated if too long.

    3. Shapefiles do not support the same data types offered in geodatabases.

    4. You lose domains, subtypes, and validation.

    5. You lose relationship classes, topology, annotation classes, network classes, etc.

    If one works with non-ESRI software like Manifold, then its best to use a personal geodatabase or an ArcSDE geodatabase. Even though Manifold doesn't support much of the functionality of a geodatabase (such as 1, 4, and 5 above) you can at least retain all the functionality if using ESRI products and be able to link to supported features from Manifold.

    volker

    1,086 post(s)
    #25-Aug-10 02:14

    teconner73 you are right.

    So i convert it to a personal geodatabase with all relation classes.

    But it`s to big to attach here.

    Orerockon, for the datas send me an email.


    http://www.thegisservicesector.de

    volker

    1,086 post(s)
    #22-Jun-11 13:11

    News on file gedatabase:

    in the new "arcAKTUELL" 2/2011 (Germany) from ESRI found an

    article that`s the file geodatabase is now open.

    Have a look at:

    http://resources.arcgis.com/content/geodatabases/10.0/file-gdb-api

    File Geodatabase API

    The File Geodatabase API provides a non-ArcObjects based means by which advanced developers can work with File Geodatabases. A common user scenario is to open File Geodatabase tables in non-ESRI applications to view or modify data. This API provides access to the low-level File Geodatabase file I/O modules.

    The C++ API allows developers to:

    • Create new geodatabases

    • Read the schema of the geodatabase

    • Create schema for objects within the simple feature model

    • Read and write data in the geodatabase

    • Perform attribute and (limited) spatial queries on datasets

    This API is targeted for advanced developers who require access to the File Geodatabase without an ArcObjects license for purposes of interoperability.

    This API does not replace ArcObjects as the recommended approach to interacting with the geodatabase.

    You can post questions and comments about the API on the File Geodatabase API Forum.

    To download the API:

    Sign in to the ArcGIS Resource Center

    Download the API


    http://www.thegisservicesector.de

    mdsumner


    4,260 post(s)
    #22-Jun-11 13:15

    [hmm, not sure about what I thought I was saying]


    https://github.com/mdsumner

    volker

    1,086 post(s)
    #22-Jun-11 13:21

    that`s bad

    Esri wrote:

    the API stands for an open and standardised exchange of geodatas.

    ESRI proves with it once more big engagement for openness and interoperability.

    ???

    so one more ESRI joke ?


    http://www.thegisservicesector.de

    dchall8
    1,008 post(s)
    #23-Feb-17 16:14

    I found this topic while looking for something else, but it is a fascinating topic. The GDB is much more than another implementation of shape files. I had an opportunity to work with a GDB format back in 2014. An oil company in Houston it seems had hired the entire graduating class from the Penn State master's in ESRI program (they call it something else). They put on a 1-day seminar for all their subcontractor surveying companies about how to submit GDBs to them. The oil company was insisting the surveyors stop submitting CAD drawings and submit ESRI's GDBs. What these kids had created was an amazingly complicated system of 3-dimensional drawings and linked databases to help them locate and drill oil wells. For example when they pulled up a site location drawing they had info on the roads leading in, the road base type and depth, surface finish, width, age, and general condition. They could tell you by volume how much more roadbed material was needed to support their their trucks and to create a square drill pad at a precise altitude above sea level. This was all data they need to put in a well, but they wanted the surveyors to provide raw data in a GDB instead of on paper or even a spreadsheet. Of course this makes sense, but my point is that using the extended capabilities of a GDB was genius and something you could not do with shape files alone.

    Also regarding Dimitri's statement, "[edit - tim beat me with a concise comment... that will teach me not to be verbose! :-)]", I don't think he learned anything. ;-)

    Okay I'm off to continue my search for whatever it was I was looking for.

    adamw


    10,447 post(s)
    #24-Feb-17 16:12

    (If this concerns file geodatabases which we didn't support before, we are working to add such support to Radian now, the first results are available in 9.0.159.1 on Cutting Edge. This is just the first iteration, it is going to improve significantly. Other ESRI geodatabase types we already supported.)

    Manifold User Community Use Agreement Copyright (C) 2007-2021 Manifold Software Limited. All rights reserved.