Subscribe to this thread
Home - Cutting Edge / All posts - Radian Studio 9.0.159.x
adamw


10,447 post(s)
#24-Feb-17 15:53

9.0.159.x

Converted to a public update.

Changes for the final build of 9.0.159.x are integrated into the notes for the public update build.

Changes for the final build of 8.0.29.x are integrated into the notes for the public update build for 8.

Changes for 9.0.159.6 and 8.0.29.3

Changes for 9.0.159.5 (designed to work with 8.0.29.2)

Changes for 9.0.159.4 and 8.0.29.2

Changes for 9.0.159.3 and 8.0.29.1

Changes for 9.0.159.2

Changes for 9.0.159.1

adamw


10,447 post(s)
#24-Feb-17 15:54

9.0.159.1

Changes

DSN editing dialog for the ODBC driver includes an option to turn on compatibility mode for Manifold 8. The option disables some of the advanced features of the driver and makes other adjustments to better work with Manifold 8. By default, the option is turned off.

ODBC driver supports relative scrolls for dynamic cursors.

ODBC driver supports relative and absolute scrolls for static cursors. Static cursors cache returned data on the fly.

There is a new GDB dataport for ESRI file geodatabases. The dataport uses ESRI FileGDB SDK. The required DLLs are included in installation packages, there is no need to install anything extra. (The dataport is a work in progress, but it has enough to list components and open vector data - currently without indexes.)

(Fix) Linking DHT and VDF files using 'All files' filter resolves them to correct dataports.

Action queries track progress and allow canceling at data collection phase.

DISTINCT, INTERSECT / EXCEPT / UNION and several other query constructs allow canceling.

CSV dataport distinguishes between empty strings (1,"",2,3 - the string in the second slot is blank) and NULL strings (1,,2,3 - the string in the second slot is NULL).

CSV dataport attempts to determine encoding for files without byte order marks - by looking at the placement of zero bytes and then, if the result is inconclusive, by inspecting statistics.

CSV dataport minimizes the number of internal data conversions for performance. (This more or less doubles the speed compared to 9.0.159.)

GPKG dataport renders images with bilinear filtering.

The query builder displays a context menu for list items. New commands: Insert Code / Insert Name - insert item code / name into the query text. Insert Definition - inserts the definition of a table into the query text. Insert Field List - inserts the list of fields for a table into the query text.

The query builder lists available data types.

Selecting a menu item or canceling a menu removes the description for the last selected menu item from the status bar.

End of list.

KlausDE

6,410 post(s)
#26-Feb-17 09:49

Here is an update of the german UI for working update 9.0.159.1

It's safe to use it with the official build 9.0.159. Items whose names have been changes in the meantime simply display the default english ui text.

Attachments:
ui.de.txt


Do you really want to ruin economy only to save the planet?

klausk115 post(s)
#26-Feb-17 15:30

thx

Where must be the ui.de.txt copied??

Dimitri


7,413 post(s)
#26-Feb-17 17:16

See the Localization topic.

adamw


10,447 post(s)
#28-Feb-17 16:46

9.0.159.2

Changes

Tooltip for the default mouse mode button includes brief description of keys used to pan, zoom, select.

The query engine allows converting binary values to any data type, with verification.

New query function: BinaryStringBase64 - takes a binary value and converts it to a base64-encoded string. Transform template: Compose Base64 String - allows composing a string for any value type.

New query function: BinaryStringHex - takes a binary value and converts it to a hex-encoded string. Transform template: Compose Hex String - allows composing a string for any value type.

New query function: StringBinaryBase64 - takes a base64-encoded string and converts it to a binary value. Transform template: Parse Base64 String - allows parsing a string into any value type.

New query function: StringBinaryHex - takes a hex-encoded string and converts it to a binary value. Transform template: Parse Hex String - allows parsing a string into any value type.

New query function: StringEncrypt - takes a string value and a key, encrypts the value. Transform template: Encrypt.

New query function: StringDecrypt - takes an encrypted string value and a key, decrypts the value. Transform template: Decrypt.

New query function: TypeName - takes a value, returns its type as a string.

New Field dialog generates a default name for the new field.

(Fix) New Field, Transform, Select dialogs correctly handle expressions on fields whose names contain '{<number>}' character sequences.

(Fix) Query builder correctly escapes names of constraints, fields or indexes which contain '[', ']' or other special characters.

Log window splits multiline error messages. Message source is appended to each of the resulting lines.

Command window splits multiline error messages in the Log tab.

SELECT with implicit group on the whole table always outputs a single record, similarly to other databases.

Loading Radian DLLs logs failures similarly to loading third-party DLLs (like those for SQLITE).

Attempting to load a Radian or third-party DLL which fails due to missing dependencies no longer sometimes displays a system error message in addition to a Radian error message / log line.

Attempting to launch Radian on a system with no installation prerequisites (.NET 4 and Visual C++ redistributable) shows a descriptive error message and opens the System Requirements page using the default web browser.

(Fix) Coordinate systems with multiple possible transforms to WGS84 no longer sometimes fail to select the transform with the best accuracy.

The default coordinate system for GeoJSON data is changed from EPSG:4326 to urn:ogc:def:crs:OGC::CRS84, reflecting changes in the GeoJSON standard. Writing GeoJSON data writes bounding box.

GDB dataport supports spatial indexes on vector data.

GDB dataport supports datetime fields.

GDB dataport removes leading slashes in component names for readability (internal component names used by ESRI SDK include slashes).

The ODBC driver automatically converts Radian geoms to Manifold 8 geoms if compatibility mode option is turned on. (No need to use GEOMWKB.)

The ODBC driver overrides metadata exposed to Manifold 8 with automatically generated values for drawings if compatibility mode option is turned on. If a particular table in the Radian MAP file contains multiple geometry fields with drawings bound to each, Manifold 8 will see one of them (a limitation of Manifold 8).

End of list.

tjhb
10,094 post(s)
#28-Feb-17 21:55

The Cutting Edge workflow you've designed is fantastic. Seriously clever management of seriously clever people. It's great to feel involved in the process, thanks.

Amazing how fast you can pick up and code small improvements that really make a difference--several here.

It would be interesting to have some comments on the design purposes behind the binary-to-string conversion functions (with string encryption). I imagine they are part of building a new interface to streaming web services (though we can use them in other ways). Can we know what algorithm is used for string encryption?

adamw


10,447 post(s)
#01-Mar-17 05:57

The binary to string / string to binary conversion functions (and the conversion of binary values to values of any type - the conversion of values of any type to binary we already had) are there to allow packing arbitrary data into blobs, then transporting them as strings somewhere else for further use there. This is useful for web scenarios, yes, and for various specialty things like packing multiple fields into one. We imagine the functions will be mostly used as part of bigger processing.

The string encryption / decryption functions use AES256. We hash the key first using 128-bit SHA, then encrypt data using the hash. The functions are intended for quick hiding of sensitive values in tables (eg, encrypt emails before sending the file to someone who should see the field - should be able to launch queries which use that field to group on it / whatever - but not the values).

Mike Pelletier

2,122 post(s)
#01-Mar-17 00:27

Adam, is there any hope that the ODBC driver could be altered to allow viewing databases in RS that are linked in from another database such as ESRI SDE? So in Mfd 8 one could see an SDE data that is accessed through RS.

adamw


10,447 post(s)
#01-Mar-17 06:01

We will think about it.

Dimitri


7,413 post(s)
#01-Mar-17 09:41

Great idea, Mike!

By the way, another idea everyone automatically thinks about when seeing the Encrypt tool is that it would be useful to be able to password-protect .map files. That's in the pipeline as well, not as a super high priority item but something that needs to get done sooner or later.

tjhb
10,094 post(s)
#01-Mar-17 10:01

Similar: to be able to password-protect components within a .map file (including a whole folder with its contents)--not to make the component(s) inaccessible, but for tables, to make them read-only, and for code, to make it either read-only or run-only (no read).

artlembo


3,400 post(s)
online
#01-Mar-17 13:37

I second that capability. Most protection of layers happens at the database level with a linked table (i.e. PostgreSQL), but if Radian is going to play in the same pond, I think having the ability to add user roles or at least some kind of protection to layers is the next logical step. Radian is already allowing for functions, indexes, etc., so that becomes a nice feature.

adamw


10,447 post(s)
#01-Mar-17 14:07

(Without making any promises...)

There is a certain appeal to have protection solely on the MAP file level (easier UI for one: just one prompt for password when you try to connect to the protected file). If you then want to have some components protected and others not, you can create a new MAP file, link to it, move the components you want to protect there, and protect the file.

Any big drawbacks? (If the hierarchy is as I described, that is, an unprotected MAP file has a link to a protected MAP file, queries have to be unprotected, because if they are protected they won't be able to reach unprotected components - you can't go to a higher-level data source, only to a lower-level one. But the hierarchy can be reversed - a protected MAP file linking to an unprotected one.)

artlembo


3,400 post(s)
online
#01-Mar-17 13:58

It looks like when reading the ESRI geodatabase (.gdb), the table with the [Shape] field does not seem to interpret the coordinate system. I get:

SRID:ARCSDE:0

Nonetheless, this is really cool, issuing SQL on ESRI geodatabases! I'm going to make a video of it today.

adamw


10,447 post(s)
#01-Mar-17 14:02

Yes, the GDB dataport does not yet include support for coordinate systems, we are working to add it.

artlembo


3,400 post(s)
online
#01-Mar-17 14:24

ok, thanks. BTW, here is a link to a short video where Radian works directly with a geodatabase:

bdg63 post(s)
#02-Mar-17 16:04

I have crashed Radian. I had loaded an earlier version of of a gdb from https://data.femadata.com/FIMA/Risk_MAP/NFHL/

I had datasources to a local sql server instance, a remote sql Server, and the gdb. It maybe related to the locked state of the remote database I reported earlier.

Attachments:
crash.PNG

adamw


10,447 post(s)
#02-Mar-17 16:08

If you didn't yet close the process and have a couple minutes of time, please create a dump file and send it to us (we can provide FTP space on request, or you can use a file sharing service and email us a link).

To do this:

Launch Task Manager, go to the processes list (in Windows 10 use either the Processes tab or the Details tab), locate the crashed process, right-click it and select 'Create dump file'.

Dump files are extremely useful when debugging things like that. (And if the crash happened outside of our code, we will see that too and might make amends as well.)

bdg63 post(s)
#02-Mar-17 16:26

Apologies it is too late. If it happens again I will be sure to follow through.

adamw


10,447 post(s)
#03-Mar-17 18:06

9.0.159.3

Changes

(Fix) Hotine oblique Mercator projection variant used for Switzerland and Hungary no longer sometimes produces wrong coordinates.

New query function: CoordSystemXml - takes a coordinate system definition and converts it to Manifold 8 XML. Not all systems can be converted (example: EPSG:5515 can not be converted, because the system type is modified Krovak and Manifold 8 does not support that), but the vast majority can.

The ODBC driver exposes coordinate systems for drawings to Manifold 8, if compatibility mode option is turned on.

The ODBC driver supports opening MAP file in read-only mode, via a dialog option. Using read-only mode allows connecting to the same MAP file concurrently from multiple processes.

The ODBC driver supports connecting to a data source inside a MAP file, via a dialog option (the user has to specify the name of the data source). This allows using the Radian ODBC driver to access data in any data source type supported by Radian.

The GDB dataport exposes btree indexes on GDB data.

The GDB dataport allows running queries using GDB engine. (As usual, you can switch betwen GDB queries and Manifold queries via '!native' and '!manifold' commands in the command window.) The dataport no longer refers to components by their paths, and uses their query names instead.

The GDB dataport exposes coordinate system data.

The GDB dataport reads multi-patch geometry and geometry with curves (not all curve types are supported, but the most frequently used - circular arcs and Bezier splines - are).

The GDB dataport reads geometry with Z values.

(Fix) The GDB dataport forces index names to not coincide with field names.

Reading data from GDB tables performs significantly faster.

Pasting data reports time to complete the operation.

Pasting data into SQLITE data sources in the Project pane performs dramatically faster. (The optimization is to open a single explicit transaction when inserting records, it might improve things for other dataports, but the effect on SQLITE will be hard to beat - easily 40x-100x the former performance depending on the field layout.)

8.0.29.1

Changes

(Fix) CSV and other imports no longer sometimes try to allocate too much memory, which makes them very slow.

(Fix) Linking a drawing using the Radian ODBC driver no longer generates wrong queries if the drawing's table contains a dot in the name.

(Fix) Linking a drawing or table using the Radian ODBC driver correctly handles 64-bit integer fields like mfd_id.

End of list.

Mike Pelletier

2,122 post(s)
#03-Mar-17 20:07

Connecting mfd 8 to RS works great for drawings. Radian is getting closer to being the dreamy container for mfd 8 drawings we've been hoping for

Any advice for determining the data source name when setting up the odbc connection to an external data source in a RS file?

Hope you can add other RS component types such as images to the odbc driver in the future.

adamw


10,447 post(s)
#04-Mar-17 05:39

Any advice for determining the data source name when setting up the odbc connection to an external data source in a RS file?

You can open the MAP file in Radian while you are in the DSN dialog to look up the name of the data source. We might extend the DSN dialog with means to scan the MAP file and provide a list of available data sources in the future.

The name of the data source in the DSN dialog can be nested - [mapfile2]::[mapfile3]::[postgresql] is fine.

We looked into exposing images and the biggest issue is that Manifold 8 does not have much in the way of code required to link them. We can obviously write / port the necessary code but we don't want to change Manifold 8 too much (would rather push Manifold 9). If we find a way to link images from Radian to Manifold 8 without making too many changes to Manifold 8, we will do this.

Also, regarding drawings and tables - most of the effort on exposing data to Manifold 8 via the ODBC was on reading. Writing currently has some issues (newly created objects might not be editable until you refresh the drawing, etc). We are working on them.

Mike Pelletier

2,122 post(s)
#06-Mar-17 22:07

Thanks Adam for the background on ODBC development. Glad you guys are putting energy into it and happy to report it's fast and easy. Turns out my trouble with names mentioned above was caused by the server being down rather than any name hangups.

Dimitri


7,413 post(s)
#04-Mar-17 07:15

This allows using the Radian ODBC driver to access data in any data source type supported by Radian.

This is huge.

tjhb
10,094 post(s)
#04-Mar-17 07:38

To spell it out a bit more:

This allows any standard ODBC client to access data in any data source supported by Radian.

Is that right?

(Please go on through. God will see you now.)

adamw


10,447 post(s)
#07-Mar-17 16:23

Yes, any standard ODBC client.

There are currently a couple of issues with applications like LibreOffice (some tables don't show their data), but we are working on them.

The idea is: take something like Access or Excel and access live data in the SHP / MIF / KML / ESRI file GDB / other supported format using Radian through ODBC. Or connect to a data set in a supported format using ODBC pass-through tables / views in databases (we can link databases to MAP files, pass-through tables in databases do the reverse).

adamw


10,447 post(s)
#07-Mar-17 16:11

9.0.159.4

Changes

(Fix) The ODBC driver no longer sometimes returns wrong data when directed to connect to a data source inside a MAP file.

(Fix) The ODBC driver no longer fails to correctly mark up NULL values in some ODBC views.

The ODBC driver registers under a different name for working update builds.

CDF file extension is mapped to the GDB dataport.

The GDB dataport parses more curve types in geometry data, including ellipsoidal arcs and variants of circular arcs.

(Fix) The GDB dataport correctly reads GUID fields.

The GDB dataport automatically adjusts variants of produced BTREE indexes for unique / nullable field properties.

Selecting all records without an index in a GDB table performs significantly faster.

8.0.29.2

Changes

Splash screens are updated for 2017.

The ODBC code recognizes the ODBC dataport provided by working updates for Radian (and treats it like the ODBC driver for an official build of Radian).

End of list.

KlausDE

6,410 post(s)
#10-Mar-17 20:37

German ui file for working update v9.0.159.4

Attachments:
ui.de.txt


Do you really want to ruin economy only to save the planet?

adamw


10,447 post(s)
#13-Mar-17 16:51

9.0.159.5

Changes

The table returned by SELECT without joins / WHERE / GROUP supports inserting and deleting records. Deleting records from such a query in a command window requires re-running the query to make deleted records disappear. (The main reason for the addition is to help clients connecting through the ODBC driver - a number of them write tables not by running INSERT / DELETE / UPDATE commands, but rather by running SELECT ... FROM <table> and performing writing ODBC calls on the resulting table.)

(Fix) Running 'SELECT Count(*) FROM t' on a GDB table no longer fails.

Indexes on OID and similar special fields in GDB tables are forced to be unique (the GDB SDK may sometimes fail to describe them as such).

Attempting to use the ODBC driver to connect to a MAP file which is already opened in another connection in an incompatible mode (read-only vs read-write) fails with a descriptive error message.

The GDB dataport logs errors returned by the GDB SDK if the geodatabase fails to open.

The ODBC driver supports delayed binding of parameters in parameter queries. (This solves many issues with traditional clients like Access, which may bind parameters either before or after asking to compile a parameter query, depending on the scenario.)

The ODBC driver supports updates for ODBC cursors. (This allows making changes to linked tables from Manifold 8 and other clients that do updates via cursors.)

ESRI's GDB SDK is upgraded to 1.5, the latest version. The new SDK uses the runtime for VS 2015, there is no need to install the runtime for VS 2013. (The new SDK remains unable to open geodatabases created with 9.x versions of ArcGIS.)

End of list.

adamw


10,447 post(s)
#13-Mar-17 17:05

As of this build, tables linked via ODBC:

  • should be readable in all applications - Manifold 8 / Microsoft Office / LibreOffice / anything,
  • should allow updating records in all applications,
  • should allow inserting and deleting records in some applications, but not others (this is the last thing we want to complete for the public build).

Please report any errors you encounter.

(The below is a screenshot of a geodatabase table opened in LibreOffice.)

Attachments:
shot-libreoffice-gdb.png

Mike Pelletier

2,122 post(s)
#15-Mar-17 16:43

Adam, a couple thoughts. Our data in ESRI SDE is in State Plane Colorado Central, NAD83, feet, however Radian and Mfd 8 read it as being under the Conic folder as Lambert Conformal Conic, NAD83, feet. I can change it to State Plane but it reverts back to the original upon a save of the project. It can display, but the speed is poor due to having to reproject. I sent my SDE data awhile back during the beta, so perhaps you could use that to look into why it applies a slightly different projection.

Also, it would be much better if links didn't have to be refreshed after a save. I may be wrong, but t seems that the relinking process is actually longer than the delay between clicks during the original linking process. I'm using this little script to refresh all the link components after a save, but it takes awhile with lots of linked drawings.

Sub Main

 Set LinkedDocuments = Application.ActiveDocument

 LinkedDocuments.RefreshAllLinked() 

End Sub

adamw


10,447 post(s)
#16-Mar-17 07:01

State Plane Colorado Central is Lambert Conformal Conic with specific parameters. I don't think there is reprojection, the projection specified in the data probably uses a slightly different name than our preset and so we are losing that name. We will check if we can map the projection name as well.

Regarding rendering speed - does it get better if you unlink the drawing, apart from not having to refresh data? If it does, tell us, the rendering speed should be about the same (unless there is really heavy use of thematic formatting). Otherwise, well, that's just the difference between Manifold 8 and Radian. :-) Give us some time and we will grow Radian into Manifold 9, eventually covering everything that Manifold 8 has, but with better speed and better everything.

Same for not having to refresh linked components after a save - that's in the design of Manifold 8, saving a file does a full reopen and so since linked components do not save their data, they have to refetch it. We will check if we can optimize the process of refetching data specifically for the Radian ODBC driver, but we can not do much to reduce the overhead on the side of Manifold 8, that requires big architectural changes.

Mike Pelletier

2,122 post(s)
#16-Mar-17 18:50

Unlinking the layer does not effect the redraw speed noticeably. The change in parameters is apparently enough to trigger the reprojection on the fly process. Capture.jpg is the incorrect assignment and Capture2.jpg is the correct assignment.

That all makes sense on the refreshing issue but figured I'd ask. The better everything part sounds great and appreciate all your work on it :-)

Attachments:
Capture.JPG
Capture2.JPG

adamw


10,447 post(s)
#17-Mar-17 06:54

Thanks for the screens. There is, indeed, a reprojection, because the difference between projections is not just in the names, the values for the center latitude and false northing parameters are (a tiny bit) different. It does not matter how small the difference is, the system does not attempt to reason whether the parameters are different enough to engage reprojection (maybe it should, but that's not easy to do cleanly and usefully).

I will check whether the first projection is a correct representation of the one used in your data.

tjhb
10,094 post(s)
#17-Mar-17 07:16

I had been wondering whether (for the second example) 1000000 US survey feet, when converted to metres, might take a binary representation that cannot be exactly round-tripped. That is probably just wrong.

Mike Pelletier

2,122 post(s)
#17-Mar-17 14:37

Here's ESRI's prj file for Colorado Central. Right or wrong it provides those parameters that are slightly different than Manifolds version. What I've done over the years is simply reassign the projection from any drawing brought in from ESRI to match Manifold. Not too many clicks but it would be nice to avoid that.

Attachments:
stateplane.prj

KlausDE

6,410 post(s)
#20-Mar-17 10:16

External applications that use Mfd8 of course need to be recompiled against the working update!

How do I report a failing reference when my app tries to load Ext.dll ? It took me some time to find out because my program just dies silently.


Do you really want to ruin economy only to save the planet?

adamw


10,447 post(s)
#20-Mar-17 12:14

You mean how do you report when a .NET application that uses the Manifold 8 object model and references EXT.DLL can not find the correct version of that DLL installed on the machine? That's outside of your control, the code in the application never gets the chance to execute. That said, there should be an error message from the system loader.

This is one more thing that works better with Radian. There are now two DLLs: EXT.DLL for the core and EXTNET.DLL for the connector / services. The application using the Radian object model puts a copy of EXTNET.DLL of the build it has been developed with into its own folder. That DLL never fails to load because it does not get updated without the application knowing. And if that DLL fails to connect to EXT.DLL, it throws an exception - and your code can then fail gracefully (for example, by proceeding without the Radian object model available).

adamw


10,447 post(s)
#20-Mar-17 16:18

9.0.159.6

Changes

(Fix) The result table of ORDER BY no longer sometimes reports to include an index that refers to non-existing fields.

(Fix) Attempting to connect to a data source with invalid or missing technology fails. (Previously, doing that was producing a blank database.)

Attempting to connect to a data source in the Project pane reports errors.

(Fix) Opening SHP or DBF files with cache turned on no longer disables compacting saves.

The ODBC driver supports inserting and deleting records using cursors.

The GDB driver builds its own temporary index on ID field on demand to make searches on that index faster.

The GDB driver allows inserting, deleting and updating records in GDB tables. The ESRI SDK snaps coordinates of the geometry objects to an internal grid, so the final coordinates of the written geometry values slightly differ from the supplied coordinates.

The ODBC driver automatically negotiates Manifold 8 compatibility mode with the connecting application. The compatibility mode checkbox in the DSN dialog is removed. The automatic negotiation allows using the exact same data source from different applications.

The ODBC driver returns keys for the inserted record to Manifold 8. This allows records inserted from Manifold 8 to appear in the components immediately without having to refresh component data.

The ODBC driver supports streaming writes of record values. This extends the number of applications that can use the driver in read-write mode.

(Fix) Attempting to edit a record in a Radian table connected via the ODBC driver to Microsoft Access no longer sometimes fails with the 'This record is already being modified by another user' message.

8.0.29.3

Changes

Connecting to the Radian ODBC driver switches on compatibility mode for Manifold 8.

Inserting data into a table linked from Radian ODBC driver recognizes the mfd_id field in a MAP files and lets the database set the value for that field automatically.

Attempting to open a MAP file created in Radian Studio fails with an error message that suggests opening the file in Radian Studio.

End of list.

adamw


10,447 post(s)
#20-Mar-17 16:30

This is going to be the last working update build in this sequence of builds.

We are reasonably happy with the state of the GDB dataport, which is now read-write.

We are also reasonably happy with the state of the ODBC driver which had numerous adjustments and improvements. These improvements significantly expanded the range of applications that can work with the driver and (a) see all the data as well as (b) avoid disabling vital editing features. We are not currently aware of any issues with using the driver from Manifold 8 or from LibreOffice. There are several issues with using the driver from Microsoft Access, but the majority of these are related to Access having spotty support for 64-bit integer values - Microsoft are preparing a fix for that according to their site. We will, of course, test how that fix will work with our code after it arrives.

If you come across any issues, please report them.

artlembo


3,400 post(s)
online
#21-Mar-17 02:51

I am away on vacation this week, but want to get back to this. I am especially interested in what kind of damage the read–right capability of the ..gdb will have. One of the dangers of the Geodatabase is that it is designed to be manipulated with ESRI ride products. I'm wondering if a third-party like Radian allows editing of the database without consideration for things like domains, constraints, and other rules. The reason, of course, is because the rules are not designed at the database level but more through a middle tier.

If anyone is able to test this before I get back it would be good, otherwise, I will do it. Basically, I would set up the Attribute Domain in the Geodatabase that requires something like the pipe diameters to only be 8, 12, and 14 inches. Then, using Radian I would update the pipe diameter and set it to say, 3 inches.

Dimitri


7,413 post(s)
#21-Mar-17 04:35

what kind of damage the read–right capability of the ..gdb will have.

I hope you are having a good time on vacation! But I cannot resist a quick question just in case you know the answer off the top of your head: do any of the ESRI published examples or sample local government / utility templates in GDB include such constraints? For example, the WaterDistribution.gdb template they published for a water system in Naperville?

Given that ESRI has published an SDK that allows, in theory, zillions of other applications to connect to GDB read/write if there is a problem with the SDK being unaware of constraints specified at some middle tier that problem will not be unique to Radian. It's true that most other applications seem to connect to GDB read-only and not read/write like Radian, but I'd be very surprised to learn Radian is the first to connect to GDB read/write.

artlembo


3,400 post(s)
online
#21-Mar-17 23:01

I think you are correct – since you didn't hack into the Geodatabase but are using the SDK I think everything should be fine. Unfortunately, I can't even remotely log into my computer to test it. But, if no one else has an answer, I will have one to you by Friday night.

adamw


10,447 post(s)
#21-Mar-17 07:51

That would be useful.

We are going through the ESRI SDK so we hope the user won't be able to do too much damage (the SDK should reject changes that are invalid on a low level), but some of the logical links that are maintained on a level higher than the SDK operates on may indeed be broken.

We are already aware of at least one thing that we should disable on our level - not mixing geometry types in the same table. (The SDK allows adding a point into a table of areas and everything appears fine on write, no errors, but then the spatial index suddenly stops returning half of the objects. While the behavior is weird and one would expect the SDK to reject writes like that if it can not deal with them later on, since it doesn't reject them and we know they are harmful, we have to reject them on our end.)

We are looking for other things like that that we have to watch for, so any testing in that area would be most helpful.

artlembo


3,400 post(s)
online
#27-Mar-17 15:25

ok, dumb question: I am attempting to test this out, but I can only connect as read-only (I'm using .160). I don't know if the problem is that Windows 10 sets a read-only property on directories. I've changed the permissions of the directory, but the read-only property never seems to go away.

Any thoughts on connecting to the gdb so that I can actually update the records?

adamw


10,447 post(s)
#27-Mar-17 16:32

Is there a btree index in the schema of the table that appears read-only?

If it isn't that, upload the database somewhere and we will take a look.

artlembo


3,400 post(s)
online
#27-Mar-17 21:54

it's a small dataset from a student, but it does have domains in there that I would like to test. A friend thinks that the SDK may not honor the geodatabase rules. I don't think that is true, so I really want to test it out.

Attachments:
WaterUtilitiesDenton.gdb.zip

adamw


10,447 post(s)
#28-Mar-17 08:46

Some of the tables are read-write (eg, Manholes), others are read-only (eg, Cleanouts).

I checked several read-only tables and they are read-only because the SDK says that the table "cannot be edited safely" (Table::IsEditable returns false).

Regarding honoring the geodatabase rules - if you want to check whether or not it is possible to set the value of a field that is restricted via a domain to be outside the domain, this seems to be possible, but then again, does not ArcGIS allow doing the same and it is just that it has a way to find records with violations like that and edit them (maintaining attribute integrity while editing involves running a manual command)?

artlembo


3,400 post(s)
online
#28-Mar-17 15:27

yes, if you stay strictly within the ESRI environment, you are "more safe", but you can still circumvent things. In the attached video, I show how the domain is circumvented in BOTH ArcGIS and Radian.

Attachments:
editgdb.mp4

adamw


10,447 post(s)
#28-Mar-17 15:42

Yup.

The good thing is that circumventing the domain does not make the geodatabase invalid. That is, you can still read it and it will behave predictably and you can edit it to fix the logical error. ArcGIS itself expects the geodatabase to operate even though the logical error is there, otherwise the error would be unfixable.

From our point of view that means that changes like that are safe, as in, the dataport does not have to prevent them, and if the SDK allows them as well (it does), they should go through to the data. We might add things to the UI to highlight records or values which seem to be in error, if that's desired, but that's a different thing. (The dataport also cannot trust the domains to hold for all records, it has to be prepared for values outside of domains. That means we cannot map it to our constraint.)

We are kind of thankful that changes that invalidate domains behave that way, because some other changes aren't so charitable (like writing a polygon into a table of points, which the SDK allows only to choke some time later) and those we have to prevent.

artlembo


3,400 post(s)
online
#28-Mar-17 16:04

right. This is an example of what happens when the rules are managed in the middle tier I think. If you created constraints (aka the geodatabase domain) or even triggers in Postgres, those cannot be circumvented, and would apply to any application trying to hit into the database. Therefore, controlling the rules at the database level is a better way of doing things.

adamw


10,447 post(s)
#30-Mar-17 14:11

A short notice: we are planning to start the next sequence of cutting edge builds tomorrow.

artlembo


3,400 post(s)
online
#30-Mar-17 16:58

Hints, please :-)

adamw


10,447 post(s)
#31-Mar-17 14:11

The date got moved, but just a bit - to Monday.

In terms of hints, among other things we are adding two new dataports:

1. A dataport for MrSID files, working through the LizardTech SDK included in the install.

2. A generic dataport for GDAL / OGR, working through whatever version of GDAL is installed on the machine (eg, from QGIS).

The latter dataport will, for example, allow access to ESRI file geodatabases created with ArcGIS 9.x. The access will be read-only, because the relevant GDAL driver is read-only, but that's obviously miles better than no access at all in that you can recover data and save to a supported format.

Manifold User Community Use Agreement Copyright (C) 2007-2021 Manifold Software Limited. All rights reserved.