atolcd / pentaho-gis-plugins Goto Github PK

The download of the plugins from marketplace fails. Also your instructions for manual installation do not work, probably for the same reason.
The link to the zip file doesn't work.

wget https://github.com/atolcd/pentaho-gis-plugins/releases/download/v1.1-snapshot/pentaho-gis-plugins-1.1-SNAPSHOT-bin.zip
--2015-11-02 13:27:58-- https://github.com/atolcd/pentaho-gis-plugins/releases/download/v1.1-snapshot/pentaho-gis-plugins-1.1-SNAPSHOT-bin.zip
Resolving github.com (github.com)... 192.30.252.128
Connecting to github.com (github.com)|192.30.252.128|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2015-11-02 13:27:58 ERROR 404: Not Found.

I've been trying to use the GIS plugin for pentaho but no matter which file I try to run by the GIS File input when I plug it into any other brick (Geoprocessing, Spatial relationships and so on) I got the following issue:

Unable to open dialog for this step
java.lang.NoClassDefFoundError: org/apache/commons/vfs/FileSystemException
at com.atolcd.pentaho.di.trans.steps.gisfileinput.GisFileInputMeta.getFields(GisFileInputMeta.java:211)
at org.pentaho.di.trans.TransMeta.compatibleGetStepFields(TransMeta.java:2057)
at org.pentaho.di.trans.TransMeta.getThisStepFields(TransMeta.java:2043)
at org.pentaho.di.trans.TransMeta.getStepFields(TransMeta.java:1874)
at org.pentaho.di.trans.TransMeta.getPrevStepFields(TransMeta.java:1943)
at org.pentaho.di.trans.TransMeta.getPrevStepFields(TransMeta.java:1908)
at org.pentaho.di.trans.TransMeta.getPrevStepFields(TransMeta.java:1894)
at com.atolcd.pentaho.di.ui.trans.steps.gisrelate.GisRelateDialog.getFieldsFromType(GisRelateDialog.java:649)
at com.atolcd.pentaho.di.ui.trans.steps.gisrelate.GisRelateDialog.open(GisRelateDialog.java:418)
at org.pentaho.di.ui.spoon.delegates.SpoonStepsDelegate.editStep(SpoonStepsDelegate.java:127)
at org.pentaho.di.ui.spoon.Spoon.editStep(Spoon.java:8789)
at org.pentaho.di.ui.spoon.trans.TransGraph.editStep(TransGraph.java:3179)
at org.pentaho.di.ui.spoon.trans.TransGraph.mouseDoubleClick(TransGraph.java:775)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.pentaho.di.ui.spoon.Spoon.readAndDispatch(Spoon.java:1359)
at org.pentaho.di.ui.spoon.Spoon.waitForDispose(Spoon.java:7990)
at org.pentaho.di.ui.spoon.Spoon.start(Spoon.java:9290)
at org.pentaho.di.ui.spoon.Spoon.main(Spoon.java:685)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.pentaho.commons.launcher.Launcher.main(Launcher.java:92)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.vfs.FileSystemException
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at org.pentaho.di.core.plugins.KettleURLClassLoader.loadClassFromParent(KettleURLClassLoader.java:89)
at org.pentaho.di.core.plugins.KettleURLClassLoader.loadClass(KettleURLClassLoader.java:108)
at java.lang.ClassLoader.loadClass(Unknown Source)
... 28 more

How can this be solved?

Best,

Trying to make it work with PDI 8.1 (on webspoon)

The first error I get is:
java.lang.NoClassDefFoundError: org/apache/commons/vfs/FileSystemException

This is fixed by adding commons-vfs-1.0.jar, but following error appears:
java.lang.NoSuchMethodError: org.pentaho.di.core.vfs.KettleVFS.getFileObject(Ljava/lang/String;)Lorg/apache/commons/vfs/FileObject;

Any suggestions what could be done to make this work ?

I have installed last version of PDI community and compiled PDI GIS Plugins with last version of maven for windows 32 bits, and jdk1.7.0_79

Compilation do break all it is correct and generate zip file with all of the pluging content.
After uncompress it in PDI_home/plugins/step and opening PDI Geospatial steps appear in PDI user interface.
I can make a new transformation and when i to concat a Coordinated system operation with an input step and try to open properties of that step a error windows appear.
The first time of that error message title:
I was unable to create a dialog for step com.atolcd.pentaho.di.ui.trans.steps.giscoordinatetransformation.GisCoordinateTransformationDialog
And content:
loader constraint violation: when resolving method "org.slf4j.impl.StaticLoggerBinder.getLoggerFactory()Lorg/slf4j/ILoggerFactory;" the class loader (instance of org/pentaho/di/core/plugins/KettleURLClassLoader) of the current class, org/slf4j/LoggerFactory, and the class loader (instance of java/net/URLClassLoader) for resolved class, org/slf4j/impl/StaticLoggerBinder, have different Class objects for the type gerBinder.getLoggerFactory()Lorg/slf4j/ILoggerFactory; used in the signature

The second on time trying to open properties of the step the message title is the same but the content is:
Could not initialize class org.cts.registry.RegistryManager

What can be the problem?

Best regards,
Miguel A. Manso

Lors de l'extraction d'une table Geopackage, PDI me retourne cette erreur sur un champ date :
Error reading features :Unparseable date: "Thu Aug 31 02:00:00 CEST 2023"
2024/05/02 11:41:14 - Extraction depuis fichier SIG.0 -
2024/05/02 11:41:14 - Extraction depuis fichier SIG.0 - at com.atolcd.pentaho.di.gis.io.GeoPackageReader.getFeatures(GeoPackageReader.java:329)
2024/05/02 11:41:14 - Extraction depuis fichier SIG.0 - at com.atolcd.pentaho.di.trans.steps.gisfileinput.GisFileInput.processRow(GisFileInput.java:126)
2024/05/02 11:41:14 - Extraction depuis fichier SIG.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2024/05/02 11:41:14 - Extraction depuis fichier SIG.0 - at java.lang.Thread.run(Unknown Source)

J'ai tenté tous les formats mais l'erreur est invariable. Quel format date doit-on fournir dans me gpkg pour qu'il soit pris en compte par le plugin ?
Merci

The step "GIS Input File" with type "Shapefile" does not recognize the DBF file if the SHP file has uppercase. So, it's only recognized the geometry column and other columns are omited.

It occurs only in linux enviroment because of the file system is case sensitive.

Using pan.sh from PDI v8.3 and Gis Plugin v1.3 and v1.4 (Linux version).

Hi,

First, thanks you for your great work !

Could you add an example folder with transformations and data files, this will be very useful to help starting using your plugin.

Thanks a lot.

Pentaho 7.1 et Plugin-gis 1.2
En ouvrant des transformations migrées de la version 5.4, le format de type geometry n'existe plus, notamment dans les steps Altération Structure Flux

Recently I built a transformation that featured the GIS File Input step to load an Esri shapefile file format into the datastream. Using the transformation and storing the transformation as a .KTR-file work fine.

However, we are long-time users of a central Kettle repository. I stored the transformation in a Kettle repository without any issues or warnings, but when I tried to open it later it throws a null error and fails to retrieve the transformation from the repository. So I tried all steps in the plugin and found the following.

1. The steps GIS File Input and GIS File Output will store in the repository but will throw an error when you try to open the transformation afterwards, making the transformation unusable.

   The exact error reads:
   "An unexpected error occurred in Spoon:
   null"

2. The step Geoprocessing will not even store the transformation in the repository but will throw an error immediately

"Unable to save step info to the repository for id_transformation=684

Unable to write step info in repository
at org.eclipse.jface.operation.ModalContext$ModalContextThread.run (ModalContext.java:113)
at org.pentaho.di.ui.spoon.dialog.SaveProgressDialog$1.run (SaveProgressDialog.java:70)
at org.pentaho.di.repository.AbstractRepository.save (AbstractRepository.java:126)
at org.pentaho.di.repository.kdr.KettleDatabaseRepository.save (KettleDatabaseRepository.java:387)
at org.pentaho.di.repository.kdr.KettleDatabaseRepository.save (KettleDatabaseRepository.java:407)
at org.pentaho.di.repository.kdr.delegates.KettleDatabaseRepositoryTransDelegate.saveTransformation (KettleDatabaseRepositoryTransDelegate.java:278)
at org.pentaho.di.repository.kdr.delegates.KettleDatabaseRepositoryStepDelegate.saveStepMeta (KettleDatabaseRepositoryStepDelegate.java:248)
at org.pentaho.di.repository.kdr.delegates.KettleDatabaseRepositoryStepDelegate.compatibleSaveRep (KettleDatabaseRepositoryStepDelegate.java:274)
at com.atolcd.pentaho.di.trans.steps.gisgeoprocessing.GisGeoprocessingMeta.saveRep (GisGeoprocessingMeta.java:370)"

All steps work fine when used in a file-based environment. This has been tested on:

• PDI 5.4.0.1-130 (as I understand there are issues regarding PDI 6 and the plugin.
• MS SQL Server 2008 R2 for the repository.

Would PostGIS support be a reasonable thing to add? Perhaps I could lend a hand.

First of all thank you for this great extension.

I am new in pentaho and I am trying to manage spatial data. I have installed your extension both from marketplace and manually but I cannot see the option in the input file to open a Geopackage file. Did I miss something?

Thanks

The plugin throws a null pointer error when it encounters a GeoJSON property which is null or an empty object:

"properties": {
  "property": null
}

or

"properties": {
  "property": {}
}

The exception as seen in the Kettle log:

2020/07/20 01:04:24 - GIS File input.0 - ERROR (version 8.3.0.0-371, build 8.3.0.0-371 from 2019-06-11 11.09.08 by buildguy) : Unexpected error
2020/07/20 01:04:24 - GIS File input.0 - ERROR (version 8.3.0.0-371, build 8.3.0.0-371 from 2019-06-11 11.09.08 by buildguy) : java.lang.NullPointerException
2020/07/20 01:04:24 - GIS File input.0 - 	at com.atolcd.pentaho.di.gis.io.features.FeatureConverter.getRowMeta(FeatureConverter.java:149)
2020/07/20 01:04:24 - GIS File input.0 - 	at com.atolcd.pentaho.di.trans.steps.gisfileinput.GisFileInputMeta.getFields(GisFileInputMeta.java:222)
2020/07/20 01:04:24 - GIS File input.0 - 	at com.atolcd.pentaho.di.trans.steps.gisfileinput.GisFileInput.processRow(GisFileInput.java:65)
2020/07/20 01:04:24 - GIS File input.0 - 	at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2020/07/20 01:04:24 - GIS File input.0 - 	at java.lang.Thread.run(Thread.java:748)

Using Kettle 8.3.0.0-371 and GIS Plugin V1.3.0

I'm trying to do a coordinate system conversion over a field through Coordinate System Operation (before: EPSG:4019, after: ESRI:102015), but I'm getting an error at the subsequent Table Output step. I've tried to manually do the INSERT INTO operation at the PostgreSQL terminal, after getting the fields values at the PDI Logging tab. The reason is that the content the Geometry field have generated to the query used at Table Output is POINT (...) without specifying the SRID before -- when I put SRID=102015;POINT (...) it works. Otherwise, I get the same error the Logging tab show:

Geometry SRID (0) does not match column SRID (102015)

In other words, without explicitly putting SRID=...;, PostGIS thinks the SRID is 0.

One possible solution is manually insert the SRID before the SQL command, but I'm having trouble in trying to convert the Geometry field into String field -- even the toString() method doesn't work!

Can anybody suggests anything?

While using your plugins from the marketplace, I noticed that there is no way of specifying distance in KM or Miles in the GeoProcessing plugin. The default seems to be the units that the geometry comes in, so for most cases it would be degrees. From a usability standpoint, every user that uses this plugin must do a conversion to distance units.
Is this something you might consider adding in the near future?

Le Geotraitement Centroide renvoi invariablement un message d'erreur

Géotraitement.0 - ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : Erreur inattendue
2017/08/16 11:11:55 - Géotraitement.0 - ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : java.lang.ArrayIndexOutOfBoundsException: -1
2017/08/16 11:11:55 - Géotraitement.0 - at com.atolcd.pentaho.di.trans.steps.gisgeoprocessing.GisGeoprocessing.processRow(GisGeoprocessing.java:337)
2017/08/16 11:11:55 - Géotraitement.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2017/08/16 11:11:55 - Géotraitement.0 - at java.lang.Thread.run(Thread.java:748)

PDI ; Version 7.1
Plugin GIS : 1.2.1

Pentaho : 8.0.0.0-28
PDI GIS plugin : 1.2.1

Bonjour. Je viens de tester une transformation toute simple composée d'un import d'une table PostGIS et d'un export de fichier shape (GIS File Output). L'erreur suivante est apparue :

2018/03/19 09:33:16 - GIS File output.0 - ERROR (version 8.0.0.0-28, build 8.0.0.0-28 from 2017-11-05 07.27.50 by buildguy) : Unexpected error
2018/03/19 09:33:16 - GIS File output.0 - ERROR (version 8.0.0.0-28, build 8.0.0.0-28 from 2017-11-05 07.27.50 by buildguy) : java.lang.NoSuchMethodError: org.pentaho.di.core.vfs.KettleVFS.getFileObject(Ljava/lang/String;)Lorg/apache/commons/vfs/FileObject;
2018/03/19 09:33:16 - GIS File output.0 - 	at com.atolcd.pentaho.di.gis.io.AbstractFileWriter.checkFilename(AbstractFileWriter.java:63)
2018/03/19 09:33:16 - GIS File output.0 - 	at com.atolcd.pentaho.di.gis.io.ShapefileWriter.<init>(ShapefileWriter.java:77)
2018/03/19 09:33:16 - GIS File output.0 - 	at com.atolcd.pentaho.di.trans.steps.gisfileoutput.GisFileOutput.processRow(GisFileOutput.java:73)
2018/03/19 09:33:16 - GIS File output.0 - 	at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2018/03/19 09:33:16 - GIS File output.0 - 	at java.lang.Thread.run(Unknown Source)

Du coup, je me demande si le plug-in est compatible avec ma version de pdi ou non ?

I'm using kettle version 6 and i can't install "PDI GIS Plugins". I try to install another plugin using marketplace and it works fine but when i try to install PDI GIS Plugins it fails occuring the following statement:
Error occurred when installing plugin PDI GIS Plugins.[ Failed to execute install, see log for details.]

Unfortunately i couldn't find where the log is to know which error is raising. It is available PDI GIS Plugins to kettle PDI v6? How can i install that?

I tried to install in Kettle PDI v5.4 and they couldn't find https://github.com/atolcd/pentaho-gis-plugins/releases/download/v1.1-snapshot/pentaho-gis-plugins-1.1-SNAPSHOT-bin.zip maybe should be https://github.com/atolcd/pentaho-gis-plugins/archive/v1.0-snapshot.zip

While trying to write out geometries to a file or using in Pentaho's group by step, I encountered an issue with the Geometry column in preview data tab having a conversion error.

dim | point
2 | Conversion error: point Geometry : Unknown type 43663879 specified.

I have attached a KTR file that demonstrates the issue along with sample data.
Please change the file extension of test_5.4.txt to ktr (github does not allow attaching of ktr files).

KTR File: test_5.4.txt
Test Data: data.txt

If different instance of a property have different numeric types (integer, decimal) then a type conversion error results:

For instance:

{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "geometry": {
        "type": "Point",
        "coordinates": [
          102.0,
          0.5
        ]
      },
      "properties": {
        "property1": 1.0
      }
    },
    {
      "type": "Feature",
      "geometry": {
        "type": "Point",
        "coordinates": [
          100.0,
          0.5
        ]
      },
      "properties": {
        "property1": 2
      }
    }
  ]
}

causes

2020/07/20 01:53:55 - GIS File input.0 - ERROR (version 8.3.0.0-371, build 8.3.0.0-371 from 2019-06-11 11.09.08 by buildguy) : Unexpected error
2020/07/20 01:53:55 - GIS File input.0 - ERROR (version 8.3.0.0-371, build 8.3.0.0-371 from 2019-06-11 11.09.08 by buildguy) : org.pentaho.di.core.exception.KettleValueException: 
2020/07/20 01:53:55 - GIS File input.0 - Unexpected conversion error while converting value [property Number] to a Number
2020/07/20 01:53:55 - GIS File input.0 - java.lang.Integer cannot be cast to java.lang.Double
2020/07/20 01:53:55 - GIS File input.0 - 
2020/07/20 01:53:55 - GIS File input.0 - 	at org.pentaho.di.core.row.value.ValueMetaBase.getNumber(ValueMetaBase.java:2035)
2020/07/20 01:53:55 - GIS File input.0 - 	at com.atolcd.pentaho.di.gis.io.features.FeatureConverter.getRow(FeatureConverter.java:217)
2020/07/20 01:53:55 - GIS File input.0 - 	at com.atolcd.pentaho.di.trans.steps.gisfileinput.GisFileInput.processRow(GisFileInput.java:106)
2020/07/20 01:53:55 - GIS File input.0 - 	at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2020/07/20 01:53:55 - GIS File input.0 - 	at java.lang.Thread.run(Thread.java:748)

I believe the cause of this error is the assumption in the GeoJSONReader constructor (line 68):
org.wololo.geojson.Feature geoJsonfeature = ((FeatureCollection) json).getFeatures()[0];

which determines the structure of GeoJSON properties by looking at only the first feature. This will also cause any property that does not appear in the first record (or has a null value ) to be ignored. (see #22 for an issue handling null property values)

Using Kettle 8.3.0.0-371 and GIS Plugin V1.3.0

Good day,

I’m using the Pentaho GIS Plugins to convert a spatial PostGIS table to a shapefile. Input Data is in EPSG 25830 (Spain - ETRS 89 – UTM Zone 30N).

The Kettle transformation is quite simple (input table >> Order >> Mapping >> GIS File Output as SHP) and works fine.

However the Coordinate Reference System of the output layers is not recognized by the most recent version of QGIS (I’m using QGIS v.3.14.15).

Taking a look at the .*prj file generated by the transformation, I can see that the definition of the SRS is the following

PROJCS["ETRS89 / UTM zone 30N",GEOGCS["RGF93",DATUM["Réseau géodésique français 1993",SPHEROID["GRS 1980",6378137.0,298.257222101,AUTHORITY["EPSG","7019"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY["EPSG","6171"]],PRIMEM["Greenwich",0.0,AUTHORITY["EPSG","8901"]]],PROJECTION["Transverse Mercator Zoned Grid System"],PARAMETER["latitude of origin",0],PARAMETER["central meridian",-3],PARAMETER["scale factor",1],PARAMETER["false easting",0],PARAMETER["false northing",0],UNIT["meter",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","25830"]]

And the definition which is understandable by QGIS looks like

PROJCS["ETRS_1989_UTM_Zone_30N",GEOGCS["GCS_ETRS_1989",DATUM["D_ETRS_1989",SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",-3.0],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]

Is correct the SRS corresponding to the EPSG 25830 in the plugin?

Configuration:
PDI - Kettle 9.0;
QGIS : 3.14.15
Plugin version: 1.4.0-bin-9

Tks for your effort