Using GDAL in ArcGIS GP service

Working on a cleaner solution...

The problem:

There is a python script provided by the client, which works in a UNIX environment. I need to set it up as an ArcGIS GP service running on windows 64.

The script heavily relies on GDAL. there are direct call to osgeo with 'from osgeo import gdal,ogr,osr',  and command line call with 'os.system'

From a java background, the python environment & dependencies system looks really messy to me, but could just be me. I struggled for quite some time to get it work. Here are the steps:

1) install GDAL tools

Install GDAL tools with https://trac.osgeo.org/osgeo4w/

Run the installer and install the tools you need.

Now all command line tools like ogr2ogr will work fine within its own shell. But ArcGIS has its own python, so there is more work to do.

2) to use GDAL directly in python

http://www.lfd.uci.edu/~gohlke/pythonlibs/#gdal

This is a super cool website in my opinion.  Things just work. You will get .whl files from it.

Install gdal & numpy. (Pick 32 or 64 based on the ArcGIS python instance. On my server there are two instances under 'C:\Python27\' and the ArcGIS server is using ‘ArcGISx6410.2’)

Now we install them to the arcgis python.  Set the path so in command line you are using the right python.  ( type ‘where python’ in command line)

Install pip if it’s not there:

https://raw.github.com/pypa/pip/master/contrib/get-pip.py

Run   python get-pip.py

Then you might need to add ‘C:\Python27\ArcGISx6410.2\Scripts’ to the system path (edit the system environment variables). Restart the console.

Now use   Pip install [filename] to install whl files.

Here I have to install the latest numpy. The existing numpy on ArcGIS is too old.  

Now this should work in python:

from osgeo import gdal,ogr,osr

3) to get everything works as GP service

Now the script works in command line, but it didn’t work as GP service. I figured the ArcGIS server doesn't seem to pick up windows environment. So the last step is setting up environment in the python script,  

I ended up with this:

#for finding module

sys.path.append(r"C:\Python27\ArcGISx6410.2\Lib\site-packages")

#system path

os.environ['PATH'] += os.pathsep + r'D:\mapserver\bin'

os.environ['PATH'] += os.pathsep + r'D:\ mapserver\bin\gdal\apps'

os.environ['GDAL_DATA'] = r'D: \mapserver\bin\gdal-data'

Here os.environ['PATH'] is used to set up the environment variables for console command. Because in my script there are lines like:

os.system('ogr2ogr -t_srs EPSG:3577 …’) 

Another year

Oh man, time does fly.It has been three years I haven't written anything here.

Now I am a solutions architect working in AAM, not Dialog anymore. And I mainly use python, JavaScript, and C#.

Oh how I miss JAVA, it's like my first language. Technically speaking, I learned BASIC when in high school, then C which I used to finish most of my homework in uni, then Pascal (delphi), with which I wrote desktop application in my first job. but JAVA was something I really loved and enjoyed. After graduated for three years, I quit the SAP consultant job and bought the book "Thinking in JAVA".  I feel that is the real start of my professional programming. From that day, I am not content to just solving the problem, I started to think a lot about how to code elegantly. I can truly appreciate the beauty of code. It opened a new world to me.

However JavaScript is the new king now. And I enjoy coding in JavaScript because it gives me more freedom and the code is shorter. And python, I think it might be the best language for personal programming.

I started smap project in 2011, and now it is 2015, The first version is GWT, then I implemented nearly the same functions a pure JavaScript, and now the latest version is written in Typescript. Why? because I can :)    http://shen.apphb.com/map/demo/basic.html . Look at it I think I am pretty good. Seems like programming is my strongest skill. haha. Good Work Shen!

My new showcase

http://smapshow.appspot.com

That is to prove that GWT app can just look like a normal commercial web site.  And it also contains my SMap showcase.

Been quite busy recently, I have stopped developing SMap for a while, but evey time I look at her I still feel so proud.  I think I can make it better than Openlayers in the future, consider I am the only developer I would say I have made a very good progress!

GWT Spring Hibernate/JPA (part 2)

Now we will add spring framework to this project. Normally I use maven to manage j2ee projects,but with GWT, I've spent a lot of time(with the GWT maven plugin) and still can't make the IDE and maven work together very well. So I decided only use maven to manage the dependencies and leave the rest to eclipse.


Attached file is my demo project(windows, eclipse3.7 with google plugin) , I think a experienced programmer would prefer trying that first. The demo also includes some design/widgets I created based on my experience and needs.
It should work straight away. (rightclick->run as->web application) .

in /libs there is the pom.xml with all dependency. Inside this file you can see
<outputDirectory>../war/WEB-INF/lib</outputDirectory>
which pointing to the output directory.

you can run mvn process-resources, that will fetch all dependency jars to war/WEB-INF/lib.

In WEB-INF, I put 3 xml files in the spring folder as usual. and web.xml needs to modify. (see my demo)

you may need to change <context:component-scan> to specify the package to scan.
In datasource.xml , this line tells spring to look for "config.properties" in classpath
<context:property-placeholder location="classpath:config.properties" />

I feel there is no need to explain normal spring  stuffs,  let me know if you have questions about my demo.

GWT Spring Hibernate/JPA (part 1)

In eclipse, using Dynamic Web Project is the most easy way to develop/debug a spring powered j2ee project. However when developing GWT apps, it's a shame not to use google's eclipse plugin, which will set the project as a "Web Application Project".

The first part will tell you how to create a eclipse project has both facets.

First you need to have the Eclipse IDE for Java EE Developers (http://www.eclipse.org/downloads/)

1 create a google GWT project with Google eclipse plug-in. (install it if you haven't https://developers.google.com/eclipse/ )

2 right click project,  properties-> project facet-> convert to facet form -> check dynamic web module facet in the list.

3 using Windows Explorer, navigate to the project folder -> ./settings -> org.eclipse.wst.common.component, open this file with any text editor,

modify this line:
<wb-resource deploy-path="/" source-path="/WebContent" tag="defaultRootSource"/>
to
<wb-resource deploy-path="/" source-path="/war" tag="defaultRootSource"/>
then refresh the project in eclipse.



After these simple steps the project is configured as google gwt project and also a WTP project.
That allows this project to be run/debugged as a GWT app or a normal web app. (right click->run as)

what will the feature hold

I'v given up fedora, looks like lubuntu is so fat the best  for me.

And my map widget is in Beta . http://smapshow.appspot.com , I will start building some showcase when I get time.

Now it supports WMS and part of WFS.  In terms of WFS. The short term goal is the function to edit layers from geo server. Feel a little bit sad though, since no one or company supports me to do the thing I am good at and interested in, and I will never have chance to lead a team to achieve something really remarkable. People just want to do the easy things and get paid, and who can we blame.

I will invest some time on GWT styling(theme). The GWT should have been more popular, if it had some nice themes. I really admire the flexibility of GWT, it's quite extraordinary comparing other RIA framework. People may miss the point that in fact With GWT, we can create any types of web page, not necessarily to be desktop-like.

Fedora 15, the ultimate tweaks

The ultimate tweaks to make Genome3 work as you want
http://intgat.tigress.co.uk/rmy/extensions/index.html