Monday, 29 June 2015

Using GDAL in ArcGIS GP service

Working on a cleaner solution...



The problem:
There is a python script provided by the client, which works in a UNIX environment. I need to set it up as an ArcGIS GP service running on windows 64.
The script heavily relies on GDAL. there are direct call to osgeo with 'from osgeo import gdal,ogr,osr' and command line call with 'os.system'

From a java background, the python environment & dependencies system looks really messy to me, but could just be me. I struggled for quite some time to get it work. Here are the steps:

1) install GDAL tools
Install GDAL tools with https://trac.osgeo.org/osgeo4w/
Run the installer and install the tools you need.
Now all command line tools like ogr2ogr will work fine within its own shell. But ArcGIS has its own python, so there is more work to do.

2) to use GDAL directly in python
http://www.lfd.uci.edu/~gohlke/pythonlibs/#gdal
This is a super cool website in my opinion.  Things just work. You will get .whl files from it.
Install gdal & numpy. (Pick 32 or 64 based on the ArcGIS python instance. On my server there are two instances under 'C:\Python27\' and the ArcGIS server is using ‘ArcGISx6410.2’)
Now we install them to the arcgis python.  Set the path so in command line you are using the right python.  ( type ‘where python’ in command line)

Install pip if it’s not there:
https://raw.github.com/pypa/pip/master/contrib/get-pip.py
Run   python get-pip.py
Then you might need to add ‘C:\Python27\ArcGISx6410.2\Scripts’ to the system path (edit the system environment variables). Restart the console.
Now use   Pip install [filename] to install whl files.

Here I have to install the latest numpy. The existing numpy on ArcGIS is too old.  

Now this should work in python:
from osgeo import gdal,ogr,osr

3) to get everything works as GP service
Now the script works in command line, but it didn’t work as GP service. I figured the ArcGIS server doesn't seem to pick up windows environment. So the last step is setting up environment in the python script,  
I ended up with this:
#for finding module
sys.path.append(r"C:\Python27\ArcGISx6410.2\Lib\site-packages")
#system path
os.environ['PATH'] += os.pathsep + r'D:\mapserver\bin'
os.environ['PATH'] += os.pathsep + r'D:\ mapserver\bin\gdal\apps'
os.environ['GDAL_DATA'] = r'D: \mapserver\bin\gdal-data'

Here os.environ['PATH'] is used to set up the environment variables for console command. Because in my script there are lines like:
os.system('ogr2ogr -t_srs EPSG:3577 …’) 



Wednesday, 24 June 2015

Another year

Oh man, time does fly.It has been three years I haven't written anything here.

Now I am a solutions architect working in AAM, not Dialog anymore. And I mainly use python, JavaScript, and C#.

Oh how I miss JAVA, it's like my first language. Technically speaking, I learned BASIC when in high school, then C which I used to finish most of my homework in uni, then Pascal (delphi), with which I wrote desktop application in my first job. but JAVA was something I really loved and enjoyed. After graduated for three years, I quit the SAP consultant job and bought the book "Thinking in JAVA".  I feel that is the real start of my professional programming. From that day, I am not content to just solving the problem, I started to think a lot about how to code elegantly. I can truly appreciate the beauty of code. It opened a new world to me.

However JavaScript is the new king now. And I enjoy coding in JavaScript because it gives me more freedom and the code is shorter. And python, I think it might be the best language for personal programming.

I started smap project in 2011, and now it is 2015, The first version is GWT, then I implemented nearly the same functions a pure JavaScript, and now the latest version is written in Typescript. Why? because I can :)    http://shen.apphb.com/map/demo/basic.html . Look at it I think I am pretty good. Seems like programming is my strongest skill. haha. Good Work Shen!