Building netcdf4, netcdf4(python) and nco on Ubuntu

Built HDF as

$ ./configure —enable-netcdf-4 —with-hdf5=/usr/local/hdf5 —enable-share


Built netcdf4 as

$ ./configure —prefix=/usr/local/hdf5 —enable-hl —enable-shared


Configure fails for nco trying to link to netcdf. The Ubuntu nco package doesn’t want to read netcdf4 files - it is using a version of netcdf it installs in /usr/lib (the one I built is in /usr/local/lib).

Tried recompiling everything with -fPIC flag, and tried many different ways of specifiying where the netcdf libaries I built in /usr/local were

./configure –enable-netcdf4 –libdir=’/usr/local/lib’ –prefix=’/usr/local’ NETCDF4_ROOT=’/usr/local’ CC=’gcc’ CFLAGS=” –disable-dap NETCDF_LIB=’/usr/local/lib’ NETCDF_ROOT=’/usr/local’

No dice - configure failed to successfully link to libnetcdf

In a final act of desperation I set NETCDF4_ROOT to /usr instead of /usr/local, linking to the netcdf4 libraries installed by synaptic. Configure works, but make still picks up ncgen in /usr/local instead of /usr. So I set PATH=’/usr/bin:/bin. Still no dice. In the end I moved /usr/local to /usr/local_temp, and ran:

sudo ./configure –enable-netcdf4 –libdir=’/usr/lib’ –prefix=’/usr’ NETCDF4_ROOT=’/usr’ CC=’gcc’ –disable-dap NETCDF_LIB=’/usr/lib’ NETCDF_ROOT=’/usr’ PATH=’/usr/bin:/usr/sbin:/bin’ –disable-shared

Got further with this approach, but now it fails when building ncap2, both with the latest cvs and the 3.9.5 release code.

Needed to install anltr. Used synaptic. Build went fine after this, but ncks still gives me this error

ac@LOULI:~/runs/lou-10$ ncks -d timestep,0,1 lou-10.nc
ERROR NC_ENOTNC Not a netCDF file
HINT: NC_ENOTNC errors often occur when NCO operators linked to the netCDF3 library attempt to read netCDF4 files. Are your input files netCDF4 format? If so then installing or re-building a netCDF4-compatible version of NCO may solve this problem.
nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nco_open()
nco_err_exit(): ERROR Error code is -51. Translation into English with nc_strerror(-51) is “NetCDF: Unknown file format”
nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)


when run on netcdf4 files. Works ok on netcdf3 files though. At this point I believe the possible problems are:

1. I’m not building the netcdf libraries in /usr/local correctly?
- they work for creating, ncdumping, etc, maybe not built properly for nco to link to.
2. The ubuntu netcdf4 libraries don’t really work for netcdf4 files?
- they certainly appear not to

sudo ./configure –enable-netcdf4 –libdir=’/usr/local/lib:/usr/lib’ –prefix=’/usr/local’ NETCDF4_ROOT=’/usr/local’ CC=’gcc’ –disable-dap NETCDF_LIB=’/usr/lib’ NETCDF_ROOT=’/usr’ PATH=’/usr/bin:/usr/sbin:/bin:/usr/local/bin’ –disable-shared


I need to leave this for now - the ideal situation is that the packages work out of the box, so the next step is to reinstall the defaulty packages, try to ncks some netcdf4 files from elsewhere, and file a bug report if they don’t work.

Reversing a numpy array in python (CDAT)

All this and more at the Johnny Lins’ excellent cdat tips page. Code for one and two dimensions below:


a = a[::-1]
b = a[:,::-1]


cdat installation

Attempting to install cdat (alpha source) into my existing macpython installation.

By all appearances all i need to do is unset PYTHONPATH and PYTHONHOME, and run python install.py.

Seemed to work - vcs and one other package failed to install, so I’ll have to revisit that. The test script doesn’t seem to have installed where the message tells me it goes (” To Test CDAT, please run:
/Library/Frameworks/Python.framework/Versions/2.5/bin/python test_cdat.py -v3 -P
and then
/Library/Frameworks/Python.framework/Versions/2.5/bin/python test_cdat.py -v3 -C –skip
You can add the option “–full” at the end of these lines for FULL testing
For more info on test_cdat.py functionalities run:
/Library/Frameworks/Python.framework/Versions/2.5/bin/python test_cdat.py –help

“), so I’ll need to look into that too. The vcs log file indicates that it failed to find cairo, which might be a problem with my cairo setup. SP also failed - haven’t checked the log.

I can import cdms2, but all my production scripts use cdms - I’ll need to ask about backwards compatibility.

SPH rendering in pyticles

Just committed a new version of pyticles with SPH rendering implemented.

SVN cheat sheet

Create new repo:
svnadmin create /path/to/repository

Add directory to repo
svn import project file:///repository_name/project -m “First Import”

Check out directory
svn checkout file:///repository_name/project/trunk project

Numpy, Scipy, Matplotlib rebuild recipie (Leopard)

The steps I used to refresh the trinity of python packages goes something like:
Nuke all old versions. *ALL* old versions! Fortune favours the bold - of course it will work!

svn co http://svn.scipy.org/svn/numpy/trunk numpy
cd numpy
python setup.py build
sudo python setup.py install



svn co http://svn.scipy.org/svn/scipy/trunk scipy

or just svn update if I’m feeling lazy. Then in the scipy source directory,

setenv MACOSX_DEPLOYMENT_TARGET 10.5
python setup.py build_src build_clib –fcompiler=gnu95 build_ext –fcompiler=gnu95 build
sudo python setup.py install

Not sure if the deployment target variable works fro 10.5, but it seems logical.

Then get the matplotlib egg (it was a pita to build the first time so screw that), and

sudo easy_install matplotlib-0.98.0-py2.5.egg


These notes are for me, for quality advice always use the scipy site (below) and the mailing lists. To do: build optimised atlas, lapack etc and configure scipy to use these.
http://www.scipy.org/Installing_SciPy/Mac_OS_X

Entropy - an afternoon’s saga

I started this afternoon somewhat ignorant about the state of thinking and debates concerning entropy and its applicability to economics. Following a link from Michael Tobis’ blog I discovered Herman E Daly, who appears to be a fine thinker. Daly was taught by Nicholas Georgescu-Roegen, whose book “Entropy and Economic Processes” appears to be the seminal work in the application of thermodynamics to economic problems. Let me say at this point that I am sure there is no end of misunderstanding and misapplication of theromodynamics, which is possibly one of the most conceptually difficult fields in physical science, to field outside of physics. Nevertheless, I think there is merit in the attempt.

Reading reviews of GR’s book on amazon, I found a critical review questioning the thermodynamic basis of his work, based on his description of entropy as a measure of disorder. Examining other reviews by the author I discovered a handful of other reviews with exactly the same text, for various other books on environmental economics. This led me to the site of one Frank Lambert, who is on a crusade to stop people thinking about entropy as disorder. I haven’t spent a lot of time reading his articles and so on, but on first inspection he seems to be overly concerned with semantics. That is his concern is that using the word disorder leads to misunderstandings. I’ve found his examples to be a bit weak, and the style to be hectoring.

I still think about entropy qualitatively as disorder - a high entropy system could be in any one of a large number of states, and requires more information to specify the state completely. A low entropy system is in one of fewer possible microstates and this requires less information to specify. For example a regular grid can be specified by three numbers: resolution, x dimension and y dimension. A random arrangement of points requires 2n numbers to specify.

Looks like there’s a bit of a Wikipedia edit war happening over this at the moment. The more I read of the anti-disorderati’s work the more I find it annoying, hectoring, and feel i’m being talked down to. I’ll stick to de Groot and Mazur for my precise definitions of entropy, i think.

I fail to see how “spreading out” and “disorder” are really that different conceptually.

Dark materials technology

This NS article outlines ideas for using artificial ‘daemons’ as identification. This is such a neat idea, based on what are probably the most literary (by that I mean deep, controversial, brilliantly written) childrens books I’ve ever read.

Lorenz

As well as his more famous work on chaos, Lorenz pioneered EOF analysis. What a champ. I must admit the EOF stuff seems a little impenetrable to me at the moment - web resources haven’t been especially clear. Nothing a good book can’t fix.

Soap bubbles

this new scientist article
got me thinking about things you could do with a 2d SPH model.