The Pragmatic Geographer

TileMill - what it does and some reasons to try it

2012-01-27T10:30:00.001-08:00

We were promised jetpacks, but I'll take Tilemill as a temporary replacement: http://mapbox.com/tilemill/

The MapBox/DevelopmentSeed team has created one of the the last pieces really needed for mainstream open source GIS to gain really massive appeal

TileMill is used for making web maps - or more specifically - for generating tiles that make up the now-ubiquiteous slippy maps we see online.

There are other desktop applications that do this, the most notable being ArcGIS Desktop. But Desktop was built for other things first: advanced analysis tools, some pretty powerful editing capabilities, and authoring paper maps.

TileMill does one thing and it does it well. It costs nothing (compared to several thousand for some flavor of ArcMap), and outputs an open tile format that you can wire up to a webmap or iPad in less time than it takes to install ArcMap.

And it is smooth. The user experience is the best I have had with a desktop application in a long while.

It also has sane, plaintext css-like styling (MSS). This may sound like a no-brainer, but your options before this were basically some proprietary binary format from ESRI (not extensible, difficult to automate, limiting, vendor specific) or SLD, which is open source but widely regarded as something of a mess for other reasons.

There is also the training issue. ArcMap is giant and powerful - and extremely complex. The market for "GIS Analysts" is still strong in a large part because of this complexity. Less experienced users will find TillMill easier to pick up and web designers (of which there is a large pool of talent) will find it very easy.

It is out for every operating system of note.

Seriously, go give it a try.

What else is needed

Conversion - minimal, well documented, mostly automated steps from ESRI - TileMill/FOSS

Samples like crazy - more or less emulate the ESRI samples, complete with documentation

Posted via email from The Pragmatic Geographer

HTML5 File System API - Basic Tips

2011-08-10T09:44:00.001-07:00

Recently went to an HTML5 Hackathon at Google Kirkland. My group's project was an in-browser IDE Chrome extension that zipped up a user-provided series of HTML/CSS/JS files into a package that could be uploaded to the Chrome Store. Issac Lewis came up with the idea after trying to develop chrome extensions on his chromebook and finding it basically impossible to do. Storing the files was a perfect use case for the FileSystem API, but I spent most of my time beating my head against the wall to get it working. Here are some of the things I wish I knew going in.

The FileSystem API is not LocalStorage.

LocalStorage is a key-value store, the FileSystem API really is an entire virtual file system, sandboxed on a user's local file system. You write, read, and create files async. It's also only implemented currently in Chrome. The documentation says 9+, but I hit errors until I switched from Chromium 12 to Chrome 13.
There's no limit to the storage, currently.

Hell yeah, cache all your map data on the user's local file system without needing an explicit download or local client built for it. That's a big deal for conditions or places with little to no connectivity. Also a big deal for massive games with a ton of art assets. They go through some good use cases here.
Debugging is a pain.

You will hit the dreaded SECURITY_ERR or QUOTA_EXCEEDED_ERR at some point, and it will be because debugging locally (file://) doesn't work well in my experience. The documentation suggests it's possible by opening Chrome with the --unlimited-quota-for-files and --allow-file-access-from-files flags, but my problems were only resolved when I started debugging as an extension rather than as a local file.

You also need to be careful about the flux the API is in. Throwing around BlobBuilder() and other pieces of the newer APIs can throw errors that can be difficult to track down. BlobBuilder didn't work for me, I needed window.WebKitBlobBuilder. That webkit prefixing shows up elsewhere as well (like window.webkitRequestFileSystem).

Feel no guilt in lifting gratuitously from the sample docs when starting out.

Async file access isn't really any wierder than any other browser async work, but there is some boilerplate code that is worth snapping up. Example:

 //error handling 
function errorHandler(e) {
  var msg = '';


  switch (e.code) {
    case FileError.QUOTA_EXCEEDED_ERR:
      msg = 'QUOTA_EXCEEDED_ERR';
      break;
    case FileError.NOT_FOUND_ERR:
      msg = 'NOT_FOUND_ERR';
      break;
    case FileError.SECURITY_ERR:
      msg = 'SECURITY_ERR';
      break;
    case FileError.INVALID_MODIFICATION_ERR:
      msg = 'INVALID_MODIFICATION_ERR';
      break;
    case FileError.INVALID_STATE_ERR:
      msg = 'INVALID_STATE_ERR';
      break;
    default:
      msg = 'Unknown Error';
      break;
  };


  console.log('Error: ' + msg);
}
//file system instantiation
window.requestFileSystem(window.PERSISTENT, 5*1024*1024 /*5MB*/, FSCreatedSuccess, errorHandler);

This kind of thing is okay starting out, but you'll want a lot more out of the error handling eventually. The message is fine, but the code tells you nothing about where the error occurred and in reference to what object or operation.

It's not CRUD, mostly.

Don't look for an explicit create method somewhere, the default is get or create via [filesystem_obj].[directory].get[Directory|File]. All reading, writing, and updating is probably going to live in a closure that starts with that first get.
Don't rush.

I made the mistake of looking at the limited time allocated and starting just throwing the example code in willy-nilly. This is not what you do with an unfamiliar and very new API. The typical help online is not there yet because it hasn't been used yet in a widespread way, throwing those error messages into google is not going to help you (unless that is how you got to this page, naturally). Start with the example code, sure, but I would carefully read the entirety of the short intro before trying random things to get it to work.

Posted via email from The Pragmatic Geographer

Some Must-See Development Summit videos

2011-03-21T09:35:00.001-07:00

In no particular order:

Making Apps that Don't Suck: UX Basics for GeoNerds (video)

Kirk Van Gorkum

Using the ArcGIS Flex API to Build Collaborative Mobile Applications Deployed on Multiple Platforms (Android and iOS)

Mansour Raad

one of his blog posts

HTML5: Not Just for Breakfast Anymore! (video)

Brian Noyle

Dave Bouwman

Mike Juniper

You Are Legend (with jQuery and the ArcGIS API for JavaScript) (video)

Creating (and sharing with you) a Vector Tile Cache for ArcGIS Server (video)

Dave Bouwman

Mike Juniper

Posted via email from The Pragmatic Geographer

Side Projects, Green Projects

2011-02-27T13:51:00.003-08:00

I love doing side projects. There are many reasons:

No better way to learn some technology than to put it into practice.
Can actually turn into a real thing, even a day job (a good example is Instapaper, which is the "killer app" for ebook readers like the Kindle as far as I am concerned).
Good for showing off to current and future employers, particularly if it is semi-related to your current domain.

I had read quite a bit of gushing on Ruby on Rails from Dave Bouwman, and I had a stupid idea for a web app that probably already existed, so I thought "why not"? I searched around and couldn't find what I had a picture in my head of.

This is what I wanted:

Simple two/three line webform where a user enters in where they are, what home improvement project they were looking at. They get back a list of what their local utility company would pay for and what contractors were available for it.

The utility information was basically public and there are actually quite a few conservation programs - every kilowatt hour saved is less that needs to be generated by building new, usually dirty power plants.

Contractors could add themselves and maybe pay to show up first. I wired up a prototype last weekend and showed a coworker who deals with conservation credits like these.

Her: "I like it."

Me: "Yeah I was going for something simple and direct like Hipmunk"

Her: *looks at Hipmunk* "Yeah, you just need an adorable mascot like they do"

Me: "Exactly! How about a big cartoony green brontosaurus with a bulbous nose?"

The next day, of course, I see a link via Twitter of a website that not only does more or less exactly what I was planning, but even had the mascot I was thinking of; only orange instead of green. The mascot bit might have been subconcious, because I remember reading a blog post from the same site (via hackernews) without ever looking at their homepage.

I'm not an expert in web design (mostly backend stuff), but I think I would prefer the simple form rather than their current landing page. I think it would be more focused on responding to what a homeowner is looking for. The energy savings calculator though, is probably the best I've seen in terms of visual style.

I might keep going with this project. The fact I couldn't easily find it on my initial search (my conservation coworkers hadn't heard of it either), the landing page, and the low barrier to entry (what do I have to lose, exactly?) means I could build something pretty quick I think.

Maybe not. I have a lot of other ideas for side projects - an open source outage management system has been rattling in my head since I've had to deal with a wonky one at work. That would be a great opportunity to do some high scalability/performance stuff outside of work.

Posted via email from The Pragmatic Geographer

Introducing (belatedly) nx_spatial

2011-02-27T13:51:00.001-08:00

It's been more or less done a while, but here is finally a blog post about it.

nx_spatial is a collection of addon functions for the networkx python graph library. What can you do with it?

Load GIS formats into networkx graphs (where you can do all sorts of crazy analytics on them)
Perform upstream and downstream traces with stopping points.
Set sources and find/repair edges that don't have the correct to/from nodes.

Example from the wiki:

>>> import nx_spatial as ns 
>>> net = ns.read_shp('/shapes/lines.shp') 
>>> net.edges() [[(1.0, 1.0), (2.0, 2.0)], [(2.0, 2.0), (3.0, 3.0)], [(0.9, 0.9), (4.0, 2.0)]] 
>>> net.nodes() [(1.0, 1.0), (2.0, 2.0), (3.0, 3.0), (0.9, 0.9), (4.0, 2.0)] 
>>> source = (2.0, 2.0) 
>>> ns.setdirection(net, source) 
>>> net.edges() [[(2.0, 2.0), (1.0, 1.0)], [(2.0, 2.0), (3.0, 3.0)], [(0.9, 0.9), (4.0, 2.0)]]

Available on pypi or bitbucket. Eventually I want to integrate it with networkx trunk (loading shapefiles is already in 1.4).

Posted via email from The Pragmatic Geographer

Random Hacks of Kindness

2010-12-14T12:36:00.001-08:00

I really didn't know what to expect.

Information on the official site and wiki was limited. The location, percise problem definitions, available resources, what we should bring - all quite hazy.

I don't think I could be much happier with how it turned out.

Random Hacks of Kindness (RHOK) is a weekend-long event in dozens of cities that puts volunteer programmers to problems identified by disaster relief experts. In Seattle, this took the form of around 20 people going after (1) mobile damage assessment and (2) identifying and providing information on populated places during disasters. Splitting off into groups, I joined the group working on the latter problem.

A decent portion of our first day was taken up with research and getting set up. The problem asked to pull information from Wikipedia for the populated regions, but the GeoNames API was vastly more suited for the task. It was decided to create a very simple Rails app that took as its input a know populated place. The user would then be prompted with a disambiguation page that includes a search radius option. The final output drops some very basic data about nearby populated places (at the moment, just the name and population), hopefully later to be expanded (haven't quite gotten around to that).

Lion's share of the credit goes to Kevin Moore and Ezekiel for doing much of the hacking. Being fairly new to Ruby I mostly helped with the spatial stuff - talking up GeoNames, OGC, and database help at the end (and in some controller tests we never really used).

Source code is up on github here.

Posted via email from The Pragmatic Geographer

Location based services are going to compete on trust, except Facebook

2010-09-30T08:34:00.001-07:00

Two articles recently came to my attention - one describing how location based services like FourSquare are used by some vanishingly small percentage of young people and basically no one else. This could be rather bad news for such services, as young men are a group that will try anything at least once regardless of its practicality or common sense.

The other article described how some mobile applications are secretly sending GPS coordinates to advertisers. This is both good and terrible news for developers of location based services. To me, it suggests two things:

They'll be competing on trust, which is remarkably difficult to do if you get big. Even if you are internally careful about the sharing of information there are third parties that can make your whole idea look like a bad one.
Quite a lot of money is being invested into location based services - startups like CloudMade and Facebook (Places) for example. Is it potentially because things like sending GPS coordinates and other personal information was what the funders were ultimately expecting in the future?

People will still use Facebook even if they find out it's been sending GPS coordinates and personal information to advertisers because everyone knows what they are getting into (or should). Everyone else has to worry about the incompatibility of competing on trust and potentially being funded on the notion that in the future you will be asked to betray your users.

Posted via email from The Pragmatic Geographer

WhereCampPDX 2010: Some of the Sessions

2010-09-28T08:03:00.001-07:00

It's going to take me some time to fully decompress and write this up, as a good portion of what I saw in Portland was new to me in some form. It is easy enough to read about the hacker community and fiddle with the cutting edge stuff like GeoCouch: it is quite another to actually meet the people and see what they are cooking up.

QTDesigner (Holly Glaser), GeoCouch/GeoJson (Max Ogden)

I hopped between sessions on this one, so I missed about half of Max's talk. The QT talk was small and had David Turner walking Holly through making a survey plugin.

The latter session was showed off an interop tool for converting shapefiles into GeoCouch and why you might want to use that datastore over say PostGIS. The larger point here, which was only briefly mentioned and which I'd like to examine more closely at a later date, is the assertion that simplicity in this sphere breeds adoption. I don't know I necessarily agree.

Domination of the Shapefile (James Fee)

I don't recall the real title, but the alt title was "Why I was Wrong in my last Keynote". It turns out Spatialite, though it was an improvement over the traditional method of bulk data delivery (masses of shapefiles), didn't have enough going for it to actually see mainstream adoption. There just isn't enough wrong with shapefiles to bother switching for data providers. Worse is simply better, at least in this case.

Still, there are problems in data interchange that need to be addressed, the big ones being (1) styling, (2) attribute relationships. I honestly don't see too big a problem with the shapefile being multiple files, since people have a tendency to zip up such things anyway and you probably want more than one shapefile at a time.

This was an interesting discussion and probably merits its own post, the examples were particularly illuminating (FGDC, GML, Gulf Spill map styling).

Keynote (Nathaniel Kelso)

Nathaniel works for the Washington Post making beautiful maps. He showed off examples - the accuracy of the coordinates for WhereCamp, Top Secret America, people estimation at the Obama inauguration - around the theme of "filling in" the map. Of an estimated 2.3 million habitated places we have good maps for a small fraction of them.

Multimodal Trip Planner (David Turner)

This talk is an odd intersection of network topology, real-time scheduling, dynamic mobile resources, and human behavior. When people say they only want to travel on bike lanes, does that mean they'll go five miles out of their way to do it? What if they want to bike only downhill by cleverly leveraging Portland's excellent mass transit system?

One thing I didn't bring up, and wish I had, is that this is an app that essentially lets people plan their travel around a relatively fixed mass transit schedule. What happens when everyone is using such information all the time? Would it be possible to start dynamically modifying the transit system itself to best integrate with user demands?

Geodata for the Masses (various)

I caught part of this talk late after briefly stopping in on the OSM editing and civic apps sessions. By the time I got there it appeared to have devolved into a talk on DIY data collection by kite and balloon.

Posted via email from The Pragmatic Geographer

"How do I get a GIS job?"

2010-08-11T12:40:00.001-07:00

I got this question from a recent grad and I'm not sure if I gave a good answer. Or, for that matter, if there is much advice that would help someone graduating into this job market. One theory going around is that timing like that can hurt forever.

This was my response:

What kind of GIS job do you want? I see the GIS field as roughly divided into administrative, technical, and special domain-specific knowledge areas. You can specialize and in some cases earn more, or diversify and be able to potentially apply for a larger number of positions. Everyone falls a little into all of these categories. Decide to what extent you wish to specialize, and in what area.

Administrative
Your project managers, leaders, and communicators. Even the nerdiest technical expert needs to be able to properly gather objectives and communicate requirements. The deeper you go into this, the more likely you will be delegating the actual GIS work.
You get better at this by getting better at written and verbal communication in general. I don't feel exceptionally qualified to talk about leadership, though increasingly I have been thrust into project management work.

Technical
GIS is a technical field. If you lack technical skills in a technical field you should fully expect to be mocked. This area can actually be further subdivided into the people leaning toward system admin/computer janitor work and dedicated developers.
I can recommend some fantastic books if you were interested in development - starting with The Pragmatic Programmer, CODE, Code Complete, and Programming Pearls. Just reading isn't enough, I would look into contributing to open source projects.

For the GIS admin/analyst types, I'd recommend working through the entirety of the book GIS for Web Developers. You can go from nothing to managing an entire open source GIS stack (which is fairly similar to ESRI's in terms of architecture).

Domain knowledge
Geographic information systems are usually serving some greater business need such as the mandates of public agencies or the profit center of a private corporation. Specialized knowledge of those needs are a career in themselves, and the more you specialize in this the more that career becomes yours. People that fall closer to this area exist on the fuzzy line between actual GIS professional and being a professional in something else.

I'm not going to claim I am the best authority on this particular topic, but I've had success with this particular mindset. A good discussion on what skills you need for each can be found on the new GIS -StakeExchange website here: http://gis.stackexchange.com/questions/883/gis-development-skills

Posted via email from The Pragmatic Geographer

Geometric Network Geoprocessing

2010-07-21T17:42:00.001-07:00

I've been hammering away at a GIS migration project that has been going on far longer than anyone expected it might take, but I've gotten the opportunity to hack away at some Python to more or less fix a problem I found on TheGISForum:

I currently have a geometric network that I've been working with using the Utility Network Analyst tools. However, I've been trying to perform a few tasks that I've noticed the Network Analyst tools provide, but the Utility Network Analyst does not.

Specifically, I want to place barriers at all locations specified by a point feature class - a functionality provided by the "Add Locations" tool in the Network Analyst Toolset.

Is there a way to use the network analyst tools on a geometric network?

The answer, sadly, is no. As far as I can tell, ESRI has not released any geoprocessing tools for their Geometric Network.

But it is possible to roll-your-own solution to get the same kind of tools by using existing open source libraries. I have a newbish work-in-progress project along these exact lines.

Posted via email from The Pragmatic Geographer

WAURISA 2010 Conference

2010-04-21T12:17:00.000-07:00

The Washington URISA conference is just about over, and it's made me realize how long it has been since I've gotten a chance to interact with the local GIS community. Some impressions:

The panels were probably the best part. I see now why the Barcamp/Unconference model has become so popular. Unfortunate no one from ESRI/AutoDesk/etc participated in these as far as I can tell - just the local/state government representatives and open source guys.

Best talk I heard was from Aaron Racicot on haiticrisismap.org and OSM. Even if you came in knowing the story, the energy of the speaker, quality of maps, and audience participation was excellent.

I was happy to see there wasn't too much stigma hopping between rooms to hit particular talks as long as you were quiet and professional about it.

It was nice to shake hands with some Well Known Nerds (WKN) of the open source GIS community.

Some excellent balance in conversations about the variety of solutions to a given problem. No one was especially partisan about a particular product, software, or method.

Vendors were not as aggressive as I had been warned.

The turnout of the CUGOS group was good. I didn't seen any equivalent ESRI or AutoDesk user group wandering around in a similar fashion.

Smart people giving poor presentations is heartbreaking. I know this might ring hollow coming from someone who didn't give a talk, but some folks should thumb through Presentation Zen again. Walls of text all over the place.

(credit)

Link Cleanup Time

2010-03-31T08:08:00.000-07:00

First, an adorable picture General nerd stuff
Android iTouch killer? $155 is cheap. Looking seriously at this or the Dell Mini 5. Vendor's site.
MIX 10: what Microsoft is doing this year.
AI book worthy of a read.
The Hacker, Architect, and Superhero. Only problem with this is the sushi pictures making me hungry.

Some programming stuff
Function Dependency Inversion - something about currying. Something else about F#.
Usability and Python; why it is a pretty good language for beginners even though it has some conventions not used in too many other languages. Also here is PyYAML. I haven't looked at it but people in irc were all hyper about it.
Parallel Programming is hard? I was certainly under that impression.
Let them eat code cake; Ruby ActiveRecord stuff.
Google Clean Code talks.
Some design patterns from MS: Data Persistence, Unit Of Work. Also some LINQ for beginners.
Regular expressions for people that don't know them or could use a reminder.

It's me, I'm a data janitor
Database change management.

The Web!
REST and preventing race conditions.

Local Open source geo in the northwest.

Spatial Metadata: Why is it so hard?

2010-02-18T09:45:00.001-08:00

Why can't my GIS data be as easy to organize/provide metadata for as my music collection?

I want a clean, detailed interface to explore my files. I want an indexed library for fast searches and simple ~~playlist~~ dataset/layer package/group/namespace creation. I want to change the owner, license, one-line description of files the same way I might change them in a Music folder in Windows Explorer, not by diving into some ugly wizard.

ESRI shouldn't throw out ArcCatalog, they should make it the iTunes of geospatial information. Move it from the smaller segmented market of GIS professionals to the mass market. Clean up the UI, throw up a "store" that connects with ArcGIS Online and the Resource Centers. Refashion the toolbox into components available for purchase from an in-application repository and open that repository to 3rd party developers (with a trusted [reviewed] and untrusted section).

Have it read every damn spatial format possible. Embed Webkit so you can jump to Bing, Google MyMaps, OpenStreetMap, WeoGeo, and everything else out there of geospatial interest. Metadata is largely a solved problem with my music in Foobar (music player). Why? Because of freedb, a license-free database of song metadata. There isn't any reason why we couldn't do something similar for spatial data.

Ideas are cheap and this is a lot of hard work, but it is something I'd like to see. There is a lot of moving parts to spatial data - a lot of sources, metadata, and datastores. It would be nice if we could abstract away all the stuff you currently have no interest in. The fact your spatial data is in PostGIS, or AGS, or in the cloud somewhere is unimportant when all you want to do is supply the key information of when it was collected and why.

Fun with SQL Spatial

2010-02-05T09:03:00.000-08:00

This is old hat by now, but I love how much cruft can be killed by outright avoiding older APIs.

A unique location number needed to be generated for an engineering design tool. A vendor contact sent an example to work off that was about 400 lines of C# code. This little stored procedure replaced basically all of it.

CREATE PROCEDURE [sde].[IntersectGrid]
@x FLOAT, @y FLOAT
AS SET NOCOUNT ON
BEGIN

DECLARE @g geometry;
DECLARE @grid int;
SET @g = geometry::STPointFromText('POINT ('+str(@x)+' '+ STR(@y)+')', 2);
SET @grid = (
  SELECT TOP 1 [areaName]
  FROM GridTable
  Where @g.STWithin(Shape) <> 0
 )
return @grid
END

How do you make this 400 lines? Easy, use the ArcObject API to do the intersect. Instantiating dozens of objects, checking out/in licenses, and using reflection to read a config file (not sure why they didn't just use AppSettings) adds up fast.

I could probably even do it with even less effort using Shapely, but no one else is really familiar with Python in this situation (vendors or coworkers).

Use Source Control For Your Thesis

2009-12-29T23:20:00.001-08:00

Why?

Back up old versions without scattering dozens of files everywhere.
Free online hosting with Google - create a message board, wiki, group commenting section
Immediately push in revisions from the committee or other trusted contributers
One central repository for your thesis, no need to back it up a dozen places

I found out how to do this too late for my own thesis, so here are the simple steps you can take to do it yourself.

Download and install Subversion client and TortioseSVN.

CollabNet

You have two choices here. If you want to make your own server (so you can easily get at it anywhere) follow this guide. For a more simple, file based approach, uncheck the SVNSERV and Apache_Mod check boxes in the installer and install it in whatever directory you want.

TortoiseSVN is a handy front-end that plugs into Windows Explorer. You can find it here.

Make a repository.

Right click on a folder where you want to put the repository (the database containing your thesis) and select TortoiseSVN-->Create repository here.

Make up some folders for the database.

Right click TortoiseSVN-->Repo-browser. The URL it asks for can be your server (if you made one) or "file:///C:/svn", without those quotes, where the folder of your repository is C:\svn.

The Repo-browser shows the contents of your database. Right now, it is empty. You pull stuff in by right-clicking (starting to see a pattern?) in the right hand panel of the browser and selecting "Add folder...", "Add Folder...", or "Create folder...".

You'll notice that everytime you add something, you can put in a note saying what you added, changed, or deleted. Very handy if you want to roll back to a previous change you made.

Add your thesis into the database.

This is easy, simply right-click as above and Add File. This will be your authoritative copy from here on in. You can check out local versions, modify them, then push the modifications to the database.

Check out your thesis.

Another right-click operation, this time target open folder with the files you want and put it in a local folder.

The place you put it becomes your "working copy". You can make as many of these in as many different locations as you want. Subversion handles how up-to-date a current working copy is, and lets you chose if you want to keep parts of the repository copy ("theirs") or your local copy ("mine") when you check it back in to make changes.

Make some change to your local copy.

The nature of your change is basically unimportant. Any change to the file will change the icon next to the file in Explorer to a red "!" rather than a green . This shows that changes are pending - there is a difference between your local copy and the server. You can check what exactly changed by Right Click-->Check For Modifications-->Right Click on a file--> Compare With Base.

If you are using a Microsoft Word document for your thesis, stop doing that. Seriously. Pick up LaTeX and save yourself six months of infuriating style editing in the future for this and every subsequent paper you do. Anyway, if you are using a Word document you can still use Subversion, you just can't use the nifty diff tool with it. Only plaintext works, Word documents are binary files.

Check in your change.

Checking in your change is simple, just Right Click-->Commit. You can also check what has changed compared to the stuff in the repository by right clicking as in the above step.

You can use this for all kinds of important documents. Anyone you give permission to can post changes, and you can see exactly what changed and whether to keep the changes (look for the Update To Revision in the right click context menu). Finally, if you do any programming or scripting at all, you are doing yourself a great disservice by not using source control. Plus, it is a great skill to put on a resume/CV.

SortaLogic

2009-11-24T23:42:00.000-08:00

The new job is different from my former one in the kind of work so far, the organization, and the environment. The responsibility for other systems - customer service, outage management, geographic information - is not on "some other team" in some other department in the company, but your responsibility.

There is only one technical team, and while other departments have personnel that are technically competent, ultimately your team is charged with making everything run on a day to day basis.

A good example here is on a rush job today - pulling data out of what we'll call a relational spatial database (CertaLogic) and putting it into an industry standard XML format (Multispeak). I am not familiar enough with CertaLogic to do an extensive complaint, but it is sufficient to say that whatever flavor we have is not easily mapped to said industry standard.

In my previous job, I would have seen this requirement coming. The confusion over whether or not it was a task I was responsible for comes from the fact that at a smaller firm, more work appears to be contracted out to 3rd parties, and one of them was charged with doing the "migration". Much of my department was surprised that this didn't include some specific parts of our GIS.

As with all unexpected requirements, there was some pressure to get it done fast.

Short answer for those that in the same situation, you have three options. Roll your own solution using a standard xml library (lxml for Python, System.Xml for .NET, etc.), XmlSpy/MapForce, or use SQL's built in XML functionality.

(Edit: hat tip Scott Hanselman for XmlSpy)

New State, New Faces

2009-11-02T20:09:00.000-08:00

So, there has been a little change of scenery since I last posted. I got a job in Washington and didn't have to be in too much of a rush to move (unlike when we moved down to Arizona). It went something like this:

Northern Arizona

Hoover Dam Some part of California Some other part of California (you'll never guess). We traveled up the Oregon coast. I would heartily recommend this. Nearly as pretty as that girl. Now we are living in Port Orchard. It is rather nice.

Still settling in, so I will leave more details for another post. The summary: doing GIS work for Peninsula Light Co, a nonprofit coop power utility, enjoying the temperate climate, and taking the ferry to west Seattle on the weekends to see friends. I've been rather terrible about updating this blog recently due to the effort required in the move, but I intend to talk more about the area and the new job in more frequent posts.

More graph tracing - this time for water quality.

2009-08-13T13:05:00.000-07:00

My last interlude with tracing the water system involved looking for hydrologic hazards - specifically other potential sources of water that could confound maintenance efforts. Apparently I impressed someone enough with it to get a new trace-oriented project having to do with water quality. I'll have a second part to this talking about my first run at it and a subsequent refactoring that I'm extremely happy with, but first I wanted to mention how I was documenting it. As far as I can tell, the corporate standard for documentation is Microsoft Word documents. At best, these have a relatively easy to navigate table of contents and the document is stored at the same location as the topic. At worst, it has neither attribute or doesn't exist - there is no practical difference between those two situations really. No one will ever find them - which for development projects is especially problematic. The Don't Repeat Yourself (DRY) principle applies outside single projects. There were some things I really wanted for the documentation I was going to produce for the product.

Fast, built in search. Amazing the difference this makes.
Something you could put up on the web with little to no fuss, but wasn't actually a public facing website.
Allowed all kinds of markup, images, other resources.
Wiki-style editing - who changed what, when, and some measure of version control.
Plaintext or in some easily parsed format.
I could quickly convert it into Word if I caught too much flak for not using it

What I eventually settled on is called TiddlyWiki (Google, Bing, Yahoo!..I'm just certain there is a rule Web2.0 stuff requires a childish name for success). Its one HTML document, thus very portable, with a bunch of JavaScript that implements all the functionality I wanted above. It also apparently has a lively plugin community that I haven't had time to peruse.

Documentation is important to preserving the intent of any project.

Stored Procedure Hell

2009-06-17T17:06:00.000-07:00

I fancy myself as a very competent user of SQL (particularly T-SQL), and from school, on-the-job training and experience I've come to understand a comprehensive list of things SQL does well:

Basic CRUD operations
See number 1
Some very basic data manipulation - some of the stuff found here.

And that's it. I've found a problem begins to occur when people start going outside this basic mold and attempt to put in real application and business logic into stored procedures. Say, calculating a custom coordinate by referencing a seldom updated flat table containing coordinate grid intersections. Or tracing a major road or utility network. It isn't too odd this happens - a lot more people know or can quickly learn SQL. Compiled on the database, stored procedures can have impressive performance. The inevitable result, however, is not pretty. Half of your application logic is living on the database, the other half hangs out on (hopefully) a real source control system. You become heavily tied to that database vendor, since they all seem to have their own SQL dialect, and undoubtedly you've had to use their more advanced and esoteric features to implement something. And I've found the stored procedures are rarely extensible for multiple applications. They can be, but in practice they are built for very specific purposes and are tightly coupled to the applications that need them. Example: a stored procedure that calculates a custom coordinate, like the example mentioned above, is entirely coupled to its table. There would be no easy way to change the data source to a more commonly updated table or, ideally, a standardized web service. Also, the calculation itself isn't broken up into discrete methods, which means modifying one part of the long procedure will quite easily break another part in ways not necessarily easy to discover.

It's for storing data, not making a calculator for it.

Google Wave

2009-05-30T15:12:00.000-07:00

Yesterday at their I/O conference, Google demonstrated a new communication/collaboration tool that appears to be intended to kill email as the standard form of personal web communication. My first feeling is: Good! Have you ever tried to use email for long, branching communications? The flat, threaded view is trash, and a tree view is not much better. I would love to see this get mainstream attention because it might not only kill email, but also kill off less collaboration tools I personally find less than enjoyable to use. Sharepoint, for instance, I have found to be clunky and have poor integration with email. That isn't really the fault of Sharepoint - email was never meant to be an issue tracker or dynamically update websites. Nor would you want it to, it doesn't have the tools. One of the first things they show off with Wave is how easy it is to do just that (particularly bugtracking). Wave is also free, open source (in some fashion or another), and federated. That last bit means your Waves don't necessarily need to live or interact with Google at all - just like internal email servers you can whip up a Wave server and apparently quite easily. The motivation for Google doesn't immediately occur to me. Best, moderately educated guess is that it might have something to do with HTML5 - the next version of HTML currently supported by roughly zero browsers yet - will be easier to index and search than Flash/Flex/Silverlight. Is Wave supposed to be HTML5's killer app, driving the browsers to pick up support for it faster? As a side note, there appears to be a conspicuous number of references to the television show Firefly in the above presentation and in the project. In the show, the standard communication term used was 'wave', the poll robot they demonstrated included an option for Serenity as the Best Movie Ever, etc. This isn't too wild for them. Keep in mind that Google Earth was apparently inspired by the "Earth" program described in Snowcrash.

Future roles of a GIS analyst: The problems of cartography

2009-05-13T07:52:00.000-07:00

The company I work for is both a water and power utility. Both sides have their own geographic information systems, with parts of IT trying to integrate both systems into one big GIS database. While there are complications on the technical end, let me try to describe some of the issues on such an integration from a cartographic perspective:

Clients, usually from one group or the other, are accustom to very specific symbols for specific map features.
All of the historic maps from either side have had consistent symbology that match said clients preferences.
The two groups symbology clashes - what is the red line; a 70kv line, water main, or a user annotation from someone in the field?
The extents of the features clash.
A lot of features are not loosely coupled: there are groups where if something is not present it makes no sense, or applications that may require it fail.
The labeling becomes very problematic for the same reasons symbols do.
Definitions! What does "deactivated" mean exactly across very different features in different groups?
There are a truly massive number of features. So many that a given client is not necessarily going to know what they really need.
How do you project it? The scales of the datasets vary immensely.
What coordinate system - clients have different preferences in this matter, and one has a custom one of their own creation.

This is all without mentioning that there are plenty of subgroups like environmental, telecom, or legal, which have their own customs and requirements. At least for us, I think the role of cartographer will still have a place here for a long time, trying to organize all this stuff for human consumption. I don't think these issues are unique. Nor do I think that they wouldn't exist if there was tighter integration between the two groups that generate the bulk of the geographic data. I think we are on the cusp of a revolution of both location-aware and location gathering technology - and more data requires better means of organizing and displaying it. Some of this can be automated, but not much I think. I have yet to see software that can easily determine if a given map - or web map application - contains the necessary data for its audience and that data is organized in such a way to be legible, queryable to the client's satisfaction, and aesthetically pleasing. ESRI has put a lot of effort into simply automating the placement of labels, but it is by no means perfect and often requires extensive customization. Perhaps Dave Bouwman is right and "GIS analyst" - already a nebulous term - is not a career that will exist in the future. But I fail to see a way to do all of the above without someone dedicated to cartography.

Future of GIS Analysts, Part 2

2009-05-01T12:55:00.000-07:00

In the first part of this post, I tried to put together a simplified list of some of the activities that I have found GIS analysts doing as a part of their job. The goal was to get folks talking about future roles and if GIS analysts have a future ten years from now. I was happy commentators posted some things I missed. I hadn't included, but was helpfully mentioned by geographygeek in the comments, the role of an analyst after the data has been collected and processed. Some (hopefully most) are trained to answer the question, "Well, what does that mean?" Another commenter, KindaSpatial (who puts out a rather good geography podcast), wanted me to talk a bit about some of the newer 3D and hyperlocal data and interpretation. I'm honestly not sure if I am qualified to, but I'll give it a shot and have people correct me later. I like to think of blogging as more of a dialog. On closer inspection, each of these should be their own blog post, so that is what I will do. For now, I will give my initial impressions by going down the list and examine each item, asking the same questions: Can this be automated? Is it easily outsourced Is another profession largely absorbing it? Here are my thoughts. They are not quite fleshed out, feedback greatly encouraged.

Map production: Outsourcing: try to make or get a good, topic specific map via phone conversation. Automation: still a lot of overhead software knowledge required for the kind of quality maps necessary for professional reports. Professional designers have all of the aesthetic abilities necessary for this, but little knowledge of the pitfalls of cartography - maps are like statistics that are even easier to lie with.
Requirements gathering: Probably impossible to automate, and you can read the hilarious results of trying to outsource it elsewhere. Increasingly the realm of project managers with enough GIS experience to know what is available/feasible.
Feature creation and maintenance: The simple stuff can and will be outsourced or automated. Stuff that requires boots on the ground simply can't, and yes it is hard to tell the difference. I don't think the dimensionality of the data makes a significant difference here.
General IT/helpdesk support: This really is the work of IT professionals, but for smaller firms or feudal departments it isn't going anywhere.
Database/content management: All information has a location-based component, and this function exists as only so long as professional database administrators are uninterested in how the middleware (SDE and PostGIS I believe) works.
Minor automation tasks: Prime candidate for professionals to do the automation - it ends up being cleaner and reusable.
Post-processing interpretation: (hat tip: geographygeek) This dovetails nicely which what I think is the core of GIS - the visual display of quantitative information. What you replace a geostatistician with? It doesn't seem to easily fit in with other professions, the methods are unique enough to be difficult to automate for easy public use.
Hyperlocal data: (hat tip KindaSpatial) The mass of data associated with this seems to require, not merely lend itself to automation. Turning that stuff into interesting visual works or meaningful statistics sort of falls under the previous point.

Based on any comments/corrections I get, I'll be going though these points individually. I haven't been at the GIS game too long, so I'd love to hear from some people from different backgrounds.

Future of GIS Analysts, Part 1

2009-05-01T11:33:00.000-07:00

A question occurred to me as a result of a comment made by David Bouwman on Twitter. Folks were congratulating James Fee, who had just been offered an opportunity to teach GIS at the Arizona State University masters program.

Teach them that "GIS Analyst" will be a rare job in 10 years - just like "Database Analyst" is today.

My question(s): what do we mean by GIS Analyst in this context - what functions do they provide today that will be unnecessary or absorbed by other jobs? My official title is something like GIS analyst, though this kind of talk might be more disconcerting if I hadn't already oriented my career towards application development. But I know people I work with, professors, and certainly students today would be interested in discovering if this was in fact the case. If we were to properly investigate this matter, lets consider first what GIS analysts (which we can probably group with specialists, technicians, etc) do that makes them necessary today, and then, in part two, we can examine what that role might look like - if it exists at all - in the future. Here is a quick list of the roles I've seen played by GIS analysts:

Map production: Your standard cartography work. Organizing the layers, layouts, titles, etc into an aesthetically pleasing package and either plotting/printing it off, or, more recently, publishing it as a service.
Requirements gathering: Client communication, identifying potential solutions from user stories, project specification and some project management.
Feature creation and maintenance: Gathering and organizing data from disparate sources, digitizing/COGO work. Associated documentation/metadata probably falls in here too.
General IT/helpdesk support: Particularly the case when it is a small firm without a real IT department or person, or if that person/department is swamped, or doesn't know anything about GIS software, or IT's grasp on individual departments is tenuous.
Database/content management: Organizing databases, particularly ESRI geodatabases - what belongs in a given dataset, should it be part of the network, etc. File management of documentation, supplementary data.
Minor automation tasks: Modelbuilder, Python, VBA stuff. Almost any programming task where not knowing the basics of object oriented programming is not much of a hindrance (though it makes for terrible code).

I'm missing probably a hundred more things analysts do and I'd like to invite everyone to help me add to the list.

GITA Arizona Conference Trip Report

2009-02-26T11:07:00.000-08:00

Geospatial Information Technology Association (Arizona Chapter) got together yesterday to present new infrastructure GIS applications in the valley. Here are the presentations and some commentary: City of Phoenix showed off their new internal Flex/ArcServer application that replaces their older and less frequently used IMS application. It looks like it duplicates much of the functionality of ArcExplorer with some additional tools for work orders, etc.

Why Flex? They found it had the best preformance, only required knowing Actionscript and MXML, and there was a lot of the UI built-in.
Multiple MXDs (i.e., web services) one interface.
They've received good feedback from users and doubled their user base.

Salt River Project demonstrated a tool many people may not know about for improving ArcServer performance. Turns out a lot of latency can be the result of the symbology, labeling, and other components.

ESRI publishes a free tool that gives you per-layer statistics on performance called MXDPERFSTAT.
Stuff to avoid: halos, definition queries (use db views instead).
Stuff to use if you can: Simple Symbols, Annotations.
Additional tips for Oracle users: watch for the high watermark issue

City of Mesa shows off some 3D & 4D stuff in Google Earth, used for land use planning around the Phoenix-Mesa Gateway Airport.

Some issues: height requirements, noise requirements.
Used ASU's "Decision Theatre".

More City of Phoenix. Interop talk about moving parcel layers around. Record number of uses of the word "Open" when describing a business process that uses nothing but proprietary software. The GIS of the Phoenix Skyharbor Airport - a city unto itself. Described as 2.25D, contains extreme detail and enters into traditional CAD territory.

Every fire extinguisher, every door, every thing appeared to be part of the inventory. $3m budget for data gathering.
Layered approach - users can select different level/terminal and the application puts you indoors.

Boots on the Ground in GIS Analysis and Development

2009-02-24T09:13:00.000-08:00

While I do a lot of GIS development these days, I am pleased to do some solid analysis work as well. I was recently given the task to delineate land use/land cover using standard remote sensing image interpretation methods. I'd like to think I've pretty well trained in this kind of work, and my experience has confirmed one particular part of that training that was universally emphasized - boots on the ground. You can create a cunning story about exactly what is happening in a given air photo and can be completely, utterly wrong in all regards. This can happen whether you are a fully certified photogrammetrist or just a novice staring at something weird in Google Earth. Seeing what you are studying up close and first hand, assuming you can, is essential to avoiding these kinds of mistakes. In the case of my current project, I would say it was essential I went out and actually traipsed through the study area. The air photos and other sources did not prepare me to interpret the vegetation I can now pick out extremely well. Many professions call attention to this. Military commanders who ignore the front can find tactical situations dictating strategy in ways they neither expect nor desire. I don't see it nearly as stressed in developer circles - books, online sources, or in my own experience (though AGILE practices come pretty close in terms of constant and face to face communication with clients). You shouldn't be learning what your users want exclusively from their management or your marketing/sales department, and in many cases I have seen, selling an idea to management only seems to be half the job. A really enlightening experience was going out into the field, sitting in the trucks with a user, and just watching them use the technology and see the actual business process in action. It is amazing how much it can differ from abstract flowcharts and user stories cooked up by others or even straight from their mouths. They'll lie to you, and I don't mean intentionally. They'll omit something because they do it every day and imagine it is second nature to their audience, or they forget, or they won't think something is relevant. It is your job to ferret out the real requirements or the real trees you are looking for. Worst case scenario: you get out of your office and see the sun for the first time in a year outside the weekends. Anyone have any stories about similar situations? Images from Googleblog, this talented Flickr user.

There is location and then there is behavior.

2009-02-05T12:03:00.000-08:00

Google Latitude, which effectively embodies the business case of Loopt until some kind of conversation occurred at Google, allows users with mobile phones to opt-in to a service which broadcasts their location. People can search for their friends locations and retrieve Twitter-like messages by them. Location is a good start, but more interesting is a presentation I recently viewed on behavior and its use for location based apps like this one. Why just try to find your friends somewhere when you can go ahead and predict where they, and people like them and you, are going to be? I rather like the privacy policy too (anonymity, you own the data you create).

From GIS User to Developer - Part 3.

2009-01-31T10:56:00.000-08:00

GIS college education typically prepares students for the very basic entry level positions. Assuming you want to have a job five years later that isn't the subject of ridicule or eliminated largely by automation, you should consider yourself a lifelong student. And just what are these entry level GIS positions? Again, my experience in the entire field is limited to just a few years, but talking to people, it seems pretty standard. Many of them are georectification/image processing monkeys. Some are brought on as folks to help maintain larger datasets like municipal water/power lines, parcel boundaries, etc, assuming there isn't a process for the drafters to input in their designs manually. Still others are cartographers in the sense they make custom stylized maps on demand. There is absolutely an art to this and, with the proper motivation, you can make a real statement (or lie) by including, omitting, or tweaking the standard map components. The proper visual display of quantitative data can shift public opinion, lead to a medical breakthroughs, and are sometimes the cause of the international dispute. It really shouldn't be underestimated. The Advice If you find yourself in the standard entry level GIS position, or applying for them, you should be looking for the means to specialize. As far as I know there are three primary ways of doing this.

Not Business
The Data
Applications

At the moment I am sort of doing all three of these things, but pushing myself more towards the applications side. I'll have a much bigger chance to do so when I've wrapped up my thesis (Feb 13th defense, March 6th text deadline). Really, I am still new to a much wider GIS world and it would behoove people still doing the entry level stuff to check out some blogs written by people more knowledgeable than myself.

From GIS User to Developer - Part 2

2009-01-23T21:10:00.000-08:00

So School I finished up a GIS oriented Geography degree from CWU a few years back. From what I have heard about other, similar programs around the United States, my experience was typical - some general GIS/Computer Science history, labs following relatively closely to ESRI published workbooks using the ubiquitous ArcInfo suite, some quantitative/statistical methods, and some remote sensing stuff. I was lucky enough to do some special projects for actual state and local agencies who wanted some free/cheap maps with perhaps a smattering of analysis work. I thought I would just walk out of the college doors and fall into a low/entry level GIS job, and I might have been able to. But I met a lovely girl, who I ended up marrying, so luckily I found a pretty good excuse to stay around the college until she finished.

I don't really have a relevant picture for this section so here is CNN embarrassing themselves.

And Yet More School Up until I came down to Phoenix, everything I learned about GIS came from Dr. Robert Hickey. He was, at the time, the only member of the faculty to know much about GIS and apparently I had annoyed him enough with not-always-easy to answer questions about his courses. This left the probably mistaken impression I was more intelligent than I actually was at the time. I was curious and motivated. I liked the work and wanted to master it to the extent that was possible. This basically remains true. Anyway, this annoyance/mistaken impression factor got me an assistantship running the GIS lab at CWU. This was basically a helpdesk/IT role that, along with some independent contracting, paid my way through grad school. I get to defend in a few weeks. The Advice The only information I left the school with in terms of automating common GIS tasks was using ModelBuilder. Don't let this happen to you. At the very least, pick up SQL. Even if you don't want to become a developer and intend to live the haunted existence of a business analyst. If you are doing GIS classes you pretty much already have the necessary logic to grasp SQL. Done an attribute query? Then you've basically done half of an SQL SELECT statement. It is a "language" with a ridiculously small amount of actual syntax but can be extremely powerful. Also, it exists in every database in the world. Also, you can be justifiably ridiculed for not knowing it. Assuming you don't take a minor in Computer Science, which I would highly recommend at this stage, try this

Make your ModelBuilder runs. Make them as complicated as you want.
When finished editing, go to File:Export:To Script. Choose Python, you'll be happier learning JavaScript later
Read this basic tutorial and try to make small changes to your script. I know I complained about documentation just a post ago, but mostly because I have otherwise been spoiled rotten by how good ESRI's usually is. Read that too.

You can learn a lot of the basics of object oriented programming from this stuff. There is an alternative, darker route involving the use of the VBA editor in ArcMap. Nothing you make there will make you proud later, and it is possible your future children will be subject to your subsequent crimes.

Some computer science is highly recommended.

Brief interlude: What the hell ESRI?

2009-01-21T10:05:00.000-08:00

Listen this makes no sense:

The documentation for your VE and GMap JS extensions - which appear to be roughly identical in function - have documentation that contains very different information. To figure out details on the VE.Geometry.ProjectToVEShapes method, I had to hunt down the similar function in the GMaps documentation.
~~Stop pretending these are really REST functions.~~ Be a little more clear about which JS functions are just SOAP requests you later make into JSON clientside. ProjectToVEShapes demands a proxy for larger requests, which means it is running into browser security features meant to prevent cross-domain scripting attacks (detailed a bit here). Oddly the demand for a proxy was only returned when I tried it in IE, in Firefox I had trouble finding it even with Firebug.
The JS objects you return from ArcServer queries differ in format to the JS objects you require for functions in the aforementioned extensions. If I make an ArcServer Query for some feature class, the object it returns is not valid as an input to Project, for example.

These API extensions, and ArcServer itself, is pretty new. I can understand some of the documentation stuff, but could you be a little more consistant about the JSON formatting? Converting between the two is a bunch of unnecessary work. Am I wrong here? I'm pretty new to this, I could be missing a simple solution or not understand the problem.

From GIS User to Developer - Part 1

2009-01-19T19:16:00.000-08:00

I've been doing this four years now and gotten a chance to experience the GIS field from multiple perspectives. I'd like to share these experiences in the hope that someone traveling along the same path can learn from my successes and, of course, the embarrassing mistakes I've made. But first, some backstory: Motivation: I have two basic motivations to get more technical in my job. First, I love tinkering. I build my own computers, jump in on the newest software betas (example: posting from Windows 7), and long before I knew it was worthwhile for employment purposes, I was messing with programming. I wrote a terrible, but fully functional C++ tic-tac-toe game at 15. This means basically nothing now of course, with 8-year-olds routinely getting major Microsoft certifications ("BOY OR GIRL!"), but it was pretty cool to be able to bend a computer to my will. The second motivation has to do with the future of GIS. Future of GIS: There are basically two schools of thought here. The technophile psuedo-utopia/distopia school - depending on your point of view - involves always on, complete geographical awareness via mobile devices or Matrix-style head jacks. Go out for a night on the town and you'll know where your friends are, you find breadcrumbs (static notes) that tell you the places that are crowded/cheaper/better/live nudes/whatever. Hold your phone camera up to the skyline and see information on all the buildings and places in the distance. Almost everything is web based, served by a number of gargantuan server farms run by Amazon (EC2), Microsoft (Azure), or Google (basically everything they do). While somewhat fanciful, a lot of this stuff is here already and running. Check out the 4D map setup D.C. put together so people that go to the inauguration might have a chance to commute out of Washington before the next election. The second school believes this change will happen more slowly, with a place for paper maps and existing uses of GIS in desktop and imbedded applications. Despite advances in mobile internet architecture, it would be foolish to marry all applications to the cloud because the possibility of service interruption for, say, rescue workers is a non-starter. Plus a ton of the data kept by organizations is sensitive in nature and the data giants mentioned above are not as good of stewards of some information. Cartographers and analysts using desktop applications won't go away, particularily for extremely hardware intensive uses. I still consider myself fairly new to the field, but it would appear to me the first group is probably closer to the mark. Of course there will be applications for GIS that will not easily translate into the web; customized statistical modeling, data model design, and anything with heavy raster manipulation comes to mind. But powerful tools for bringing GIS to the masses, to shamelessly add an abominable cliche to this tirade, have been found to be increasingly popular. The dominant player in GIS - ESRI - is quickly moving in the direction of the first school. To make this happen you need developers who are creative and interested in geography. You need analysts and data modelers to make the back end work, and cartographers/designers to make it usable and pretty. The common thread with these folks now is that, to get your stuff online, you'll have to know at least Javascript. Oh sure. You can get away with using a wizard or template system to make a web map, but to do anything really interesting you'll want to pick up a scripting language and probably more. Thus begins this story. I was a student having just finished up a GIS oriented Geography degree. If you are in a similar situation, or anywhere between that and an actual GIS dev, this might help you. Sorry this first bit is somewhat slow.

Government Jobs

2008-12-16T17:01:00.000-08:00

Some folks, particularly in this economy, tend to look more for government positions because of the perceived job security. The problem is you have to contend with stuff like this, sent to me by a friend. She told me it took them six months to hear back from them, and she had forgotten she had applied to the job at all. Everything is stripped of personal/identifiable information to protect the stupid or annoyed. Red highlighting is mine.

*----*----*----*----*----*----*----*----*----*----*----*----*----*----*----*----*----*----*----*----* Replies to this email will be sent to Joe Sixpack *----*----*----*----*----*----*----*----*----*----*----*----*----*----*----*----*----*----*----*----* December 15, 2008 Jane Doe 1243 Fake Street Springfield, IL 78841 Dear Candidate: Thank you for applying for the position of Systems Analyst with the Government Department. As part of the evaluation process, you are being invited to participate in a performance examination. The exam will be conducted using computer based testing programs. The exam may consist of data entry, clerical proofreading, basic office skills, and customer service representative skills. EXAM CHECK-IN Check-in will begin ten minutes prior to your scheduled exam time. Do not arrive any earlier, as there is no waiting area outside of the testing room. Your exam is scheduled for the following time and location: Date: 12/23/2008 Time: 01:30 PM Duration: 1 hour, 45 minutes Location: County Building, Room 120 123 10th Ave Makin, GA 62479 You must bring official picture identification, such as a driver’s license, to be admitted to the test (no exceptions). Instructions for Test Confirmation To confirm your appointment, please e-mail test@county.gov. no later than 4:30PM, DATE, 2008. In the subject line, write: Systems Analyst, Yes I will attend testing session *OR* Systems Analyst, No I will not attend testing session. Include your first and last name in the body of the e-mail. Due to the volume of responses, we are unable to reply to email confirmations. If you are unable to attend your scheduled appointment time and need an alternate test time, please leave a message at 555-1234. We will try and accommodate your request. However, due to limited testing facilities and time constraints we cannot guarantee an alternate time. Parking There is limited on-street parking (applicable fees charged) off 10th Ave between Parkplace and Broadway. There are several parking lots and garages nearby that provide parking for a fee. We look forward to seeing you. Directions to this location are provided at http://www.someurl.gov Sincerely, Joe Sixpack Human Resources Division PLEASE DISREGARD THE FOLLOWING MESSAGE: If you applied online, please verify you have received this message by visiting: https://www.wtf.com *----*----*----*----*----*----*----*----*----*----*----*----*----*----*----*----*----*----*----*----*

Lets list the hilarious parts, in order:

The exam will be conducted using computer based testing programs

Whoa, tests on computers now! Fancy. I bet it uses the same fantastic generic form program this letter was (still)born from.

The exam may consist of data entry, clerical proofreading, basic office skills, and customer service representative skills.

Great, a basic office skills and data entry test. Not at all a waste of time for a Systems Analyst.

Do not arrive any earlier, as there is no waiting area outside of the testing room

"People arriving early will be forced to run on a treadmill until the time has elapsed." I don't get it. Do people cheat through walls now? Has the office banned chairs because of the tightening economy?

12/23/2008

I assume the six month wait was so they could have the test on the second to worst day of the year.

no later than 4:30PM, DATE, 2008

"I am too lazy and/or incompetent to actually fill in all the fields in this form letter."

PLEASE DISREGARD THE FOLLOWING MESSAGE

On second thought maybe this isn't sheer stupidity - perhaps it is the test itself! Anyone stupid enough to click on a link in an email from a vaguely trusted source (that even goes to the trouble of telling you not to click it) would be a useless Systems Administrator. Actually anyone given a computer that does this is a network outage waiting to happen. The people qualified for the job, assuming they didn't get one already (my friend did) should answer it like this: "Subject: Systems Analyst, No I will not attend testing session. Is this a joke? Is a form letter really that hard to fill out? Why do I want to work with such an HR department? You wait six months to assign me to a test for which anyone applying for this position will be embarrassingly overqualified for the day before Christmas Eve. If this isn't a joke, get bent."

Some choice quotes from an economic catastrophe

2008-11-17T10:40:00.000-08:00

I haven't posted for a while, but now that I think I've finally gotten a somewhat decent grasp on it, I'd like to talk about the economy. Forget about the whole "bad loans to people who couldn't afford it" shtick. The reality is that people from lower income brackets can get subprime loans and the economy need not collapse. Slate has a very good article up on subprime mortgages that are doing just fine because the companies in question actually bother gauging risk correctly instead of shopping around at ratings agencies.

Like a bunch of present-day George Baileys, ethical subprime lenders evaluate applications carefully, don't pay brokers big fees to rope customers into high-interest loans, and mostly hold onto the loans they make rather than reselling them. They focus less on quantity than on quality.

Those ratings agencies aren't the biggest issue, but some of the stuff that is now known about their risk assessments defies belief (from this excellent article by Michael Lewis):

He [Eisman] called Standard & Poor’s and asked what would happen to default rates if real estate prices fell. The man at S&P couldn’t say; its model for home prices had no ability to accept a negative number. “They were just assuming home prices would keep going up,” Eisman says.

And when S&P wouldn't give them a good (AAA) rating on something, just ask another rating agency! (Bill Ackman on Charlie Rose):

ROSE: One person after another has come to this table and just cast huge criticism at the ratings agencies. ACKMAN: Yes. ROSE: It’s all deserved? ACKMAN: It’s deserved. ROSE: How did it come to that? ACKMAN: [...] There was a lot of rating shopping. An investment bank would walk into Moody’s and say, look, here is a risk. We’re looking for a Triple-A rating. It would be analyzed by Moody’s and they’d say, OK, it’s Triple-A. The investment bank would say, what, if I throw this in, is it still Triple-A? They would say, looks Triple-A to us. What if I threw in a few more bad mortgages, is it still Triple A? At some point Moody’s would say, it’s no longer Triple-A. They’d say OK. Then they’d walk across the street to S&P. They’d take a look at it, and if they said yes, they’d get paid a 600,000 dollar fee. Yes, it’s Triple-A. If they said no, they wouldn’t make any money.

These bad loans were the basis for bad insurance policies. They called them credit default swaps because if you actually called them insurance you would need money to back them up (thanks Phil Graham and the congress who voted for that little exception in 1999). Of course, if you can get insurance on something, you tend to perceive the risk to be less, so you take out more loans. There is a reason why insurance companies need to have the money to back up their policies - natural disasters would otherwise constantly be giving the economy a full bodyslam. These default swaps made what probably would have been isolated losses of billion into trillions that are now poised to hammer every part of the economy, and be potentially the start of a second great worldwide depression. A good portion of my family or friends aren't being hit by this yet. We work in tech jobs and often for government(ish) agencies. It doesn't feel like we are actually in a recession, but I think we are just starting the lovely cycle they call a liquidity trap. The song goes something like this:

Reduced demand -> cut production, jobs -> fired people don't buy things -> reduced demand -> next verse, same as the first

Many believe that this is happening right now. That is why we've been hurling money at financial institutions and they've basically stuffed it in a mattress rather than loaning it. They are terrified of the next Lehman Brothers-like company crashing and taking them along, so they horde money for that rainy day. Give that money to just about anyone right now - automakers, consumers, dogs - they aren't going to spend it. And not because of rampant dog-as-customer discrimination. Ask yourself this: how willing are you right now to buy a new car? Maybe a little less than a year ago? The fix is finding some sector of the economy that will always spend more, then give them the means to do so. In the last sentence, if you started mouthing "the Congress of the United States," other people have the same idea. Even people who previously thought this was bad idea - Secretary of the Treasury Paulson is a good example - are now doing it one way or another (buying control of banks rather than just making loanshark-style loans to them). Some economists are skeptical of the existence of liquidity traps. One thing that is generally agreed on: it isn't always obvious to an a given person they are in such a trap, because different industries/job sectors go from step 1 to step 2 at different times. If it hasn't affected you yet, and it continues, you might just need to wait a little while.

Twitter

2008-10-22T12:02:00.000-07:00

I've broken down and starting using Twitter, largely to eliminate the habit of finding a link on the internet and using work email to send it to my private email. Also so I can follow people smart people who rarely actually make blog updates. My account name is Bwreilly. At one point I will probably collect the links I wander into and repeat them up here as it is an embarrassingly easy way to make a post and also because the links themselves are often of interest and it would be a shame to deny them to people who disdain Twitter.

Flex vs JavaScript for created online maps

2008-10-21T12:54:00.000-07:00

An interesting discussion over at James Fee's blog on this subject. It looks like more folks are going with Flex for some very simple reasons: 1. Sure, its dedicated IDE costs money, but so does Visual Studio. 2. Adobe Air allows for movement of the app to the desktop. 3. Browser compatibility, particularly with IE, is a massive problem and Flex sidesteps it. 4. Reported better performance. Not to repeat the whole thread, but the most interesting comment was revolving around how Adobe has done what Java was designed to do, but couldn't - get people to install a plugin that was basically browser agnostic and had more functionality than JavaScript (credit to Matt Giger). Update: More commentary from the developers of the Geocommons website. Geocommons is worth a whole post I'll probably do sometime soon, it is pretty nifty and uses Flex.

MPAA: Kings of Irony

2008-10-21T07:48:00.000-07:00

Here's a quick timeline: ~2000: Anyone who could afford a DVD drive for their computer could rip movies with some minor hassle with free tools floating around the internet. ~2003: The free tools become easily usable by anyone with a brain stem and interest in doing so. The programs crack DVD encryption methods with frankly embarrassing ease and speed. 2007: RealNetworks tries to make a legitimate tool for ripping DVDs while leaving in some DRM (i.e., restrictions on use). 2008: The Motion Picture Association of America (MPAA) sues them for aiding infringement. The EFF files a brief supporting RealNetworks position, basically claiming it is fair use to copy your own DVDs and people have been doing it for almost a decade. Today: MPAA mocks the EFF for "living in the past". So, with dozens of commonly used programs out there for backing up DVDs - a perfectly reasonable thing to do, since the lifespan of data on commercial hard drives is basically infinite if you back it up properly - the MPAA decides to target one that actually keeps the encryption that they original put on the disks. Their definition of the past is pretty funny too, since (eight years ago/today/in the foreseeable future) DVD backup (was/is/will continue to be) easy and legal under the terms of fair use and really, really easy. I'm kinda surprised RealNetworks even found a market for it with the number of effective free tools out there.

Federal Lands in the US

2008-10-06T19:03:00.001-07:00

An excellent example of a graduated symbol map. My only complaint is Alaska is not actual size (if you can believe it, in proportion to total federal lands it rivals Nevada).

GIS In the Cloud

2008-10-04T15:22:00.000-07:00

Cloud computing has been the hot topic in many GIS circle for the last year or so, largely for the same reason it is building steam more recently in most IT circles in general - datacenters and bandwidth speeds are nearing the point where the promise of mass cloud computing is feasible for corporate users (which is where the money is). For anyone previously not versed in this topic, cloud computing is basically the movement of applications, services, and data from local storage to massive datacenters run by people like Google, Yahoo, Amazon, and Microsoft. You probably use it already - say if you use GMail rather than a local application like Outlook or Windows Mail. Maps have obviously moved there too. If you own a computer you probably use some kind of online map for directions. It doesn't necessarily need to be in a browser either - Google Earth and NASA's WorldWind might be local applications, but all of the data and services are running off of datacenters somewhere else. It is believed this somewhat slow progression is going to accelerate as activities typically preformed by local IT departments for small and medium-sized businesses are increasingly replaced by cloud apps offered for basically free by the above organizations. It might seem foolish to marry yourself to a particular platform or business in this fashion but (1) a lot of these things are built on open standards like LAMP anyway and can be transferred around and (2) companies marry themselves to a vendor all the time (see SAP). Traditional desktop software vendors are shifting to do their stuff at least partially as a service (and thus online). Windows 7 isn't going to have a mail program, it is going to fill that functionality with Live Mail. ESRI, the biggest GIS software vendor, has made it a point to make it extremely easy to put online data services into map documents just as you would add in layers on a local computer. A great number of GIS data providers, largely governmental, are not by and large going to venture into the cloud all that soon, not without intervention by legislators. Why? Liability, tradition, data sensitivity. What data public exists is often of variable quality, especially when overlaid with other information from other sources. Throw some bad data out there, even if you include metadata that includes a hefty disclaimer, morons will still use it to hike into a blizzard and and sue you for having to eat their children. Even with good data, there is the problem of interpretation. Take a parcel layer from any given city/county government in the United States and throw it on Google Earth. GE does a pretty good job, but I would wager good money the satellite imagery isn't accurate to a quarter of an inch. The parcel layer is, by law. The number of people who will go on to Google Earth and stir up property disputes without this knowledge is probably substantial enough to be a factor in deciding to release it. And of course there is the security issue. Knowing what substation can black out a particular city block, what water main is feeding that block, communication lines, emergency vehicle GPS locations - this stuff could be used not just by some existential terrorism threat but by normal criminals to cause all sorts of mischief and evade capture.

Programmatically tracing a network in ArcGIS

2008-09-16T12:09:00.000-07:00

Recently I was tasked with tracing a water network, and after studying a previous programmer's work on the subject, I realized the solution is just a modified tree algorithm that is largely language independent. The trick is a recursive function that continues to call itself until it has gone down the totality of a single path. When it can't keep going - because it reaches the end of the line or because you convince the program it is the end of the line - it hops back one function call and tries to take an unvisited path. If it finds one, it repeats the whole process. The ArcObjects API contains the extensions for Network and Utility Network tools, but as far as I can tell,those are mostly for listening for trace events rather than starting your own. Since a network like that is just a modified topology (it inherits from ESRI's topology object), you can use ITopology to create a TopologyGraph which allows access to the Edges and Nodes required. But you don't need to use C#/VB.NET/VBA to pull this off. It would be possible with the API ESRI provides with its Python scripting object (IDispatch), as that contains start and end points in its Geometry object, or by creating your own node/tree structure. I understand this is a fairly elementary use of recursion, but as a novice programmer its a lot of fun putting it into action for a real project.

Administration and Iraqi government close to drawdown deal

2008-08-16T12:05:00.001-07:00

How the Olympics and Georgia are completely pushing this off the radar is beyond me, but the question of opposition to the war and the exact timing of the pullout might be a moot question by 2009: link

American soldiers will withdraw from cities across Iraq next summer and all US combat troops will leave the country within three years, provided the violence remains low, under the terms of a draft agreement with the Iraqi Government. In one of the most detailed insights yet into the content of the deal, Hoshyar Zebari, the Iraqi Foreign Minister, has also told The Times that the US military would be barred from unilaterally mounting attacks inside Iraq from next year. In addition, the power of arrest for US soldiers would be curbed by the need to hand over any detainee to a new, US-Iraqi committee. Troops would require the green light from this joint command before conducting any operation. The Pentagon refused to comment last night on the proposals laid out in the draft agreement between Baghdad and Washington that covers the status of US forces beyond 2008. Britain will strike its own deal with Iraq but Gor-don Brown hopes to withdraw most British troops from Iraq by next summer, reducing the number of soldiers from 4,100 to “a few hundred” by then. Mr Zebari said in an interview: “Our negotiators and the Americans have almost brought it [the accord] to a close. It is not a closed deal but it is very close.”

This is good news, as we could really spare the resources to invest into Afghanistan - a country with a long history of appearing conquered and then exploding. In terms of the 2008 elections, it seems as though with this issue potentially out of the way and the economy recovering, we might actually have a debate about the issues I am most interested in: the limits of executive power, the future of civil rights in this country, and technological innovation (green technology and telecommunications). Oh sure, McCain could call Obama's opposition to the surge a bad decision that would have prevented the current progress from taking place (though some analysis has suggested it had more to do with the earlier Sunni Awakening). And Obama could say that the decision to go to war at all was a bad idea built on a series of frauds. But I think Americans are more interested about the future rather than the past in this election. I am fairly worried a pullout from Iraq will still leave a heavily balkanized country, rather ripe for a civil war from a historical perspective. Dividing neighborhoods by religion/ethnicity and putting up large concrete barriers between them might limit violence in the short term, but it doesn't make for a very cohesive nation. I mean look at this:

Poor design

2008-08-15T21:04:00.001-07:00

Sorry I haven't updated in a while, I have been busying myself with some reading and general related work. More and more I am putting myself into the position of a GIS developer rather than an operator - creating tools with ESRI's ArcObjects API. Whether this is "real" programming or not depends, I suppose, on your definition. I still consider myself something of a novice to the actual field of computer science, but I feel comfortable creating custom scripts and operator tools for ArcGIS in C#, Python, VB.NET, and T-SQL. I could say I am going to the trouble of learning all of this because it increases my earning potential, but it's actually a lot of fun. Computers are rather poor at what the human mind generally does - analysis and higher order modeling - and human minds simply cannot match the power of computers to instantly recall and organize memory (not to mention their potential for networking with the vast knowledge of the internet). Interfacing with a computer at progressively lower levels greatly increases the collaborative power that comes as a result of normal use, and it is a joy to be working in such a way.

The i9/11 and the iPATRIOT Act

2008-07-14T16:16:00.000-07:00

Excellent, excellent talk on the future of the internet with some very smart people. Jonathan Zittrain: Author of "The Future of the Internet and How to Stop It" (available free here). Lawrence Lessig: Lawyer and author of dozens of books about free culture, recently having switched from talking about copyright specifically to a more broad range of IT and government corruption issues. Vin Cerf: He basically invented the internet with two other guys. No, really. Currently he is Google's chief internet evangelical (that is his real job title). The whole video is good, but the thing that stands out the most is a comment made by Lessig - the PATRIOT Act wasn't written really quickly in the wake of 9/11, it was sitting in a desk drawer waiting for the opportunity to be passed by a frantic legislature. He asked Richard Clarke, former chief counterterrorism advisor to numerous presidents, if a version of the PATRIOT Act for the internet was waiting for some kind of major internet-based attack which could be compared to 9/11 in terms of disruption and financial loss. Clarke said that, without question, there was. The heart of the problem is our national infrastructure is based on a system that our representatives are largely entirely ignorant of. Here is Senator Ted Stevens demonstrating such ignorance. Here is John McCain admitted to not knowing how to use a computer, much less the internet. None of this would be a problem, except these are the kind of people that can (at least attempt) to legislate internet activity in the country. And when some major incident occurs, which is inevitable due to the open nature of the internet, they may kill the largest and most important communication/content/distribution instrument ever conceived by humankind.

Taking a picture of an image without using light that interacted with it.

2008-07-01T23:53:00.000-07:00

When you look at something - any object - your eyes are reading information from all of the light reflected off that object. A leaf appears green because it absorbs or scatters all other kinds of light.

Light actually consists of individual particles called photons. An odd thing about particles - if you "entangle" one particle with another, they still are effected by one another over great distances. Change the spin of one, and the other one reacts. It doesn't seem to make too much sense, but that is quantum physics.

So what happens if you could capture the information of light particles that are hitting an object, but you only have contact with their entangled friend? Turns out you can make an image out of it anyway. The information is, I am told, only useful if you get the other one back too, but it is still pretty nifty. Below is an image of a toy soldier they viewed by this indirect method:

This will be sort of a big deal, as in the future, it might allow for the ability to see through all sorts of things. I know is seems strange, since we only see a cloudless Google Earth whenever we want, but satellites still need clear days for good imagery. This might be another way of mitigating those pesky clouds.

The artificial barrier of licensing, a GIS/Surveying example

2008-07-01T01:19:00.000-07:00

There was news recently of a feature story pulled from a professional survey magazine because the work in question was, according to the State Licensing Board, depicting activities that should only be done by licensed surveyors rather than GIS professionals or anyone else. James Fee doesn't know what part of this mess is the worst, but I'd like to take an amateur stab at it. The artificial barrier that government licenses produce can in fact be a good thing when the occupation is such that a minimum standard is required to avoid large-scale fraud, but in so many cases could probably be done by private organizations. Many are put in place by a vocal and monied minority in an attempt to create what amounts to a cartel. I believe one example involved a manicure license that costed thousands of dollars. Professional survey licensing may fall under this, but my limited knowledge of that industry compels me to limit such rhetoric. This isn't the biggest problem with what occurred. Actually, the biggest problem wasn't even the suggestion GIS professionals - or for that matter, simply knowledgable members of the public with increasingly cheap GPS devices - are not competent to do location based field work. For so much geographic data, a 10m resolution is a godsend where previously no one was collecting data. And as more nations put up satellites it will likely become even finer resolution for smaller and more casual devices. This is not to suggest there isn't a place for professional surveyors - both for ultimate liability responsibility and expertise with the more effective tools and procedures. But the percieved elitism is somewhat disconserting. As a GIS/programming guy, I don't find anything I do so absurdly difficult that it would pain my eyes to read an article about an amatuer trying it. The biggest problem is that a state licensing board can effectively kill an article. There isn't any way in which this is a good thing, and I hope whoever has their hands on it leaks it to the internet.

Losing the Argument: Why the Democratic party is probably doomed

2008-06-30T23:33:00.000-07:00

Elections are not simply about the raw numbers or even the shifting of legislatures. They often represent the public's perception of right and wrong - a national discourse on where the country is headed and whether it should stay on that path. The elections of 2006 was a victory more about the excesses of the former Congress rather than the arguments for the current one. A good portion of the public felt betrayed by the selling of the war-turned-occupation in Iraq, by rampant mismanagement of federal institutions (FEMA being the poster child), and of the corruption and fraudulence of the individual legislators. I would submit there was not a kind of national argument made by the Democrats that framed the election beyond. It was basically: we don't like these guys and we want someone else to do something different. This is the opposite of the other blow-out election of recent history, the Republican takeover of the legislature in the mid 90s. Here, there was a concerted argument made for a reorientation of the path the nation was taking. It didn't make this argument by itself, but was the result of deep resentment by both social and fiscal conservatives for more than a decade driven by not just one or two wedge issues but a complete ideology. And the Republicans didn't simply take these issues individuals, but as a total and (relatively) coherent vision. The evidence this represented a general shift to the right is not difficult to find. The same has not happened in 2006. There was no Contract With America, no grand vision. Or if there was, it has been utterly decimated by triangulated, tactical voting with no thought to a long term strategy. By voting record, there is very little to distinguish the Congress of 2006 from their predecessor on the topics that were at the forefront of the election: the war in Iraq, civil liberties, agency competence, and accountability of the Bush Administration. Nothing at all truly significant was done on these fronts. And the Democrats in 1994 were in a much more powerful position with a popular president (who regardless largely conformed to the ideology of the opposition). Large scale ideological movements are an extremely powerful force in history: the liberal revolutions of the 18th and early 19th centuries which created this country and profoundly changed others, the rise of Facism and Communism in the early 20th century, and at a national level, the rise of the modern American conservative movement. These powerful forces are founded on intellectual arguments that were extremely compelling in their own time and place - some continue to be so and others have been refuted with horrible loss of life and human dignity, but it would unwise to doubt their power. Democrats have not made an intellectual argument on this vein and their majority will be ephemeral if they do not. Their activities on basic issues that were provided as the reasoning to vote for them - checking executive power, the war, securing civil liberties, were not really pursued, much less some larger ideological goal. It has become simply the party of everyone who isn't a Republican for whatever reason. When the excesses and unpopularity of the current batch of Republicans is forgotten, they will still have an argument. Even a poor ideology trumps none at all. They could fix this by coming together and deciding why they are against the things the Republicans are for and visa-versa. Right now it is a mishmash which suggests they could change their tune at the drop of a hat (and judging by their voting records, they will). Republicans can rightly attack them as having no core principles or values because as a party they don't seem to. Don't believe me? Try a Google search of "democratic party mission statement". The whole first page is either local party offices with statements that mean nothing (the Maricopa County party's mission is just to get elected; okay guys, what then?). The actual party website is not much better, nothing in their "What we stand for" page is an argument - never is it explained why they believe what they do and what that means for policy. The RNC website is no better, but their ideological argument was really made here. The original contract is actually kind of interesting to read because the parts about government transparency are so completely counter to the current party's actions when it last had the majority it is quite stunning.

Political spam emails and computer safety.

2008-06-23T21:28:00.000-07:00

"Fwd: FWD: Fwd: fwd: OBAMA TO DISARN THE MILITARY!" reads one of those mass spam political emails. Surely everyone has somehow received one of these at some point. No doubt they are extremely popular because they confirm already established assumptions about a candidate. The internet has a wonderful way of justifying any opinion or claim because all human beings are characterized by some measure of credulity. The body of the email contained a link to a video in which Obama was said to have laid out this particular stance. Long have the creators of malicious code used gripping headlines to pull unsuspecting individuals on to suspicious websites. The Storm Worm is a classic example that persists to this day, creating an army of zombie computers ready to attack anyone or anything for hire. The email that is the subject of this post linked back to a throwaway Wordpress site and was originally sent by an email address consisting of what appeared to be part of an MD5 hash. Besides involving Nigerian banks, there isn't much more an email can do to find itself on the radar of a moderately aware user. Before trashing it, I alerted everyone it was sent to it might be dangerous. The final humor of this scenario is the the person who sent this to me is still miffed I suggested it was a possible attack site. I had assumed they were already compromised and it was automatically being sent to everyone in their contacts list. It hadn't occurred to me it might just be stupid rather than a security concern. Better safe than sorry though - I recommend the new and extremely popular Firefox 3 web browser with Adblock Plus and the NoScript addons. (As an aside: the funny part is I would be really happy if this particular email was true. The United States spends more on its military than the entire rest of the world combined. I'm reminded of the words of Dwight D. Eisenhower, possibly the last competent person with some measure of integrity to hold the office of President:

Every gun that is made, every warship launched, every rocket fired signifies, in the final sense, a theft from those who hunger and are not fed, those who are cold and are not clothed. This world in arms is not spending money alone. It is spending the sweat of its laborers, the genius of its scientists, the hopes of its children. The cost of one modern heavy bomber is this: a modern brick school in more than 30 cities. It is two electric power plants, each serving a town of 60,000 population. It is two fine, fully equipped hospitals. It is some fifty miles of concrete pavement. We pay for a single fighter plane with a half million bushels of wheat. We pay for a single destroyer with new homes that could have housed more than 8,000 people. This is, I repeat, the best way of life to be found on the road the world has been taking. This is not a way of life at all, in any true sense. Under the cloud of threatening war, it is humanity hanging from a cross of iron.

This was no pacifist. He was Supreme Allied commander in the European theater and served with distinction before then. He'd never be elected today with these views, and anyone trying to claim we face a greater threat today than the Soviet empire at the end of the 50s is either fooling themselves or needs to pick up a history book.)

HeyWhat'sThat: The coolest mapping site you've never heard of

2008-06-12T11:08:00.000-07:00

What happens when someone has too much free time, programming talent, and a burning question? This.

Summary: HeyWhatsThat is basically a rather advanced map hack site which includes viewsheds linked with government/public name systems (names with xy data) so from a given point you can see all the major peaks nearby. The proprietor and designer, Michael Kosowsky, has steadily been adding stuff to it and it now includes pathing (which gives you an elevation profile), and a night view.

I'm moderately curious why Google hasn't bought him out/hired him/stolen the idea. As far as I can tell, Kosowsky built all the tools (minus the Google Map of course) from the ground up. It includes some interesting little quirks based on his response to user comments, like the way it handles the curvature of the earth and the fact it switches to metric if outside the United States.

(To be completely fair, you might have heard about this site. Several other bloggers, including Ogle Earth, have wrote about it.)

The viable solution to internet piracy

2008-06-10T17:28:00.001-07:00

The sad fact of the matter is piracy as it is currently defined cannot be stopped without enormous breaches of privacy. Absent the restriction of tools to even create programs that can effortlessly defeat protection methods, you can't stop people from copying digital information. Mostly because the mere act of being on the internet means you are required to copy things just to view/listen to them. This is a sample of some of the points of a paper I recently read at Cato called The Future of Copyright. My personal favorite part:

When American troops liberated the city of Luxembourg in 1944, they made a strange capture: a machine capable of recording sound on magnetic tapes. Shortly after the war, this German military invention made its appearance in private homes. Tape recorders integrated listening and reproduction in one device, but as separate functions. That’s no longer the case with digital technology. Today, to use digital information is to copy it.

Computers operate by copying. They couldn’t care less whether the physical distance between original and copy is measured in micrometers or in miles; both work equally well for them. Copyright law, on the other hand, must somehow draw a line between use and distribution. That means putting an imaginary grid over the chaotic myriad of network nodes, delineating clusters of devices that can be attributed to individuals or households.

Every person reading this article is actually copying it illegally. You can't help but do it. As you surf the internet your computer is constantly caching (saving locally) data that you are viewing. Anyone who has ever used the internet for anything is almost certainly a pirate. I'll let you all off with a warning this time. Humor aside, the problem is one that the music industry found itself in a long while ago with the invention and mass adoption of radio. John Philip Sousa was convinced anything but live shows would completely destroy music. The solution, after much complaining, was to just license the distributor. Now of course the distributor and consumer and creator is anyone and everyone on the internet. The compromise is an idea that is not new and is even talked about by both sides of the issue with increasing interest: mass public electronic media licensing. If everyone on the internet is a pirate, license everyone. Everyone who wants to buy in at least. You've solved 99% of your piracy problems by facing the reality of a system, the internet, that requires copying. You also don't even have to host the files themselves, users are more than willing to do so. The idea has merit but the problem is the pie to be divided. We need an organization as trusted as the Nielsen statistics are for downloaded media content to correctly award creators and their labels/studios/whatever. The only alternatives are to entire restrict piracy with the kind of locking down that would destroy the internet and privacy in general (people could still just rip stuff from friends/renters) OR to see an attempt to destroy internet and privacy while the major content holders die a slow and painful (for everyone involved) death at the hands of unrelenting technological innovation. Edit: Hahaha. Bonus quote from a Guardian article linked from the Cato piece:

"For somebody who has spent 30 years in the music industry, you instinctively know this stuff is going on. But when you actually sit looking at your computer and see a number that says 95% of people are copying music at home, you suddenly go, 'Bloody hell'," he said.

Turns out you could nuke the internet from orbit and the current copyright model is doomed. Note: Copying this work is no longer actually illegal in any sense, since I have licensed everything with Creative Commons.

Monday Links

2008-05-23T16:23:00.000-07:00

MS Server 2008 as an OS: Amusingly enough, for all the hatred generated for Windows Vista, a product with supposedly a virtually identical code base is whooping up on it as far as performance goes. Some folks in the comments have suggested this might be because Server 2008 doesn't have the more intrusive DRM that currently is the biggest reason I have no interest in upgrading ("Oh, so I get a performance hit, maybe two useful new features total, and one day you can just take over my computer to prevent me from copying stuff I bought legitimately? SIGN ME UP"). Server 2008 with some minor modifications can do just about everything that Vista can do (music, movies, games, file management) and still move faster. If this forces updates like Vista does and gets at all popular, I expect Microsoft will toss in the DRM stuff. They can't seem to help themselves, they really really want control over your computer and all the software on it. There is a lot of money in it, so I can see the motivation. Brain Rules: John Medina is entertaining in this authors@google video, explaining his new book on some of the things you can do to aid cognitive function, help in relationships, and live longer without dementia. Strange Maps: Exactly what it sounds like. Want a cheese map of Canada or a 1942 map of the "new world order" (a vision of a post war world from the very depths of the war)? This is the place to find it. Gas Prices relative to income: Who is getting hit hardest by gas prices (which is to say, what percentage of your income is being eaten up by it)? I am curious if this map takes into account the average (or median, as one might prefer) gasoline consumption of the areas. It doesn't appear to, but I will attempt to recreate it when I have time to see. Map of School Vaccinations: These are the areas to point fingers to when we see a resurgence of childhood diseases. Normally I would be live and let live about this kind of bottomless stupidity, but a population that will sustain a stable habitat for such diseases will allow said diseases to adapt and render the vaccines ineffective. Also, children should not be responsible to the mass idiocy of their parents. I believe one day I will make a density map of stupidity and this will certainly be one of the variables. Top Five Programming Languages to Learn: Just like learning a foreign language but probably worth far more money if you are really good at speaking it. C# tops the list and I would agree. You can use it to add on to anything having to do with Windows, use it for web applications, Silverlight (a .NET version of the ubiquitous Flash player) and understanding the libraries will come in handy even in an open source/Linux environment with Mono.

Geodata distribution, piracy, and the future business models.

2008-05-19T12:01:00.000-07:00

The big talk in the GIS world is the presentation Jack Dangermond of ESRI and John Hanke of Google at a recent conference (Where2.0). In it, they effectively outline the plan to do as much as possible to encourage the greater GIS community to openly offer their data to be served up by their products, which are going to be integrated in a much closer fashion when 9.3 comes out in a few weeks. A lot of firms and even public agencies offer geographic data for money, and they are probably going though a variety of stages of "freaking out". Already a lot of data that they could formerly milk has been put up in some form or another in Google Earth, VE, or NASA's WorldWind. Anyone who has taken a passing interest in the internet for the last decade can probably tell you this model, selling data for profit, is not going to work forever in the same way it always has. The sustainable model in the digital age is in offering expert, specialized services rather than trying to produce artificial scarcity via DRM or hiding yourself entirely behind a firewall. Ask the entertainment industry how the latter two options are working out. For geographic digital data, this is already happening. Go on any given piracy website and prepare to find the latest TomTom or other proprietary maps/data. Undoubtedly there are more specialized and discrete sites out there just for geodata (or if there isn't then there will be when the demand exists). This kind of thing can be mitigated by making your product very accessible (easy to buy) and superior to the free alternatives. Working with Google or other distributors to set up a one-click purchase system for geodata together with free samples and an open format. Continue to improve by offering semantic and interest associations (people who bought this also bought this, etc). Young people - time rich and cash poor - will still pirate the heck out of it. But when they join the business world they will be pitching your data to their bosses/clients. And they'll be buying it themselves because they will eventually be relatively poor in time and rich in cash. Selling to businesses will still be perfectly viable though (they assess the risk of piracy a little differently than individual actors), but the mass market is a different animal.

The developers of this program should be punished by being forced to use it frequently

2008-05-15T15:55:00.000-07:00

Microsoft's SQL Server Business Intelligence Development Studio is something you may never have heard of. Consider yourself lucky if that is the case. The stupidly long name is just the first in a series of entirely avoidable errors. Pretend you need to make a report - basically a table - and it combines three related tables. Simple, right? Just join the tables together and select whatever columns you want. Now add a wrinkle - the user gets to filter these results. If they only want rows related to a certain area, that is what they get. Here are the very stupid things about Visual Studio that makes it the worst environment to do this in.

SQL formatting options in this thing are utter trash. Have fun writing your queries when there is no color coding and no tab delimiting. Even better: whatever formatting you paste in is also going to be ruined. Intellisense, that super awesome thing that autocompletes common code and is one of the main draws of Visual Studio for other applications? Gone.
The visual tools are similarly crap. The whole drag-and-drop type of scripting is stupid on principle and apparently MS is trying to prove that point by making really bad tools to do it. Try adding a parameter into a WHERE clause in their "handy" little designer under Filter. Even if you perfectly type one that already exists, it will make a new one (throwing a 1 at the end to differentiate it).
Options, Preferences, Report Properties, Report Parameters, Customize, Settings, etc. Your UI team should be flogged for this kind of thing. The vast majority of these could be condensed into a single dialog with multiple tabs. Having 987,345 places to look to find some kind of preference or option makes users silently pray for your death. You can find this in a lot of Windows Apps and it is why people smarter than me are also telling you (more politely) to reexamine your UI decisions.

The benefits of being a late bloomer

2008-05-06T17:38:00.000-07:00

This isn't a post about an embarrassing teenage angst. It is actually a story about natural resources, the industrial revolution, and programming for Windows. Ars Technica, one of the better tech sites on the internet, recently did a series on the lead up to the most recent malaise we've seen with Windows programming. As I read the piece, I kept noticing parallels between Microsoft's difficulties and the early innovators in the industrial revolution. Despite the fact that you probably know nothing about the 19th century about Britain besides the fact the nascent United States kicked its ass off the better parts of North America, it is actually pretty relevant. Rule Britannia Britain ran the 19th century. It controlled all of the sea lanes, dominated world trade/finance, and ran nations which were ten times its landmass and population by occasionally machinegunning down crazy-brave hordes of natives who were attacking them with spears. It got this way because it not only was the first to recognize and reap the massive benefits from new industrial machinery, but also because it ruthlessly closed itself off in terms of technology to retain their advantage. A lot of the first major industries that started in America were based on stealing the technology anyway - not completely unlike piracy in China at the moment. In the late 19th century this initial advantage actually started to hurt them in places. A "second" industrial revolution started when better techniques for creating stuff like steel were developed. Nations a little behind the times compared to Britain could buy better stuff fresh without the perceived cost of replacing older equipment. A newly united Germany bears mention here. Relevance to Windows development Programming in Windows is a bunch of old equipment that needs replacing according to the Ars series (they are not at all alone in this suggestion). Old 16bit windows stuff will still work in the new 64 bit systems, but the odd things you have to account for to maintain this backwards compatibility makes programming new stuff for it a tremendous pain. Relatively late comers (in terms of popularity) like Apple and the more user friendly Linux distributions like Ubuntu can and do start from scratch in order to build better tools and not incur the kind of fallout that would happen if MS did the same. Apple pulled this of by creating a virtual machine for their old apps. Desktop Linux distributions don't really have to worry about this because their user base really isn't so large as to cause mass customer loss at breaking compatibility. The apparently sexier place to do development is for the web, which is OS neutral and can increasingly do stuff that was once limited only to the desktop. That stuff that still needs to be done on a local machine - heavy audio/image/video processing - have a lot of good tools on alternatives to Windows. The only exception here seems to be in the gaming industry, which in general steadfastly refuses to develop for multiple operating systems because, from what I understand, the open tools for rendering graphics (OpenGL) are more difficult to use than Microsoft's DirectX.

Geography, law, and leaving things up to the states to decide

2008-04-30T15:00:00.000-07:00

The problem here is not that some states are unusually incompetent or corrupt (I'm looking at you Texas), but because people move more frequently and federal law has ballooned into places where the original signers of the Constitution never envisioned. Take the current rather bigoted opposition to gay marriage and throw transsexuals into the mix: the equation becomes hilariously/tragically more complicated. Money quote from the NYT article on the subject (registration is free but I don't like to encourage sites to put up that trash, so I recommend BugMeNot, an addon for Firefox that you should try to only use legally):

Urging the United States Supreme Court to tackle the issue in 2000, lawyers for Christie Lee Littleton, a Texas male-to-female transsexual suing her husband’s doctors for wrongful death, noted the confused landscape: “Taking this situation to its logical conclusion, Mrs. Littleton, while in San Antonio, Texas, is a male and has a void marriage; as she travels to Houston, Texas, and enters federal property, she is female and a widow; upon traveling to Kentucky she is female and a widow; but, upon entering Ohio, she is once again male and prohibited from marriage; entering Connecticut, she is again female and may marry; if her travel takes her north to Vermont, she is male and may marry a female; if instead she travels south to New Jersey, she may marry a male.” The Supreme Court declined to take the case.

The real future of mobile GIS

2008-04-25T11:51:00.000-07:00

This. This is it. Here are a pair of guys that have integrated spatial objects into video, allowing a user to pan around with a Android phone (or any camera probably) and have that information appear. Next step, combining that functionality with an artificial contact lens and, if we felt like being really sci-fi, possibly thought activated controls. The technology basically exists but requires the engineering for the best practical applications and integration. All of it would be noninvasive - no need for Matrix-style headjacks or major surgery - and it could just be taken off when you don't want to expend the effort. Then, integrate this kind of thing with existing applications like Whereyougonnabe and Twitter. Never have trouble finding your friends at a social event again, always know the efficient bike/walk/car path to follow, and the ability to immediately search and find anything interesting you want nearby. You could search around you in a fashion that you would otherwise need the internet for, finding friends in a wide-Facebook/MySpace network style and nearby interests by drawing resources from the largest repository of human knowledge ever conceived. Interaction wouldn't be far away. See a bad driver on the road? Vote them down and people will know to avoid him/her. Some area particularly dangerous and you will know. Which bar/club is currently overcapacity and not worth going to visit until later? What food on the menu is actually good? The simple fact is that eventually the concept of "spatial data" will not be discernible as something independent because nonspatial data will be considered so worthless by comparison.

THE POLLS!

2008-04-22T12:54:00.000-07:00

This image is still less insulting than the comment I recently read on SA suggesting the only reason Clinton hasn't conceded (due to a nearly insurmountable Obama delegate lead) is that women are bad at math.

Edit: To qualify that "insurmountable lead" bit, lets take tonight's nearly double digit victory for Clinton into the larger perspective:

Obama now has 1723 delegates, Clinton has 1593.

There are 303 uncommitted supers and 408 unassigned pledged delegates, a total of 711. These means Clinton must now win 59.1% of the 711 remaining delegates to break even with Obama.

Before PA, he had 1650 to her 1508, and there were 869 delegates outstanding. At this point, she needed to win 58.2% of remaining delegates to break even.

So, after all is said and done in PA, Clinton must now take a 1% larger share of the remaining delegates to break even with Obama.

In a decade or two, anything less will be like being illiterate

2008-04-07T16:46:00.000-07:00

I casually linked a 12 year old who is doing a lot of development on a major Javascript framework in the previous post. Here he is giving a talk at Google on jQuery. The tools will only get easier and the next generation will have no problem picking it all up with their sponge-like brains. There are four consequences of this that I can immediately think of:

More open source development. It looks great on a resume, there is a possibility of making a big name for yourself (publish something in academic journals, giving talks at Google, etc.) the satisfaction derived from contributing, and naturally you, like everyone else, get the benefits of the improvements.
Possibly a citizenry that can more easily identify logical fallacies. Perhaps nothing is more important for a democracy besides free speech. As with the open source stuff, people should demand more transparency from their elected officials, and even if they don't get it though official channels, they will always find ways to fulfill this need.
A greater appreciation for mathematics. Hopefully higher level math taught earlier, when the kids can absorb basically any information.
More rational discussion of basic scientific facts about the world around us. Less of this tragic, mortifying stupidity.

What language should you learn first?

2008-04-07T10:07:00.000-07:00

The breakdown Read some stuff on the internet and you will see everyone has an opinion on which language you should learn if you are just starting to program. 1. Hippies: Python, Smalltalk, Visual Basic, Java. This school assumes most beginners don't need to know anything about pointers or garbage collection, and a lot won't at all. If they ever have to learn these things, they are better off picking it up after getting the basics of object oriented design and general logic down. 2. Masochists: C++, C. Learn the hard way and everything else will be easy. This school weeds out anyone not wholly committed to the idea of coding their whole lives. They'd have you do assembly if they had the time to proofread it and comment how terrible and hopelessly inept you are, assuming you can get anything to work. This method works best if you are a wunderkind that heads development on major Javascript frameworks at aged goddamned 12. 3. Pragmatists: C# or .NET in general, Javascript, maybe Java, whatever your company/school uses. It's possible you'll pick up the concepts necessary to be a decent programmer, but you need to be able to code several things right now and the theories of software architecture can take a back seat until you have more time. An of course there is mixing and matching, which is probably more common than any complete adherence to any school. My story I cut my teeth on C++ back when all the cool kids were doing it in high school, but when I started doing Running Start classes at the local community college in my junior year my embarrassingly poor math skills at the time precluded taking basically any CS course. I did manage to make a very inelegant Tic-Tac-Toe program with a pretty decent AI, but that was the sum total of my experience. The book I had probably wasn't a good idea to use (I picked it up at the remainder book store I worked for at the time). Later, when I got into Geography (and GIS specifically) I went through a slow evolution: from the moderately crummy visual ModelBuilder program in ArcGIS to VB.NET - the latter via a course my wife took before I did and aced - to Python, and finally now fooling around with .NET stuff but looking back at C++ again. Where to go now At the moment I am kind of between #2 and #3. The one professional coder I know insists I do some lower level (meaning, closer to raw machine language; meaning harder in general) language. Specifically C++, which he could then critique. This would be a really really good idea and is extremely nice of him to offer to do. Long run, I'll probably be a better programmer for doing it. At the same time though, I know a bit of C# already, everything in the office is .NET, and there is a really good chance I'll be using it very soon. Particularly because the small stuff I have written at work (mostly Python) is likely going to be kept around and maintained in some way. And that means moving it to .NET because no one really uses Python here. I'd have real projects to do as opposed to homework from my friend. Also, there are free in-house training programs for it. I'm almost certain to go with #2 because of the long term prospects of actually being good at this (which means versatility) and also because if I am hired on here as a GIS Developer they'll train me anyway. "Specific language is actually unimportant," I've been told. What matters is understanding what you are doing when you give a computer instructions, what object oriented design means, and good practices & algorithms. Anyone who knows the basics, logic, and elegance behind programming can pick up just about any major language in a day or two. My largely uninformed opinion on language C++ is actually scary as hell though, and never something I'd give a beginner. As other people have observed better than I could, C makes you do more work (garbage collection and memory management), higher level languages and .NET does it for you, and C++ appears to do it for you but can fail catastrophically if you don't know what is going on under the hood. Had I known about Python in my high school days I would have started with that, no question.

Perfect

2008-03-28T16:09:00.000-07:00

There isn't any way in which this does not represent the kind of custom software I have worked with in ArcGIS. It is the inevitable result of shifting user requests and lack of an integrated design from the start.

Hybrid firms and Proprietary Software Licensing

2008-03-12T16:22:00.000-07:00

This kind of thing is why I wouldn't want to open my own GIS oriented business with proprietary software.

ESRI have told us that as an educational charity we are no longer allowed to have an educational discount for using their software and, not only that, our license codes will cease to work at the end of this month. So, we have 3 weeks, with the Easter holidays in the middle, to extricate ourselves and our ongoing projects from ArcGIS and into something else or find the many thousands of pounds to buy the full licenses for all of our staff.

This is my general blog, so for those that don't know, ESRI is the "big dog" proprietary GIS software firm. At least in the United States, it is relatively accurate to think of them as the Microsoft of the computer mapping world (though little can really compare with the OS market share Microsoft commands). The problem here is not limited to ESRI. The idea of time-duration licenses for any good that is tempered by special discounts for educational institutions, charities, etc, is one fraught with difficulties. This is especially as you cross cultural/institutional lines (as is the case with the above example). You can see something similar with annual income taxes, the problems inherent there need really no detailed mention. The risk associated with a change in the licensing regime - specifically in discount - will always result in some firms that are blindsided with an outrageous (and possibly unintentional) cost increase. The risk associated with this happening - which includes not merely the higher licensing fee but also the cost to assess whether it is worth paying for - combined with often already high costs resulting from interoperability issues and current fees. Fair warning or not, a switch being forced by a change in fees is going to cost money. Basically: Cost of Being Locked In + Licensing Fees + Risk of Fee Changes You avoid all of that with open source software, though there is the question of maturity. QGIS, the likely contender for the heart of the user otherwise locked into ArcMap, has come a long way but my short experience with it wasn't entirely pleasant in terms of stability. For other situations, open source software is often equal or superior - Microsoft SQL Server has only just caught up with Postgres for spatial querying. It all depends on the given software. The costs of starting a business is already high. My hope is by the time I can start a GIS consulting firm there will be a simple, stable open source GIS platform/viewer. I don't need it to do a lot really, just display data and allow for easy editing. The spatial analysis can be done though some scripting with Python/SQL, and dissemination though the web with the future version of MapServer/OpenLayers. It sounds simple, and would be perfect for what I am doing right now, but who knows what GIS will be in the future. At the end of the day it is all about what the client wants.

Mapping Science Itself

2008-03-03T12:33:00.000-08:00

This is a Google Tech Talk called Scholarly Data, Network Science, and (Google) Maps. The creative use of maps in this presentation is impressive, but it is interesting to see that the vast majority of the time and effort to produce them consisted solely of data cleanup. This kind of data cleanup - specifically the kind that requires some expertise in the subject (in this case, the metadata associated with huge datasets of high variability), some technical skill to automate processes when you can, and recognizing when it is prohibitive to do so - is likely to be steady, if sometimes boring, work for as long as such data is in demand. I would be curious how much of it they have considered handing off to the crowd (though scholarly work may not really lend itself to the concept as well as, say, images do via something like Google's Image Labeler) Another interesting thing regarding this presentation is the composition of the team responsible for the maps. Geography is always billed as a highly interdisciplinary field and this is a perfect example (though you will notice an abundance of mathematicians, graphic designers/cartographers, and some computer science folks). These guys sell their maps here. All of the money goes to getting such maps into schools where they can do some real good.

My problem with Javascript

2008-02-26T14:32:00.000-08:00

Imagine a computer language that is used for almost everything on the internet that is interactive (seriously, try disabling it and going to your favorite sites). Also imagine it has the worst conventions for a language you can imagine, almost everything about its design being haphazard or shortsighted. I think you see where I am going with this. Apparently it is easier to get going as a beginner copying and pasting known code (say for mapping APIs used by all the cool, or at least, dominate kids). You can't really watch the experts in the language explain it without being struck by how many errors and outright mistakes there are still in there, destroying lives and sanity. JavaScript is probably just as powerful as he says, but if I could be using any other language to do the above online mapping I would probably take it. Additional: The best thing that can be said about Javascript is the fact it isn't VB. Everything that I've said about Javascript applies perhaps doubly for VB (VBA being the worst). I am by no means alone in this. RE: Disabling JavaScript on browsers. This is actually a very good idea in general. Start using Mozilla Firefox if you don't already, and install NoScript. It lets you very easily allow or not allow JavaScript to run on a given site (default off).

Links for Today

2008-02-14T11:50:00.000-08:00

Happy generic Hallmark created holiday everyone. Here is a list of interesting things.

The brilliant anti-corruption lawyer and author Lawrence Lessig on Obama. I apologize if at any time my rather strong support for Barack Obama has offended anyone who reads this. If you at all on the fence about this guy or just don't really know a lot about him, this is a good thing to see.
The Reuters news agency releases its semantic web tool. Put in a document of anything and it will spit out metadata, tags, and other information. Very handy for searches. If you could have this accept spatial data you could whip up a generic search tool like what appears on Google Earth.
Firefox 3 beta 3 is out. Faster and more stable than Firefox 2, which means several orders of magnitude better than IE7.
Miro. I don't care if I mentioned this in a previous links post, it is still awesome. Free shows and music videos (TED talks, PBS Frontline, The Onion).
The Escapist; funny game reviews. Funny even if you never played them or only have a passing interest in video games.
This comic is almost as misogynistic as the comment I left on my mother's blog.

Telcom immunity passes Senate, Clinton doesn't bother showing up to vote.

2008-02-12T16:06:00.000-08:00

The Senate bill includes every single thing Bush wanted. One senator was mentioning with glee that not one of the amendments that could have caused a veto made it through. McCain show up to vote for immunity, Obama voted to oppose it. Not even the amendment that stipulated that this was the only legal eavesdropping program the president can create passed (this was Dianne Feinstein's, to give her credit for not being a complete shill). The lack of that amendment means a president can just make up a new spy program if for some reason (s)he find this law to be too constricting. Knowing this congress, they would just make it legal afterward anyway. The only way you can justify the striking down of these amendments is if you honestly believe the president has the right to spy on any person for any reason. I will be highly amused if Clinton - who's campaign is already Nixonesque - gets the nod and the right-wing echo chamber suddenly reverses its position with the kind of amnesia normally reserved for budget, midday soap opera programming. Here is a petition to the House not to back down. Their bill isn't perfect, but it doesn't include immunity. I also recommend calling your representatives.

McCain effectively locks up Republican nomination

2008-02-07T15:12:00.000-08:00

And if Democrats were interested in beating him, they should basically just show this picture on every possible two dimensional surface.

There. Barring the unlikely event of a massive terrorist attack that can't be blamed on the administration (ha) or Iraq suddenly having obvious political progress (which would be wonderful in any case, though still doesn't justify the launching of the war) you just won the election. Except for 20-25% of this country that just can't learn from (or are too proud to admit) their mistakes, Bush is radioactive for McCain.

My new mapping project: IKEA

2008-02-04T16:30:00.000-08:00

This is addressed to all my friends that suggested going to IKEA: I hold you in utter contempt. I suggest chocolate to my wife to mitigate this. Las Vegas casinos would die to have they kind of layout your average IKEA store has. I've been to a great number of casinos and never have I experienced the seemingly impossible method in which IKEA can get you directly to the items you came in looking for (trashy and overpriced as their selection is) but make it completely impossible to leave without seeing everything else in their inventory. Somehow I also managed to pick the store all of their fattest and oldest patrons gravitate to as well. I want to map IKEA stores to see how they pull this off. It would be like the opposite of the efficiency networking studies found in journals depicting good city street layouts. I could create an objective metric to measure impassibility (or find an existing one) and measure IKEA stores, casinos, and other people walking places with a new method. I need a name for my metric. Something to capture how infuriating the experience is.

Polls are projecting Barack Obama as the Winner of my Heart

2008-02-04T16:08:00.000-08:00

He was against the war while Clinton was making a passionate speech for why it was necessary to give that decision to Bush, abdicating a key constitutional responsibility given to Congress (Section 8). He has the best policy on network neutrality and privacy rights - IE, keeping the internet from being a worthless telcom-dominated commercial enterprise. He has as much real political experience as Clinton (both relatively new Senators) and has actually gotten good government transparency legislation passed - a searchable database of where government money actually goes.

One sentence movie summaries

2008-02-01T21:11:00.000-08:00

I am Legend: I know when I build my impregnable zombie-proof fortress I'm gonna pack way too many explosives in the ground around it so it will blow out huge chunks of my own defenses and allow them to get in easy - it's really just common sense. American Gangster: ++ would sleep through again Juno: Accidental teen pregnancy has never been so funny. The fact this apparently didn't get a wider release is a testament to how stupid the movie industry is. Charlie Wilson's War: This is about a blow inhaling Democratic Congressman from Texas that kills the Soviet Union with his bare hands; if you need anything more to sell you on going to see it you are probably a dirty red. Michael Clayton: This is basically boring as hell until the very end and then Clooney has a scene that appears to have been written by Aaron Sorkin (meaning it actually had snappy, entertaining dialog). Fantastic Four Rise of the Silver Surfer: To everyone who paid money for the first one, and thus justified the production of this sequel: letters have been sent to your families instructing them to love you less.

RE: Arizona's new Immigration Law

2008-02-01T15:20:00.000-08:00

The Phoenix area at least gets a lot of its powerhouse economy from ruthlessly exploiting probably innocent prisoners in chain gangs and illegal immigrants. So the natural response to the housing crisis, which is probably going to bankrupt the state government by itself, is to toss one of these groups under a bus with an employee sanctioning law. I suppose if you were attempting to change one group into the other, this would be your plan. At the moment you can frequently find a free six month apartment lease as a prize in your cereal box...which you would get for free as part of the current sweetheart lease you signed rewarding you free food for a year and full use of the renter's daughter for your meth-fueled pornography business. Okay, admittedly such places are utter ratholes. But they drive the prices down for all other apartments nicely. They will all probably go away and the sudden absence of their residents will drive any number of other businesses to ruin. And all that is ignoring the substantial amount of taxes that immigrants paid (sales and even parole taxes with faked Social Security numbers). Now they will move home/to other states - with the businesses likely following them - and the ones that can't move will be forced to crime for survival. Addendum: This law wouldn't be nearly so stupid and shortsighted if there was an extremely easy mechanism to turn currently "illegal" workers into a guest worker/working-on-citizenship status and letting them stick around but with all of the advantages of them being on the books while still maintaining the necessary labor force. Of course the whole immigration system with its astoundingly racist quota system should be overhauled, but if you are going to pass a law to ruin a bunch of people's lives, you should attempt to provide a way in people guilty of what amounts to a rather minor crime (if 50+ million of the people in your country are guilty of something - alcohol consumption during prohibition, nonviolent drug use, minor [even accidental] personal copyright infringement - you really can't consider it a major crime).

The presidential debates

2008-01-31T18:17:00.000-08:00

I reject the moderator's questions and substitute my own canned, tested subset of my stump speech. Imagine all of the candidates doing that. There, I just saved you several hours of pretty boring television. That holds for when the nominees are decided too.

Telcom immunity

2008-01-27T13:44:00.000-08:00

This is an easy story to tell and it is really easy to choose a side. In short: the Bush Administration asked some telecommunications companies to spy on their customers illegally. The ones that did it profited from it and now they want to be immune to the lawsuits filed against them by their spied-on customers. Glenn Greenwald probably said it best:

Telecom immunity entails virtually every corrupt, defining aspect of how our political system works: Telecoms have poured money into the coffers of key Senators, who then dutifully became their key advocates. Telecoms have sent a bipartisan cast of lobbyists (former government officials, of course, with incomparable access), to pressure key Senators, who swing their doors open wide for those lobbyists. And immunity is the most extremely illustration of what Sen. Obama calls "Lewis Libby Justice," as Congress passes a law with no purpose other than to protect retroactively the most well-connected private parties from the consequences of their lawbreaking.

If you feel that large corporations should be immune to criminal and civil prosecution because they donate enough money to Harry Reid and Jay Rockefeller's campaigns, then by all means don't get involved. If you feel otherwise, I suggest you call them and ask them why they are trying so hard to push though immunity. Even if you support fewer checks on wiretapping for some reason, this is a highly corrupt way of going about it. If the Telcoms were doing nothing wrong, why do they need Congress-granted immunity? If the law needed to be changed, then that should have been done prior to the spying.

Newer operating systems: Initial impressions

2008-01-19T11:55:00.000-08:00

I've been using Windows XP for about half a decade or so, as have the vast majority of other computer users (at least in the United States). In response to a lot of press newer operating systems have getting, I've decided to take a look at them. Using a trial version of VMWare, I was able to review the following operating systems - trashing them as I saw fit - and then able to erase them without a care. These little reviews are merely initial impressions. I don't feel qualified to comment on operating systems from anything beyond a new user's perspective. The virtual machine in question is pretty standard, if not rather weak for newer computers. 16gb of disk space (more than enough for any of the operating systems) and 1gb of RAM (the minimum at least for Vista. Ubuntu 7.10: Yes, I am aware there are a variety of distro's and Ubuntu isn't necessarily the greatest and best at everything Linux has to offer. But this will be literally the first time for me using basically anything Linux. Originally I had problems with the installation of all things - lockups and such. I know that isn't common, but it happens that the iso from the Ubuntu website had some issues when I ran the image check utility it comes with. I redownloaded and reburned the disc a number of times following exactly the instructions on the site and had no more luck. This was a pity, as my wife's laptop was first on the list of actual hardware I was going to commit to Ubuntu (it needed a good reformat anyway). It was constantly freezing up during the installation - even directly mounted from the ISO - on the virtual machine. Apparently this problem isn't completely unheard of. Luckily you can still mess with Ubuntu because the Live disc starts up in basically a temporary Ubuntu install. Outside gaming, Ubuntu has all the functionality you would expect from a good operating system (mail, office software, internet browser, basic games including Sudoku) with some additional neat stuff to boot. Specifically, the Synaptic Package Manager lets you choose from programs you can search for, installs them, then adds them to the top applications menu. These are all community created software packages as far as I can tell, and include a lot of stuff I use already and general stuff like Ipod utilities. The only problem with these things is that there are too many potential options for a given task and they don't appear to be rated in some fashion (number of downloads, user rating). Most users will have little time to test each option individually. Windows Vista Ultimate: It is hard to come to the new Windows with a positive impression. XP still works just fine for everything I need to do on a computer for any reason, so why upgrade? Additionally, there are some security concerns - not from 3rd party malware, but from the kind of Trusted Computing content protection schemes available, ready at any notice from Microsoft, to disable software or media that isn't to their liking. I buy hardware because I feel I own it, to do with it what I will. Vista can, though it doesn't at the moment, take some of those classic property rights away from its users. With that in mind, I cut this Vista virtual machine off from the internet before I installed the OS. Why take chances? There are complaints online regarding the amazingly poor speed of Vista in loading up or such. I did not experience this, the install I used was somewhat stripped down using Vlite. This does stuff like disables Aero and (I believe) the indexing that tends to slow down navigation and booting. Everything was actually going swimmingly until I tried to get at the files in the host computer (to see what Vista could run and how fast). My efforts at setting up simple file sharing, like what is easy to accomplish between two XP machines or even XP and a lot of Linux stuff (including my modified Xbox) failed miserably. To get the files with Ubuntu was jokingly easy. Vista wanted to actually install something on the host machine to facilitate transferring files which could not defeat the purpose of using an encapsulated virtual machine more. So I couldn't really test anything out besides the stuff that came with the install. Which is a problem, there is a lot of software I use that is free and/or open source and may not necessarily function as well in Vista. Additionally, I don't want to have to contend with the possibility of DRM effecting the freedom I have with my media files, and now I can't test it. Synthesis Bottom line, these are the main concerns (in order of importance): 1. I don't have to worry about compatibility or DRM issues. 2. Performance. 3. New features compared to XP. What is going to make me switch? 4. The Pretty. It is my understanding Microsoft has really locked up much of the gaming market with DirectX and the result is Linux has to use WINE to effectively emulate (I know WINE folks hate that word, but from a user perspective I see no difference) major titles. This is fine if you, like me, enjoy older or less hardware intensive games (Red Orchestra, MMORPGs when I can get friends to play, HL2 and its derivatives). Truthfully, since starting to work full time, I've been using my computer more for general internet browsing, media storage, office tasks, and some GIS. Since that is the case, if I had to choose between the two OSs above based on my limited experience with them, I would probably go with Ubuntu. I'll never be able to afford ESRIs GIS offerings at home, so there I would stick to open source alternatives like QGIS and/or GRASS. For everything else they seem fairly similar, with Ubuntu beating Vista both in price and in the peace of mind knowing no one will be listening in, forcing updates which hijack the hardware I legitimately own, or viruses/malware (for which there are few things written for Linux). That is if I had to switch. XP still does all of the things above as well or, in the case of games, better. I also don't have to worry about some cool new thing I discover being incompatible or having to research every single piece of software on my computer before I can use or (or know it is worthwhile to use). Linux, or Ubuntu at least, is getting there is usability and addressing my computer needs, but it doesn't offer anything that I can't do in XP besides some malware/privacy stuff reassurance.

The Siege

2008-01-12T22:18:00.000-08:00

It took me weeks to hunt down the movie The Siege. I watch it now for the ridiculously prescient dialog (see below). It should have a warning label now: "Although this film involves muslim terrorism suspects being mercilessly tortured by the US Military, it isn't in fact a documentary." That is just the tip of the iceberg. There is a whole section about a protest march - against the occupying military presence in New York - invoking the chant "no fear". While there is no military presence in New York (though the police are getting pretty close to a military presence), such a protest would warm my heart. Our fear of terrorism is going to end up far more dangerous to this country than actual terrorism. We always say we are willing to bleed and die for our freedoms, this is basically the test. If the Military Commissions Act and Patriot Act are any indication, we fail that test handedly.

Favorite nerd stuff of 2007

2008-01-08T23:25:00.000-08:00

Yes, everyone seems to be doing something like this. And probably got it finished a week ago. So sue me.

StumbleUpon: I first heard about it as an up-and-coming search-like engine in Wired. Effectively, it is a psuedo search engine that relies on user generated interests - not passively as Google does with link backs, but with actual intentional voting. There is something similar on the open source front (Wikia). Link

xkcd: How did I miss this gem before? Link.

DS R4: Play movies, listen to music, emulate classic games, and back up your whole DS game library for a little more than the cost of a single DS game. No mincing words, you want one of these if you own a DS. If you plan on buying one you'd be a fool not to get an R4 (or an M3, which I've been told does basically the same thing). Link.

Mozilla Firefox 3 beta - It is still beta but amazingly stable. Automatically restarts with the tabs you had open when you closed it, prettier search bar, and way better multi tab support (no slowdowns or crashes with the 987,345 tabs I tend to have open). Bad news: perhaps it is just me, but the bookmarking isn't as good and more difficult to manage. Still really nice for beta.

A finished Methods section draft: It is a good feeling having this thing 1/2 to 2/3rds done. Lets see how viciously my thesis chair, Bob Hickey, tears it apart. I think I might post it and have the 0 people that visit the site mock it.

Ubuntu/QGIS: Though not ready to take over the huge ESRI-house I work for or actually threatening Windows desktop domination, the speed and good looks of the new open source GIS platforms that can be created for zero cost (and without having to reformat your computer - use use VMWare Player to create a virtual machine).

Death of DRM music: For those that don't know, DRM is the reason you can't move around music you buy off iTunes to whatever you want as many times as you like. All of the major music content holders have realized what people on the internet have been telling them for years - only legitimate customers are hurt by DRM. Pirates - a term with already includes probably everyone younger than 20 - simply hack it or rip CDs their friends get during parties. Video will be next as it is easily as trivial to defeat.

@GoogleTalks: It will be a supreme challenge for Google to keep its relatively good public image even as it grows more massive and run by more suits than engineers. They have given the internet many, many good things. One that is less commonly mentioned is this series of talks which include all of the major candidates, many famous/influential authors, and some other folks. Link.

Portal: Game of the year out of nowhere. Witty dialog, excellent ending, and interesting mechanics.

Online SQL Formatter

2007-12-27T08:15:00.000-08:00

I'm sure there are dozens of these around but turning ugly SQL statements into pretty ones is something that should be done every time for maintenance/upkeep purposes, so I don't care if I am repeating information for some people. My only worry is this will breed lazy formatting/convention use into me. Instant SQL FormatterM.

Pydev and Python

2007-12-22T09:19:00.000-08:00

I am not a software developer and I have no wish to be. However, I am young and being able to customize stuff in GIS is a very marketable skill. Python is an excellent language for this. It's been adopted as the primary scripting language of the industry leader (ESRI), it's popular, easy to learn, easy to maintain, and open source. The only downside is it can't access everything ESRI has implemented in their API - for instance, I am having to basically rewrite the Utility Network tracing tools so I can do some analysis on SRP's water system - but ESRI may not be the industry leader forever. In terms of simple viewers/online mapping it is getting mighty competition from NASA's WorldWind, Google's offering, and an increasingly good package of independently created but easily interoperable open source solutions. Any way it goes however, Python seems like it would help for scripting (unlike the alternative in VBA). JavaScript would be useful too, but I learned Python first and thus I've been spoiled rotten by a language that isn't jam-packed with issues (it is also two big scoops of ugly). Getting to the point, PyDev is an addon for Eclipse that gives you a superb platform for writing stuff in Python. I've been messing with it for barely an hour and it by all appearances is sleek, organized, and fast. It is probably overkill for my one-off scripting needs, but I'm loving the tabbed views, on-the-fly error checks (that are not the annoying crap you get in VBA), and the Outline view. It also does a bunch of other languages as well.

Open Government Act Revitalizes FOIA

2007-12-18T21:59:00.000-08:00

Good news for fans of a more transparent government (AP). For once the legislative branch of the United States does something right. Highlights: 1. "Presumption of disclosure" restored. Agencies should release information by default unless there is a reason not to. 2. It passed unanimously and with a large number of sponsors from both sides of the aisle (more likely to be veto-proof). 3. Twenty day deadline or the agency has to pay for the search and duplication fees - likely to lead to much faster turnaround time. 4. As mentioned in the AP story, this also works for government records held by third party contractors. I am proud but moderately surprised to see John Kyl (AZ) as one of the sponsors. I was pretty sure there would never be anything I would agree with him on. For once I can send him a note of actual support:

I'd like to express how happy I am with the Senator's support of the new Freedom of Information Act legislation he is sponsoring. A great republic such as ours requires accountability in its people and its government, and it sounds as if this will make it easier to assure such for the latter. Thank you.

What needs to happen now is the enforcement of current laws limiting the power of the executive specifically, rather than passing laws that justify the lawbreaking done by that branch. This is a good, though small, step away from the direction the country has been headed for six years.

Arctic may be ice free in four years.

2007-12-13T09:29:00.000-08:00

At least, in the summer (BBC):

Scientists in the US have presented one of the most dramatic forecasts yet for the disappearance of Arctic sea ice. Their latest modelling studies indicate northern polar waters could be ice-free in summers within just 5-6 years.

Once again, damn people with their "science" forcing my thesis into obsolescence before it is even finished. Sure, I could revise my methods section. Again. But by the time I am finished, there will be a new study which proves it actually happened 20 minutes ago. Think I am exaggerating? From the same article:

"Our projection of 2013 for the removal of ice in summer is not accounting for the last two minima, in 2005 and 2007," the researcher from the Naval Postgraduate School, Monterey, California, explained to the BBC. "So given that fact, you can argue that may be our projection of 2013 is already too conservative."

My methods are premised around a timeframe of about 50 years, as arctic ice was not expected to disappear until then (and thus free up Churchill for shipping year-round, or as close as matters for the economic impact). Now that could be compressed by more than 40 years and there is the additional possibility of a rapid collapse of the polar bear population followed closely by the ecotourism business it supports.

Google Earth locations now editable.

2007-11-19T14:39:00.000-08:00

It was only a matter of time really. You can now edit points on Google Earth. To prevent people from trashing what is a pretty decent database of locations, you can also immediately view the original location as an option. I'll probably have a lot to say about this when I get done with work, but my immediate thought is how much more open will it get (user uploaded polygons? lines?), what will be the synergy with Android (mobile computing will no doubt integrate GPS technology), and if you can add points/lines/polygons willy-nilly, when is the government going to take a keen interest in semi-anonymous, amateur members of the general public plopping down points on things they'd prefer wasn't easily searched for? Update: My comments above are probably premature. You cannot, as far as I can tell, add points to Google Earth (in the sense those points are immediately and automatically shared with everyone else). Though I will admit I haven't played with GE for a while. If you could add points like that, you would likely start having the Wikipedia effect - vandalism and advertisement would become quite rampant without dedicated administrators and probably a secondary staff of trusted volunteers. Since Wikipedia hasn't and quite honestly are unlikely to solve that problem, it becomes a question of the costs vs. the benefits. I would say it would be worthwhile if such user-created features were implemented such that the whole of them would be turned off as default. Some basic/super popular layer categories could be created ("good neighborhoods", "pleasant views", "hiking trails"). Anyone entering it of course could add dynamic tags that you could search for in another search tool. Update II: From CNET

To curb potential abuse, users aren't able to edit the location of a business that has already verified its location via Google's Local Business Center. There's also an official review system that has to double check your edit if it's more than 200 meters away from the original location.

Update III: Well check out how ridiculously behind the times I am. http://wikimapia.org/ This is pretty neat. Although a lot of the entries are clearly advertisements and and some vandalism, but that is the double edged sword of public created data.

Nature Conservancy and the Cumberlands

2007-11-15T15:59:00.000-08:00

The Nature Conservancy has just completed a massive land deal to save the Tennessee Cumberland from development. For those not familiar, the Nature Conservancy is the most successful conservation group you'll never hear about. The reason? They are not a very political group. What they do is purchase land development rights (or just outright buy the most important/endangered land). By that and other land protection mechanisms which are rarely legislative in nature, the Conservancy can achieve its goals (open space, environmental protection, biodiversity, etc) without being bent to the fickle will of legislators and election cycles. In terms of conservation, it doesn't matter who is in office as long as the property rights are enforced (and we will be having bigger problems if that isn't the case). This kind of thing is especially handy in situations where agriculture is being forced out by sprawl. Farmers see their property value rise and they can't pay the resulting taxes. Many enjoy farming but are simply forced to subdivide and sell some or all of their land to survive. Here is where the Nature Conservancy can step in and buy up development rights, which devalues the property and massively reduces the taxes. This is an amazingly simplified example, but one that is easy to conceptualize and I can't think of anyone that could possibly be opposed to such things. Everyone wins in such a scenario. Property rights are commonly referred to as a bundle of sticks. In the case of the Cumberland, the sticks the Conservancy have bought are often very specific. Logging companies were part of the deal and still own a lot of land - but they can by condition of contract only harvest new growth and must allow public access for recreation. The Conservancy retains the right to do any other kind of harvesting by what is called an easement (essentially a very specific property right) and it isn't too likely they'd ever give it up. This kind of "everybody wins" attitude coupled with sustainable practices and the preservation of natural beauty is why the Nature Conservancy is, if they needed some more GIS expertise, a job I would be willing to take a pay cut to accept.

Spending Iraq conditioned on withdrawl by Dec 08

2007-11-14T23:29:00.000-08:00

It seems the more centralist DLC Democrats have finally bowed to pressure from the rest of their party regarding Iraq. Having capitulated on or ignored every other issue - torture, wiretapping, habeas corpus, executive authoritarianism - the more pragmatic needs of being reelected in 2008 have edged out previous reservations on Iraq. I was always amazingly unconvinced regarding the excuses of this group of Democrats in failing to preform as they suggested they would in 2006. They must realize how unpopular their opposition is, and how they could stop the war instantly with the "power of the purse" invested in them by the Constitution. Even if you do not believe this is the right course, you must concede that the Democrat's in Congress deception on this issue lacks the transparency and honesty they had promised. For as defeated as the Republicans seem to be, their libertarian-authoritarian breaking points do not seem as wide as the gap between the DLC and the masses that elected them a year ago. As a side note, I would just like to add Harry Reid is a vapid, corrupt legislator. His continued support of the Pirate Act - demanding stiffer and criminal penalties regarding copyright infringement - demonstrates his surrender to special interests whose motives are entirely counter to the public good. Similar to how taxes work (at a very high tax rate, you can lower it and actually increase revenue by reducing the incentive to cheat and because there is more consumption and growth in general), extremely protected copyright rules are unenforceable and actually reduce the innovation they are meant to encourage. Wow that was a long side note. UPDATE on copyright sidenote: In his talks, Cory Doctorow has repeated one of his many complaints of the current copyright legalism - everyone is guilty of copyright infringement, intentional or otherwise, and (to quote him) "Once everyone is a criminal, no one is free." If you criminalize normal behavior there is no need to trump up charges to quash dissent. I'll admit at first I was skeptical of such rhetoric. The first part is undoubtedly true. In a very technical sense humming a few bars for a friend or the mere act of your computer passively caching pictures on websites as you surf could be considered copyright infringement. The second part - that it would be used to suppress free speech - I felt needed an example. The Russian Federation has generously provided one. Unquestionably this is simply the start, and you should take care not to fool yourself. This will happen in the United States. It wouldn't need to be by government, the private sector would have no problem causing a similar chilling effect on free speech.

Happy GIS Day

2007-11-14T23:00:00.000-08:00

An excellent post on Vector One

You may not realise it, but GIS is not just a technology; but it is built on the concept of integrating spatial thinking into everything that we do. This enables us to quantify and qualify how we live more closely, and to understand it. This is why you are seeing GIS embedded into the decision making processes of so many businesses around the world and reaching billions of people.

It is rather interesting to consider the future of GIS. Typically I do so to aid in my future employability or general usefulness at work - like learning Python and exploring new GIS software.

But a step ahead of all that is the implications of having very easy to use GIS technology available to everyone, even passively gathering useful data though ubiquitous GPS receivers now found on basically any consumer electronic device. Ignoring for the moment the massive impact on privacy - in an ideal world such data would be gathered in an entirely aggregate anonymous fashion - the possibility of harnessing crowdsourced information is stunning.

How much more accurate will traffic monitoring, supply chain management, and store spaces be? The applications of such masses of data are huge for site management and land planning as well; in addition no doubt to hundreds of other applications that we can't really begin to imagine.

One constant will be, baring a nuclear war or increasingly stupid stances that are being made on the freedom of the Internet, most of these applications will make their way online. Maybe not in browsers, perhaps to start in an open source application similar to how PDF files can be opened upon data retrieval by Adobe or Foxit or whatever else is out there.

I'd honestly write more on GIS but my current efforts are directed primarily at finishing my thesis. I eagerly await the time I will have after I finish it to read some GIS/Planning/Project Management literature.

Nanisivik chosen for Canadian Deepwater port

2007-11-08T17:54:00.000-08:00

Picture from the CASR.

A excerpt from my thesis in progress:
Canadian claims to the Arctic Ocean hinges on their retention of the massive northern archipelago ceded to them by the British in the 1880s. Efforts to enhance the strength of said claims include making the area a separate territory (Nunavut) and subsidies that drive population growth and the economy in the strategically chosen capital, Iqaluit. This falls under the stipulation by international treaty that remote islands and their coastal waters have their title established by a “continuous and peaceful display of state authority”. Besides perhaps Russia, Canada has been most visible with its Arctic claims due to the possibility of controlling access to the newly opened Northwest Passage. The Harper government has commissioned new armed icebreakers and a new deepwater port is planned for the military base on Nanisivik (CASR, 2006). Such shows of force combined with withdrawal from international judicial bodies are an attempt to compensate for the precariousness of Canadian arctic claims.

Land is one thing - comparatively easy to defend a claim on, but the extent of Canadian exclusive claims to the Northwest Passage are going to be difficult to sustain in the face of pressure from every other state with an interest.

Churchill isn't the only likely boom town

2007-11-06T17:02:00.000-08:00

They don't get as much press as I would like, and finding the ones I did took forever due to lack of easy to obtain data; fast growing arctic communities manage to occur in multiple nations and for a variety of reasons. One generally ignored player in the Arctic is the Danes, who have sovereignty over all of Greenland. One of its territories, according to The Economist, is exploding:

...warming is good for business. Unemployment in the town is zero. A glacier next to a nearby zinc and lead mine has retreated since the site closed in 1990, exposing an outcrop of metal-rich ore, where drilling will start again soon. Ships supplying the only factory in town, which processes the local catch for Royal Greenland, a huge state-owned prawn supplier, can now use the harbour throughout the winter (it was previously inaccessible for three months of the year). The warmer water seems to be bringing back the cod fishery as well.

Also greatly effected is the tourist industry. This is of particular importance to my thesis as it is currently the major base economic activity in Churchill.

But the tourist industry is warming fastest. Around 15,000 tourists visited last year and twice as many are expected this summer. Hotels are booming and additional tourist guides are being trained.

I do not expect this will happen in Churchill. The glaciers are long gone and the bears simply can't survive as they are accustom when the ice disappears. They can live like their extremely close grizzly cousins (whom they can mate with and produce viable and fertile offspring in the wild), but then they are not nearly so viable for ecotourism. That is, assuming the land-based environment can support the influx of suddenly starving bears. Perhaps whale watching and historic tourism - Fort Churchill and the old Cold War sites - will supplant the bears in the future. Greenlanders are also farming as their doomed Viking ancestors did during the medieval warming period, but it is rather unlikely the people will suffer a similar fate. The other side of the coin for Greenland is this screws the Inuit hard. Of all of the indigenous people's of the new world, they have perhaps been the safest due to isolation and the otherwise lethal climate they've adapted impossibly well for. The while the Vikings mentioned above starved to death, they lived comfortably in the same climate. You need not match the technology or organization of a group if they cannot survive in the very ground they live on. Or as Dennis Miller once put it, "Sure, the lion is the king of the jungle. But throw him in Antarctica and he is just some penguin's bitch."

Coast Guard Establishes Base near Barrow

2007-10-19T13:19:00.000-07:00

From the NYT, picture from the AP:

For most of human history, the Arctic Ocean has been an ice-locked frontier. But now, in one of the most concrete signs of the effect of a warming climate on government operations, the Coast Guard is planning its first operating base there as a way of dealing with the cruise ships and the tankers that are already beginning to ply Arctic waters.

The New York Times has been especially good about keeping up on arctic/subarctic news as it relates to warming. In fact, my thesis was in part inspired by the multipart series they put out on the subject - with some of the geopolitical consequences this blog attempts to address - a year or two back.

One key concern for the United States in regards to sealanes in the Arctic is the shared border with the Russian Federation. Russia has been particularily bold in its claims on the Arctic; often invoking Stalin's immense claims to the area (almost a half of the sea) from his commissioned expedition in 1937.

Still, the only major Arctic claimant that hasn't signed the UN Law of the Sea is the United States. Simplified, this treaty stipulates the continental shelf as the seaward border of the territorial claims that could be made. The result has been a scramble to chart the Arctic Ocean's bathymetry using deep sea sonar.

Regardless of the method of delinating the borders, it is only a matter time before conflict arises over the distribution of artic resources. With the big prize being undersea oil and natural gas, it would not at all be a stretch to suggest slant-drilling (if not feasible now for undersea oil exploration, it will likely be by the time large scale exploration ramps up) may be the cause of such a conflict. It undoubtably has been the given reason for a number of armed conflicts, the most immediate example being Saddam Hussien's justification for the invasion of Kuwait.

The North West Passage, Arctic Resources, and coming conflict

2007-08-31T23:27:00.000-07:00

I'm currently writing my thesis on the topic of land use in Churchill Manitoba. I can actually thank the New York Times - particularly their multipart series regarding the changes in the Arctic - for first informing on the subject. The "Great Game" was, in simplified terms, the Cold War of the 19th century. It was a face off between Britain and the Russian Empire and is rarely taught in any history course. It was omitted from the entirety of my history degree at CWU, even though it explains the British invasion of Afghanistan, the expansion of the Russian Empire in the effort to obtain a warm water port, and many other events). Now to compare it to the current state of Arctic claims may not be fair. Not every contest of independent nations is a "Great Game" or "Cold War" situation. What you seem to need is diametrically opposed ideologies and foreign policies designed to confound a specific designated enemy (proxy wars). That hasn't happened yet, but two factions seem to be gearing up for it - an energetic and increasingly fascist Russia and western nations that, while they bicker over the exact claims, are more likely to have a problem with Russia's far more extensive claims (all of the Arctic and good portions of Mars) and will band together to oppose them. The stakes are high enough to precipitate another "Great Game" scenario in the near future.

First post

2007-08-12T20:05:00.000-07:00

This is my first post on my new blog. I plan on talking about a variety of geography, but often just anything on my mind. I don't expect anyone will read this, but today I'll be talking about the sorry state the Republic. It's a fairly popular topic these days to be sure. Now more than ever because of an increasingly unpopular war, but it is interesting to consider we always seem to be perceiving ourselves in some kind of crisis. In the 80s/90s it was competition from the Japanese and a burgeoning crime rate, in the 70s it was the end of Vietnam, Nixon, outrageous inflation, and the resurgence of folk music as a legitimate thing to preform I would put forth it is the result of the younger generation at the time having finally grown old enough to recognize the extraordinary gap between what little they have learned in school about the founding and the course of the nation to this day. That isn't to in any way suggest what we are going though isn't really a crisis. The problem is the crisis isn't as noticeable and dramatic as the war itself. The problem is the precedent being set by this administration - the idea that the president is entirely above the law and not accountable to anyone. The best part is it is utter hypocrisy. So many of this White House's policies are centered around accountability - No Child Left Behind for instance - that only when it applies to them does it seem it unnecessary. It is sad, because many that work in government really do so out of a sense of civic virtue (it doesn't otherwise make sense that they work for the government, because they could be employed in the private sector for much greater compensation). These people are almost never elected, they have to be employed by their merits. I am privileged to work with folks like this at the Salt River Project, a federal reclamation district that serves the city of Phoenix and nearby towns. It has been my intention to work for a public agency of some kind because I feel I could best serve the interest of this nation (and therefore humanity in general - for though our hegemony is not perfect by any stretch, I believe incessant fighting between bickering nations of roughly equal power is certainly more likely to happen in the absence of said American hegemony and is less pleasant for everyone involved). Though I would hardly oppose working for a socially responsible private company.