Assuming you have the Base GIS stack installed, and you have a shiny Ubuntu desktop, then you might as well go ahead and install some more stuff.

QGIS

QGIS is a nice desktop app. Easily view/edit your vector and raster data, including data stored in PostGIS. Also some nice GRASS integration, not to mention the new python scripting.

  • Modify your /etc/apt/sources.list and put in the following line near the end
    deb http://ppa.launchpad.net/timlinux/ubuntu gutsy main
    
  • run the following to update your sources list and install QGIS
    sudo apt-get update
    sudo apt-get install qgis
    
  • Now QGIS will get nicely updated whenever you run your general Ubuntu updates

GRASS

  • Grass is awesome, and does all the analysis you want, though it has a bad reputation for user-friendliness. Now its actually got a nice UI, but we'll these instructions are for accessing it's functionality via QGIS anyway. After installing QGIS, just install the grass plugin:
    sudo apt-get install qgis-plugin-grass 
    
  • That was easy. Now you have access to all the GRASS kernel functions, with a nice UI. Enjoy!