h1

One mile South, one mile West, one mile North

July 22nd, 2014

This is another puzzle I found to be an interesting variation from the original that I have heard previously.

Basic puzzle: how many places are there on Earth (if any) that you can you walk one mile South, one mile West and one mile North and be back in your original position.

Answer A: 1 — right at the North pole, so you form a geodesic triangle.


“True” answer B: infinitely many – one mile above the the slice of earth that is one mile in circumference.

Scale is all wrong yada yada but the visual concept should be clear enough (I hope, and such is the intent).

h1

8, 9, 12 balls problem

July 21st, 2014

Lately I’ve been reading some common interview puzzle problems and came across the 8-ball problem. I’ve come across this problem before, along with some more complex variations. In particular, http://learntofish.wordpress.com/2008/11/30/solution-of-the-12-balls-problem/ already gives a very good explanation of the solution. My only complaint is that I found it a bit difficult to keep track of the numbering. Thus, this post is not much more than my personal rendition of the problem and solution, using colour rather than labels.

The most basic variation is that you are given 8 balls but one is heavier than the rest. You are asked to find the minimum number of times you would use a balancing scale such that you would definitely find the heavier one.

basic 8 balls problem

basic 8 balls problem

I think it helps when you know that the solution is two times and you get to work that out. Here’s a visual solution:

basic problem solution

Solution to basic problem

This is the same solution for 9 balls, one heavy — the only difference is the second weighing is always the bottom case. However, a more interesting variation is of the same practical scenario but this time you are not told whether the one ball is heavier or lighter, just that it is different.

One ball has a different weight

Less knowledge: one ball weighs differently

This time solution to this is three times — this is somewhat straightforward for the 8 or 9 balls case, as an extension to above. But for the 12 ball case, the reasoning is much more subtle.

Addendum: It has been almost a year since I’ve last visited this post, I might one day finish with the visual version of the 12 ball case, but for now, I’ll just have to leave published as is. The original link‘s explanation will have to do.

 

h1

Handy command line thingies

December 23rd, 2013

A compilation of handy commands I use when working remotely and/or on command line.

File sharing from remote server

I used to use scp a bunch for transferring files between my local machine and some server. However, I found that ssh file share is waaay handier.

Essentially, you define a local folder from which you mount a directory in a remote server by typing the following:

sshfs <username>@<servername.host.domain>: <name_of_local_folder>/

So, for example, here I’d use sshfs sammi@nonado.net: MyNonadoFiles/ in the location where my MyNonadoFiles/ folder is.

Search previous commands

Another handy shortcut I use a lot since learning to use the command line is reusing previous commands. Pressing the ‘up’ scrolling button is fine but if I’ve been typing a lot of cd here and there or running scripts repeatedly then it can take a while to find a specific command (such as the sshfs above) which I use seldomly but is super useful. Here’s a couple of shortcuts:

  • Ctrl-r + “what you’re searching for”
    • e.g. pressing Ctrl and r, and then ssh would find your most recent command that has ssh in it.
    • To look further back, keep pressing Ctrl+r
  • history -”number of lines”
    • e.g. history -100 returns your most recent 100 commands

 

h1

Code snippet: all paths between two nodes in a graph

January 13th, 2013

An adapted version (of code found at Python patterns) for finding all the paths between two given nodes in a NetworkX graph, with the additional limit of length k (where k is the maximum number of edges in the path).

def find_all_paths_lim(graph, start, end, k, path=[]):
  path = path + [start]
  if start == end:
    return [path]
  if not start in graph:
    return []
  paths = []
  for node in graph[start]:
    if node not in path:
      if len(path) < k+1:
        newpaths = find_all_paths_lim(graph, node, end, k, path)
        for newpath in newpaths:
          paths.append(newpath)
  return paths

So, for example:

import networkx as nx
G=nx.Graph()
G.add_edge("spam","white pudding")
G.add_edge("spam","black pudding")
G.add_edge("spam","sausages")
G.add_edge("white pudding","black pudding")
G.add_edge("egg","spam")
G.add_edge("toast","egg")
G.add_edge("beans", "egg")
find_all_paths_lim(G,"beans","black pudding",3)

gives you

[['beans', 'egg', 'spam', 'black pudding']]

and

find_all_paths_lim(G,"beans","black pudding",4)
gives you
[['beans', 'egg', 'spam', 'white pudding', 'black pudding'],
['beans', 'egg', 'spam', 'black pudding']]

Note: loops are omitted and works on MultiDiGraphs (i.e. directed and multi-labeled NetworkX graphs).

h1

Upgrading Ubuntu

December 18th, 2012

“Error during update

A problem occurred during the update. This is usually some sort of network problem, please check your network connection and retry.”

I haven’t upgraded my Ubuntu distribution since Lucid Lynx (10.04) and didn’t want to do a fresh install to get the current Precise Pangolin (12.04). This requires upgrading Lucid one intermediate distro at a time to get to my final goal. But, since nearly all the intermediate distros have become unsupported in the meantime, this means the standard way of asking the Update Manager do this all for me was a moot point. After a bunch of googling and lots of waiting (I had to go 10.04 -> 10.10 -> 11.04 -> 11.10 -> 12.04) I finally got there, despite some errors along the way.

In short, the main trick was this:

1. First, download the “Alternate Install CD” for the distribution version you want. Then:

2. (i) You can either burn the ISO onto a CD and upgrade by booting from it

or

2. (ii) Type in the terminal (don’t literally copy + paste the italics!):

  • sudo mount -o loop /Location/Name_of_downloaded_distribution.iso /media/cdrom0
  • gksu “sh /cdrom/cdromupgrade”

Remember to choose the upgrade without getting updates from the internet or there will likely be errors because it can’t find the no longer supported distribution.

I also ran into other errors that wouldn’t allow me to update but I can’t remember the precise details so I won’t bother trying to  dig it up. Something along the lines of removing unsupported software did the trick, I think.

h1

Network Data sources

July 19th, 2012

Data data data, it’s so much easier (and reassuring) to use data that is preprocessed and previously analysed. Here are some nice sources of them:

  • KONECT - The Koblenz Network Collection. These are proper big networks ranging from few ten thousands of nodes to a few ten million. The page layout is broken down by types of networks e.g. Authorship, Lexical, Ratings etc. with neat symbols beside the name of them to show what type of graphs they are, bipartite, directed, weighted, temporal etc.
  • Social network datasets - a page with datasets associated with some social network analysis course. Most of the datasets were originally collected by sociologists studying the behaviour of animals/people so are understandably small (<100 nodes). However, very useful as well-studied and ground truth is, well, well-grounded XD Format is less pretty – it’s pretty old-school in layout, links that are names of authors/collectors of datasets that goes to the section of the page that describes it.
  • Clique Datasets - a list of links to datasets that have been created by Clique researchers, which have been made available for academic use. A smaller collection of datasets, but well-documented with links to related publication and blog posts (when there is one).
  • Stanford Large Network Dataset Collection as well as their Web and Blog datasets - of which a bunch are part of the KONECT datasets. They are mostly large (again ranging from ten thousands to ten millions nodes) datasets from Jure Leskovec‘s page for their Stanford Network Analysis Package (SNAP).
  • A bit meta here and I link a couple of links with links:
h1

Webcomics and other miscellany (to keep me sane)

May 9th, 2012

Some miscellaneous comics which summarise:

h1

List of European Airports and their City Codes

March 1st, 2012

(Resource entry)

List of International Air Transport Association airport codes with their associated City or Airport name for geographically-defined European countries:

Albania, Armenia, Austria, Azerbaijan, Belarus, Belgium, Bosnia and Herzegovina, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Georgia, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Liechtenstein (not in database), Lithuania, Luxembourg, Macedonia, Malta, Moldova, Monaco, Montenegro, Netherlands, Norway, Poland, Portugal, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey, Ukraine, United Kingdom, Jersey, Guernsey, Isle of Man, and the Faroe Islands; Russia was omitted as its landmass spanned further East than was required.

These were taken from the database provided by OpenFlights.org in December 2010.

IATA Airport Code City/Name Country
TIA Tirana Albania
EVN Yerevan Armenia
GRZ Graz Austria
INN Innsbruck Austria
KLU Klagenfurt Austria
LNZ Linz Austria
SZG Salzburg Austria
VIE Vienna Austria
GYD Baku Azerbaijan
KVD Ganja Azerbaijan
NAJ Nakhchivan Azerbaijan
MSQ Minsk2 Belarus
ANR Antwerp Belgium
BRU Brussels Belgium
CRL Charleroi Belgium
LGG Liege Belgium
BNX BanjaLuka Bosnia and Herzegovina
SJJ Sarajevo Bosnia and Herzegovina
BOJ Bourgas Bulgaria
SOF Sofia Bulgaria
VAR Varna Bulgaria
BWK Brac Croatia
DBV Dubrovnik Croatia
OSI Osijek Croatia
PUY Pula Croatia
RJK Rijeka Croatia
SPU Split Croatia
ZAD Zadar Croatia
ZAG Zagreb Croatia
LCA Larnaca Cyprus
ECN Nicosia Cyprus
PFO Paphos Cyprus
OSR Ostrava Czech Republic
PRG Prague Czech Republic
BRQ Turany Czech Republic
AAL Aalborg Denmark
AAR Aarhus Denmark
BLL Billund Denmark
CPH Copenhagen Denmark
EBJ Esbjerg Denmark
KRP Karup Denmark
RNN Ronne Denmark
SGD Soenderborg Denmark
KDL ArmariAirForceBase Estonia
URE Kuressaare Estonia
EPU Parnu Estonia
TLL Lennart Meri Tallinn Estonia
TAY Tartu Estonia
FAE Vagar Faroe Islands
HEL Helsinki Finland
IVL Ivalo Finland
JOE Joensuu Finland
JYV Jyvaskyla Finland
KAJ Kajaani Finland
KEM Kemi Finland
KTT Kittila Finland
KOK Kruunupyy Finland
KUO Kuopio Finland
KAO Kuusamo Finland
LPP Lappeenranta Finland
OUL Oulu Finland
POR Pori Finland
RVN Rovaniemi Finland
SVL Savonlinna Finland
SJY Seinäjoki/Ilmajoki Finland
TMP Tampere Finland
TKU Turku Finland
VAA Vaasa Finland
VRK Varkaus Finland
AGF Agen France
AJA Ajaccio France
NCY Annecy France
AUR Aurillac France
AVN Avignon France
BIA Bastia France
BVA Beauvais France
EGC Bergerac France
BZR Beziers France
BIQ Biarritz-bayonne France
BOD Bordeaux France
BES Brest France
BVE Brive France
CLY Calvi France
CCF Carcassonne France
DCM Castres France
XCR Chalons Vatry France
CFE Clermont-Ferrand France
DOL Deauville France
DNR Dinard France
FSC Figari France
GNB Grenoble France
TLN Hyeres France
LTT LaMôle France
LAI Lannion France
LRH La Rochelle France
LEH Le Havre France
LIL Lille France
LIG Limoges France
LRT Lorient France
LYS Lyon France
MRS Marseille France
ETZ Metz France
MPL Montpellier France
MLH Mulhouse France
NTE Nantes France
NCE Nice France
FNI Nimes France
ORY Paris France
CDG Paris France
PUF Pau France
PGF Perpignan France
PIS Poitiers France
UIP Quimper France
RNS Rennes France
RDZ Rodez France
EBU St.-Etienne France
SNR St.-Nazaire France
SXB Strassbourg France
LDE Tarbes France
TLS Toulouse France
TUF Tours France
BUS Batumi Georgia
KUT Kutaisi Georgia
TBS Tbilisi Georgia
AOC Altenburg Germany
SXF Berlin Germany
TXL Berlin Germany
BWE Braunschweig Germany
BRE Bremen Germany
CGN Cologne Germany
DTM Dortmund Germany
DRS Dresden Germany
DUS Duesseldorf Germany
ERF Erfurt Germany
FRA Frankfurt Germany
FDH Friedrichshafen Germany
HHN Hahn Germany
HAM Hamburg Germany
HAJ Hannover Germany
HGL Helgoland Germany
HDF Heringsdorf Germany
HOQ Hof Germany
JUI Juist Germany
FKB Karlsruhe-Baden Germany
RLG Laage Germany
LEJ Leipzig Germany
LBC Luebeck Germany
MHG Mannheim Germany
FMM Memmingen Germany
FMO Muenster Osnabrueck Germany
MUC Munich Germany
NUE Nuernberg Germany
PAD Paderborn Germany
SCN Saarbruecken Germany
STR Stuttgart Germany
NRN Weeze Germany
GWT Westerland Germany
ZQW Zweibruecken Germany
GIB Gibraltar Gibraltar
AXD Alexandroupolis Greece
JTY Astypalaia Greece
ATH Athens Greece
CHQ Chania Greece
JKH Chios Greece
JNX Cyclades Islands Greece
HER Heraklion Greece
JIK Ikaria Greece
IOA Ioannina Greece
KLX Kalamata Greece
JKL Kalymnos Greece
AOK Karpathos Greece
KSJ Kasos Greece
KZS Kastelorizo Greece
KSO Kastoria Greece
KVA Kavala Greece
EFL Keffallinia Greece
CFU Kerkyra/Corfu Greece
KIT Kithira Greece
KGS Kos Greece
KZI Kozani Greece
LRS Leros Greece
LXS Limnos Greece
MLO Milos Greece
JMK Mykonos Greece
MJT Mytilini Greece
VOL Nea Anghialos Greece
PAS Paros Greece
GPA Patras Greece
PVK Preveza Greece
RHO Rhodos Greece
SMI Samos Greece
JTR Santorini Greece
JSH Sitia Greece
JSI Skiathos Greece
JSY Syros Island Greece
SKG Thessaloniki Greece
ZTH Zakynthos Greece
GOH Godthaab Greenland
UAK Narssarssuaq Greenland
SFJ Sondrestrom Greenland
ACI Alderney Guernsey
GCI Guernsey Guernsey
BUD Budapest Hungary
SOB Sármellék Hungary
AEY Akureyri Iceland
EGS Egilsstadir Iceland
IFJ Isafjordur Iceland
KEF Keflavik Iceland
RKV Reykjavik Iceland
NOC Connaught Ireland
ORK Cork Ireland
CFN Dongloe Ireland
DUB Dublin Ireland
GWY Galway Ireland
KIR Kerry Ireland
WST Leixlip Ireland
SNN Shannon Ireland
SXL Sligo Ireland
WAT Waterford Ireland
IOM Isle Of Man Isle of Man
AHO Alghero Italy
AOI Ancona Italy
BRI Bari Italy
BGY Bergamo Italy
BLQ Bologna Italy
BZO Bolzano Italy
BDS Brindisi Italy
CAG Cagliari Italy
CTA Catania Italy
CRV Crotone Italy
FLR Firenze Italy
FOG Foggia Italy
FRL Forli Italy
GOA Genoa Italy
SUF Lamezia Italy
LMP Lampedusa Italy
CUF Levaldigi Italy
EBA Marina Di Campo Italy
LIN Milan Italy
MXP Milano Italy
VBS Montichiari Italy
NAP Naples Italy
OLB Olbia Italy
PMO Palermo Italy
PNL Pantelleria Italy
PMF Parma Italy
PEG Perugia Italy
PSR Pescara Italy
PSA Pisa Italy
REG Reggio Calabria Italy
RMI Rimini Italy
FCO Rome Italy
CIA Rome Italy
TRS Ronchi De Legionari Italy
TRN Torino Italy
TTB Tortoli Italy
TPS Trapani Italy
TQR Tremiti Islands Italy
TSF Treviso Italy
VCE Venice Italy
VRN Villafranca/Verona Italy
JER Jersey Jersey
RIX Riga Latvia
KUN Kaunas Lithuania
PLQ Palanga Lithuania
VNO Vilnius Lithuania
LUX Luxemburg Luxembourg
OHD Ohrid Macedonia
SKP Skopje Macedonia
MLA Malta Malta
KIV Chișinău Moldova
MCM Monaco Monaco
TGD Podgorica Montenegro
TIV Tivat Montenegro
AMS Amsterdam Netherlands
EIN Eindhoven Netherlands
GRQ Groningen Netherlands
MST Maastricht Netherlands
RTM Rotterdam Netherlands
AES Alesund Norway
ALF Alta Norway
ANX Andoya Norway
BDU Bardufoss Norway
BJF Batsfjord Norway
BGO Bergen Norway
BVG Berlevag Norway
BOO Bodo Norway
BNN Bronnoysund Norway
VDB Fagernes Norway
FRO Floro Norway
FDE Forde Norway
HFT Hammerfest Norway
EVE Harstad/Narvik Norway
HAA Hasvik Norway
HAU Haugesund Norway
HVG Honningsvag Norway
KKN Kirkenes Norway
KRS Kristiansand Norway
KSU Kristiansund Norway
LKL Lakselv Norway
LKN Leknes Norway
MEH Mehamn Norway
MQN Mo I Rana Norway
MOL Molde Norway
MJF Mosjoen Norway
OSY Namsos Norway
NVK Narvik Norway
OLA Orland Norway
HOV Orsta-Volda Norway
OSL Oslo Norway
RRS Roros Norway
RVK Rørvik Norway
RYG Rygge Norway
SDN Sandane Norway
TRF Sandefjord Norway
SSJ Sandnessjoen Norway
SKE Skien Norway
SOG Sogndal Norway
SVG Stavanger Norway
SKN Stokmarknes Norway
SRP Stord Norway
LYR Svalbard Norway
SVJ Svolvær Norway
TOS Tromso Norway
TRD Trondheim Norway
VDS Vadso Norway
VRY Værøy Norway
VAW Vardø Norway
BZG Bydgoszcz Poland
GDN Gdansk Poland
KTW Katowice Poland
KRK Krakow Poland
LCJ Lodz Poland
POZ Poznan Poland
RZE Rzeszow Poland
SZZ Szczechin Poland
WAW Warsaw Poland
WRO Wroclaw Poland
IEG Zielona Gora Poland
CVU Corvo Portugal
FAO Faro Portugal
FLW Flores Portugal
FNC Funchal Portugal
GRW Graciosa Island Portugal
HOR Horta Portugal
TER Lajesterceira Island Portugal
LIS Lisbon Portugal
PIX Pico Portugal
PDL Ponta Delgada Portugal
OPO Porto Portugal
PXO Porto Santo Portugal
SMA Santa Maria Island Portugal
SJZ Sao Jorge Island Portugal
ARW Arad Romania
BCM Bacau Romania
BAY BaiaMare Romania
BBU Bucharest Romania
OTP Bucharest Romania
CLJ Cluj-Napoca Romania
CND Constanta Romania
CRA Craiova Romania
IAS Iasi Romania
OMR Oradea Romania
SUJ SatuMare Romania
SBZ Sibiu Romania
SCV Suceava Romania
TSR Timisoara Romania
TGM TirguMures Romania
BEG Beograd Serbia
INI Nis Serbia
PRN Pristina Serbia
BTS Bratislava Slovakia
KSC Kosice Slovakia
TAT Poprad Slovakia
ILZ Žilina Slovakia
LJU Ljubliana Slovenia
POW Portoroz Slovenia
ALC Alicante Spain
LEI Almeria Spain
OVD Aviles Spain
BJZ Badajoz Spain
BCN Barcelona Spain
BIO Bilbao Spain
RGS Burgos Spain
JCU Ceuta Spain
CQM Ciudad Real Spain
FUE Fuerteventura Spain
GRO Gerona Spain
GRX Granada Spain
LPA Gran Canaria Spain
VDE Hierro Spain
IBZ Ibiza Spain
XRY Jerez Spain
LCG LaCoruna Spain
ACE Lanzerote Spain
LEN Leon Spain
ILD Lleida-Alguaire Spain
ABC Los Llanos Spain
MAD Madrid Spain
AGP Malaga Spain
MLN Melilla Spain
MAH Menorca Spain
MJV Murcia Spain
PMI Palma de Mallorca Spain
PNA Pamplona Spain
REU Reus Spain
SLM Salamanca Spain
EAS SanSebastian Spain
SPC Santa Cruz De La Palma Spain
SDR Santander Spain
SCQ Santiago Spain
SVQ Sevilla Spain
TFN Tenerife Spain
TFS Tenerife Spain
VLC Valencia Spain
VLL Valladolid Spain
VGO Vigo Spain
VIT Vitoria Spain
ZAZ Zaragoza Spain
AGH Ängelholm Sweden
AJR Arvidsjaur Sweden
BLE Borlange Sweden
GEV Gallivare Sweden
GSE Gothenborg City Sweden
GOT Gothenborg-Landvetter Sweden
HFS Hagfors Sweden
HAD Halmstad Sweden
HMV Hemavan Sweden
JKG Joenkoeping Sweden
KLR Kalkmar Sweden
KRN Kiruna Sweden
KRF Kramfors Sweden
KID Kristianstad Sweden
LPI Linkoeping Sweden
LLA Lulea Sweden
LYC Lycksele Sweden
MMX Malmoe Sweden
SQO Mohed Sweden
MXX Mora Sweden
NRK Norrkoeping Sweden
ORB Orebro Sweden
OER Ornskoldsvik Sweden
OSK Oskarshamn Sweden
OSD Östersund Sweden
RNB Ronneby Sweden
SFT Skelleftea Sweden
ARN Stockholm-Arlanda Sweden
BMA Stockholm-Bromma Sweden
NYO Stockholm-Skavsta Sweden
SDL Sundsvall Sweden
EVG Sveg Sweden
TYF Torsby Sweden
THN Trollhattan Sweden
UME Umea Sweden
VST Vasteras Sweden
VXO Vaxjo Sweden
VHM Vilhelmina Sweden
VBY Visby Sweden
ACH Altenrhein Switzerland
BSL Basel-Mulhouse-Freiburg Switzerland
BRN Bern Switzerland
GVA Geneva Switzerland
LUG Lugano Switzerland
ZRH Zurich Switzerland
ADA Adana Turkey
ESB Ankara Turkey
AYT Antalya Turkey
BAL Batman Turkey
BJV Bodrum Turkey
CKZ Canakkale Turkey
TEQ Çorlu Turkey
DLM Dalaman Turkey
DIY Diyabakir Turkey
EZS Elazig Turkey
ERC Erzincan Turkey
ERZ Erzurum Turkey
AOE Eskissehir Turkey
GZT Gaziantep Turkey
HTY Hatay Turkey
ISE Isparta Turkey
IST Istanbul Ataturk Turkey
SAW Istanbul Gökçen Turkey
ADB Izmir Turkey
KSY Kars Turkey
ASR Kayseri Turkey
KYA Konya Turkey
MLX Malatya Turkey
MZH Merzifon Turkey
MSR Mus Turkey
NAV Nevsehir Turkey
SZF Samsun Turkey
GNY Sanliurfa Turkey
SIC Sinop Turkey
VAS Sivas Turkey
TZX Trabzon Turkey
USQ Usak Turkey
VAN Van Turkey
UKS Belbek Ukraine
CWC Chernovtsk Ukraine
DNK Dnepropetrovsk Ukraine
DOK Donetsk Ukraine
IFO Ivano-Frankivsk Ukraine
HRK Kharkov Ukraine
IEV Kiev Ukraine
KBP Kiev Ukraine
LWO Lvov Ukraine
ODS Odessa Ukraine
SIP Simferopol Ukraine
UDJ Uzhgorod Ukraine
ABZ Aberdeen United Kingdom
BRR Barra United Kingdom
BFS Belfast United Kingdom
BHD Belfast United Kingdom
BEB Benbecula United Kingdom
BHX Birmingham United Kingdom
BLK Blackpool United Kingdom
BOH Bournemouth United Kingdom
BRS Bristol United Kingdom
CAL Campbeltown United Kingdom
CWL Cardiff United Kingdom
DSA Doncaster, Sheffield United Kingdom
DND Dundee United Kingdom
EMA East Midlands United Kingdom
EOI Eday United Kingdom
EDI Edinburgh United Kingdom
EXT Exeter United Kingdom
GLA Glasgow United Kingdom
GLO Glouchestershire United Kingdom
HUY Humberside United Kingdom
INV Inverness United Kingdom
ILY Islay United Kingdom
KOI Kirkwall United Kingdom
LBA Leeds United Kingdom
LPL Liverpool United Kingdom
LHR London-Heathrow United Kingdom
LTN London-Luton United Kingdom
STN London-Stansted United Kingdom
LCY London-City United Kingdom
LGW London-Gatwick United Kingdom
LDY Londonderry United Kingdom
MAN Manchester United Kingdom
MSE Manston United Kingdom
NCL Newcastle United Kingdom
NQY Newquai United Kingdom
NRL NorthRonaldsay United Kingdom
NWI Norwich United Kingdom
PPW PapaWestray United Kingdom
PZE Penzance United Kingdom
PLH Plymouth United Kingdom
PIK Prestwick United Kingdom
NDY Sanday United Kingdom
SOU Southampton United Kingdom
SEN Southend United Kingdom
ISC St Mary’s United Kingdom
SYY Stornoway United Kingdom
SOY Stronsay United Kingdom
LSI Sumburgh United Kingdom
MME Durham Tees Valley United Kingdom
TRE Tiree United Kingdom
TSO Tresco United Kingdom
VLY Valley Holyhead United Kingdom
WRY Westray United Kingdom
WIC Wick United Kingdom
h1

Plotting Heatmaps in R

January 24th, 2012

I had recently had to create a heatmap visualisation as a part of our results in a paper we had submitted for a conference and as it took way more time than I had anticipated, I figured it’s something worth documenting. My first point of call was obviously Sgt. Google and the first hit given was How to Make a Heatmap – a Quick and Easy Solution, which I naturally liked since the sample dataset was a basketball stats one :D However, I quickly realised that this was not enough for what I wanted – my x-axis showed time and instead of nice, fat blocks, my heatmap/graph showed thin, coloured lines. Another problem I ran into was interpretation. I initially had something like this:

n = 50
matrix_to_be_plotted <- rnorm(n*n) # generate 50 x 50 = 2500 random numbers
dim(matrix_to_be_plotted) <- c(n,n) # change vector to matrix of dimension 50 x 50

heatmap(matrix_to_be_plotted, # as name suggests, the matrix of the data to be plotted
scale = “row”, # this is important; I did not realise this at first and spent an evening wondering why data values did not match with what I was describing in the heatmap (I’d assumed the darker the shade, the higher the value). Basically, you can control colour scaling by row or column, default is row. This is so important in my opinion that I’ll quote the documentation: “character indicating if the values should be centered and scaled in either the row direction or the column direction, or none. The default is “row” if symm false, and “none” otherwise.”
main = “Greyscale heatmap – squares”, # name/title of figure
Colv=NA, Rowv=NA, # set to NA or columns/rows of matrix would be rearranged into hierarchical clusters according to R’s dendrogram function
margins=c(3,3), # column/row margin space, the higher the number, the more space you get
cexCol=1, cexRow=1, # size of column/row labels, 1 is default
col=grey(seq(1,0,-0.01)) # colour is greyscale, sequence from 1 (black) to 0 (white) in steps of 0.01
)

which was a grey-scale version of what I wanted. You can read the documentation of the parameters of R heatmap, but my own explanation/interpretation of the parameters in the context of what I’ve given is written.

Here’s a picture of what should come out (tips on Saving Plots in R):

Example of Greyscale heatmap - squares

Not bad: Example of Greyscale heatmap, square matrix

This doesn’t look so bad, and if this is all you need, great. However, here’s a rectangular matrix, i.e. more similar to what I had in originally, which can be emulated by changing the dimensions like so:

dim(matrix_to_be_plotted_thin) <- c(n/2,2*n)
colnames(matrix_to_be_plotted_thin) <- rep(” “, 2*n)
colnames(matrix_to_be_plotted_thin)[seq(1,2*n,3)] <- paste(“Wk”, seq(1,2*n,3))

The column names are changed so they don’t get so squished on the x-axis:

Example of Greyscale heatmap, non-square matrix

Kind of Ugly: Example of Greyscale heatmap, rectangular matrix

This is less visually appealing. Moreover, it didn’t suit my graph because looked like this:

Greyscale insufficient

Yuck: My own graph, Greyscale insufficient

Part of the problem was that I didn’t realise I hadn’t the scaling properly – by default it was row-scaled, I was describing it as it were column-scaled. However, in retrospect, it still wouldn’t have worked if I had the correct scaling because this, as a figure taking up about barely one-sixth of a page, it was fairly difficult to read when printed. (I couldn’t figure out how to get a border around the graphic either so if anyone knows, please comment and let me know!) Anyway, my Saturday morning thus became an online google image and documentation hunt with keywords of R, heatmap, image, matrix image, visualisation,  <what have you>, and finally I converged to using an upgraded package of heatmap (library being heatmap.plus) with gplots – to prettify the graph with custom colours.

This apparently Easy Guide To Drawing Heat Maps To PDF With R (With Color Key) was a great starting point, but ultimately, I found that the full heatmap.2 documentation combined with this colorRamp pdf most useful as they actually explained what you needed to do to customise.

A slight detour – colour

I am a bit particular about my colours, both from an explanatory viewpoint and from an aesthetic one. I utterly despise bad graphs and badly-coloured ones even more so. In academia (or any good documentation that requires printing out), the best thing to do is to have graphs free of colour, such that it still makes perfect sense when printed in greyscale, and I do prefer that. However, when needs be for colour, I think it’s important to get it right, e.g. the most common type of colour blindness is red-green, so avoid using those two for distinguishing.

This arbitrary compulsive requirement of mine lead me to actually create my own palette for my heatmap…(Yay, as if I don’t have enough ways to spend my time.) More importantly, graphs are supposed to be a more efficient way of explaining data, not made just for the sake of them. If I am going to use colour in my graph, there should be a reason for it, and it shouldn’t require more text to explain why. (Picture – a thousand words – that sort of thing.)

A useful resource for creating your own palette is to look at this R colour chart. Here are a few examples:

TestPalette <- colorRampPalette( c(‘aliceblue’,'aquamarine1′,’azure3′,’blue’,'blueviolet’, ‘darkcyan’,'darkblue’,'darkgreen’,'darkmagenta’,'darkolivegreen’, ‘darkmagenta’,'darkviolet’,'black’))
WarmPalette <- colorRampPalette(c(‘antiquewhite’,'pink’,'rosybrown3′,’rosybrown4′,’saddlebrown’,'brown’,'black’))
CoolPalette <- colorRampPalette(c(‘lavender’,'mediumslateblue’,'blue’,'turquoise4′,’seagreen2′,’seagreen4′,’black’))
BluesPalette <- colorRampPalette(brewer.pal(9,”Blues”))(100)

brewer.pal is the in-built palette (as described in detail in the colorRamp pdf I linked earlier) – the handiest query for me was:

display.brewer.all()

which shows the name of the palette (like “Blues”) and the range of colours in it. Also, (100) in the BluesPalette example gives how fine you want the shading to be. So, if you had (3) then there’d be three varying shades of blue of something like Dark Blue, Blue, Light Blue, (100) gives you on hundred shades varying from dark to light.

Finally…

The code for my final plot and comments for explanation of new things:

library(gplots) # for colour panel of heatmap
library(heatmap.plus)

heatmap.2(MyMatrix,
Rowv=FALSE, Colv=FALSE, dendrogram= c(“none”),
cexCol=1, cexRow=1,
key=TRUE, keysize=0.1, # display colour key
density.info=c(“none”), # options of different plots to be drawn in colour key
trace=”none”, # character string indicating whether a solid “trace” line should be drawn across ‘row’s or down ‘column’s, ‘both’ or ‘none’.
margins=c(5,9),
lmat=rbind( c(0,3), c(2,1), c(0,4) ), lhei=c(0.2, 8.5, 2), # where to display colour key
col=CoolPalette # custom colours for colour key
)

….and the result:

heatmap_cool_palette

Much nicer: Heatmap with Cool Palette colours

My only and final annoyance with this is the “Value” which floats a bit too near the displayed numbers, but I don’t think it impedes on the readability so much that it’s worth unnecessary tweaking.

Funnily enough, the greyscale versions don’t look as bad on screen as a blog post. But trust me, it makes one hell of a difference on paper.

h1

Web Science Doctoral Summer School 2011 (belated)

August 26th, 2011

Here’s a somewhat very belated post on the Web Science Summer School I attended over a month ago (July 6, 2011 – July 13, 2011).

My thoughts? It was great! Fortunately, there were also enough others who felt compelled to write about it so that I don’t feel a huge urge to have a massive brain squeeze of whatever memories I have of it. That said, there’s always room for a few (short) thoughts.

It was an intense week of lectures/tutorials, often starting at 9am and finishing at 6pm, which meant many of us were very much drained at the end. Tiredness aside, the highlight for me (minus the social bits) was the mini-project. Over the course of a few days, we collected different types of communication data (specifically, face-to-face communication, facebook & twitter) and pretended to be web-social scientists and tried to make as much sense of it (until silly hours). With a bit of help of compewters and algorisms to make it all sound legit, of course. (Actual details on it can be found at Aaron’s post, which I’ve linked below.)

Here’s one example of what coordinated collective action at this school resulted in:

Some other people’s posts (thoughts) on the school:

  • Clare Hooper’s personal perspective on it, as well as the start of it (with subsequent posts on the attended lectures – well worth a read as summaries to decide if you want to watch the allotted ~1.5 hours for each talk).
  • Aaron McDaid and Owen Phelan‘s posts on the school – the former has more details of our team’s mini-project.

Resources:

  • Here are some links to the lecture/tutorial videos and the corresponding slides.
  • Here are also some flickr photos of the school.
  • In particular, very useful supporting material to the tutorial given by Derek Greene, post-doc at UCD. (Not just promoting a colleague ;) It really is a good intro. to important concepts in social network analysis and relevant software (networkx, gephi), with concrete examples and datasets.)

Oh, and my favourite quote of the week (and aspire to partially be – i.e. automate as much ‘trivial’ stuffs as a I can so that what I am is the creator/storyteller rather than command follower (explanation especially for Ursula, even I’m not usually this pedantic!) ):

“You are what you do not automate” — Marc Smith