Notes on "What is Hadoop? Other big data terms like MapReduce? Cloudera's CEO talks" video

• Map reduce - spread processing of data over many computers so data can be processed in parallel with cheaper hardware.  No transactions or schema.  Aimed more at analysis (Reads) rather than full CRUD?
• Relational databases not good for unstructured free text  
• Hadoop - open source, consists of (1) distributed file system to spread out data (HDFS), (2) way to push code down to do data analysis on the data 
• Scalable because can just drop in more servers 
• Memcached - in memory cache of relational database, push through writes "incrementally"? 
• NoSQL - distributed hash tables
• Sharding (not discussed in video) - taking rows of a relational table and distributing across computers

Altova MapForce review and tips

Altova MapForce is an interesting tool for creating XSL transformations visually.  Looking past the obvious drag-and-drop nature of the tool, here's my review:

What I liked:

• Promotes the creation of libraries of re-usable functions.  
• Flexible:  Can swap in another XSD without losing mappings, by editing MFD file directly.  The invalid mappings will be highlighted in red for correction

What I disliked:

• Generated XSL is not meant to be tweaked, and it would be very difficult to do so
• No built-in debugging facilities
• No support for non-function-based facilities (e.g. xsl:analyze-string)

Hard to say whether good or bad:

• Everything that is re-usable is function-based, that is, consisting of input (optionally) and an output.  It reminds me of Lisp programming.   

 Tips:

• do not copy and paste functions between MFD files using text editor
• make higher-level functions if find repeating mapping
• If parsing the text of a node which has embedded tags (e.g. <addr>PO Box 123<delimiter />c/o Dave<postalCode>90210</postalCode></addr>), in this case, addr and delimiter, respectively, use the MapForce distinct-values function

The container 'Maven Dependencies' references non existing library

If you copy an Eclipse-Maven project from one computer to another, you may get a Build Path Problem resembling:  "The container 'Maven Dependencies' references non existing library 'XXXXXXXXXXXX'".  This is obviously due to a difference in the location of the local Maven repository from one computer to the other. 

To fix this, right-click on the project > Maven > Disable Dependency Management.  Once Eclipse has finished rebuilding the project, right-click on the project > Maven > Enable Dependency Management.  This error should now be gone.  

Maven error "Cannot find wagon which supports the requested protocol: ftp" and Hudson

It is important to understand that Hudson comes bundled with Maven but you can also configure Hudson to use an external installation of Maven (Manage Hudson > Configure System > Maven 3 section).  When you add a Maven build step to your Job, you have the option to use the bundled Maven or your external installation.  However, the default is the bundled Maven. 

This can cause problems if you need additional classes required by your Maven plugins that are not automatically downloaded, e.g. FTP support for the Deploy Plugin or Wagon Plugin (http://mojo.codehaus.org/wagon-maven-plugin/).  That is, if you have dropped the required JAR into the "lib" sub-folder of your external Maven installation but the Job is configured to use the bundled Maven, you will continue to get "Cannot find wagon which supports the requested protocol: ftp" (in this case).