Educational video on ‘Quants’ and their role in the financial crisis

YouTube: “Quants: The Alchemists of Wall Street (Marije Meerman, VPRO Backlight 2010)”

Related: “The Modelers’ Hippocratic Oath”: I will never sacrifice reality for elegance without explaining why I have done so….I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension.

Programing Links March 7th, 2009

Mike Taylor on Kernighan and Plauger’s “The Elements of Programming Style” (gotta buy and read this)

Eli Bendersky: “The server-side Javascript meme”

PragPub–March 2010: Jason Huggins: “JavaScript: It’s Not Just for Browsers Any More”

“Algorithms are Thoughts, Chainsaws are Tools”, reviewing a short film by Stephen Ramsey on “Live Coding”. Read it and watch it.

Alex Clemesha: “Tools of the Modern Python Hacker: Virtualenv, Fabric and Pip”

Deseret Tech: “NoSQL v. SQL is the worst holy war ever.”

Dennis Forbes: “Getting Real about NoSQL and the SQL-Isn’t-Scalable Lie”

Jesse Legg: “Emacs for Python Programmers: Basics”

the official hudson weblog: “Hudson at PyCon”

The website of Lei Chen: “Setup Perfect Python Environment In Emacs”

Extra Cheese: “Python vs. Ruby: A Battle to The Death”

Martin Fowler: “ConversationalStories”

Code Monkeyism: “What Developers Need to Know About Agile”

Coding Horror: “Cultivate Teams, Not Ideas”

Geoff Sowrey: “What makes a Senior Developer?”

Related:

“The Modelers’ Hippocratic Oath”

Practical Common Lisp

impromptu

10 years of weblogging

I’ve been trying to write a riff on Garret’s 10 year anniversary piece on blogging for a while now. But every time I start, it ends up way too long. So just read his piece and come back.

Okay, did that? Because as is old fashioned blogger custom I expect you to derive context for my following thoughts from my links (did you read Garret’s piece yet?) and from what I usually talk about here.

The march towards a plethora of walled-garden-social-networks has been a drag. And maybe it will be standards that will provide us a way out of the counter-personal-ownership of data mess we’re in right now. I’m hopeful. And I hope to do some hacking along the way to try and put together some duck tape of my own.

But the important thing is here we are.

Flashback to 1999. Conservatives were accusing Clinton of ‘wagging the dog’.

We were about to intervene in the Kosovo conflict. I felt our intervention in the Kosovo crisis was misguided for different reasons than those on the RIght. I felt that bombs couldn’t be the answer.

Yes, I was (and am counting who you talk to) a peace loving hippie.

I wanted to share my view, but I realized my voice held little weight, so I collected stories that supported my opinion and added them to a headline feed.

I ran that feed of headlines into My.Netscape and My.Userland so that people who might be interested could follow.

The feed reached people around the world even though I believe there were only a few hundred subscribers. People from Russia and Kosovo sent me emails to comment.

Let me repeat that again “People from Russia and Kosovo sent me emails to comment”.

I had made some kind of connection, with people from different countries, talking about war.

Me.

All it required was a text editor, searching for interesting stories that reflected my view, and manually writing out the RSS XML and storing it on a Web host. I registered the feed with My.Netscape and My.Userland and away it went.

Today, any of us can open an account at WordPress.com or TypePad and do that and so much more. Everything we post to Facebook, Twitter, our blogs, our forums generates RSS and Atom. These common communication formats helped lay down what is becoming the foundation of the real-time web. Where any of us have the potential to reach anyone else, anywhere.

This very post, when it goes live, will appear in Twitter, and Facebook, and even more amazingly, Google and Yahoo! in the order hours if not minutes.

What Tim O’Reilly had called the “Architecture of Participation” and Dave Winer called the “Read-Write Web”, way back when, continues to evolve and grow.

There is still much to do for it to reach its full promise. It has never lived up to its potential to enable those who need to be heard to be heard. Human nature is human nature after all and we tend to tune into voices that resemble our own. But the potential still is there to make a connection across our own biases and our own filters. The potential and capability.

For all the negatives that still abound, all the opportunities left to explore, the challenges left to solve, blogging has helped me connect with Garret, and many other terrific online travelers across the world and here in my home town. People who I consider teachers. Thought provokers. Inspiration. Friends.

You know who you are.

Thank you to all the folks who laid down this architecture for all of us to participate, twist, turn, innovate on, and completely take for granted. And thank you to all those who have made that connection with me and enlarged my heart, my mind and world.

“A Conversation with Martin Fowler”

Bill Venners interviewed Martin Fowler back in 2002 that resulted in six part series filled with engineering wisdom to absorb.

Among the many exchanges was the following on flexibility that reinforces some principals I try (not always successfully) to put in practice:

Bill Venners: In Refactoring you write, “Before I used refactoring I always looked for flexible solutions. Because design changes were expensive, I would look to build a design that would stand up to changes I could forsee. The problem with building a flexible design is that flexibility costs.” What is the cost and what is the alternative?

Martin Fowler: The cost of flexibility is complexity. Every time you put extra stuff into your code to make it more flexible, you are usually adding more complexity. If your guess about the flexibility needs of your software is correct, then you are ahead of the game. You’ve gained. But if you get it wrong, you’ve only added complexity that makes it more difficult to change your software. You’re obviously not getting the payback.

It’s not hard to guess wrong about flexibility needs. You can guess wrong if requirements change. What you think is a requirement for flexibility now may go away or change in the future. You can also guess wrong if you put extra code into the program to improve flexibility but you don’t get it quite right. You get more complexity without getting the flexibility you were after.

The alternative is to use the XP approach and not put the flexibility in at all. XP says, since most of the time we get it wrong, just don’t put the flexibility in there. Now if you can’t evolve your design safely, then that is a foolish route to take. But if you can evolve your design safely, it becomes quite a nice approach. In fact it becomes a self-reinforcing approach. If you strive to keep your design as simple as possible by avoiding speculative flexibility, then it’s easier to change the code because you have less complication to deal with. The code is easier to understand and easier to change. As a result, you can make changes much more quickly.

Read the whole series:

Refactoring with Martin Fowler

Design Principles and Code Ownership

Evolutionary Design

Flexibility and Complexity

Test-Driven Development

Tuning Performance and Process

A great description of the bug elimination process

John Graham: “A bad workman blames his tools”:

1. Find the smallest possible test case that tickles the bug. The aim is to find the smallest and fastest way to reproduce the bug reliably. With heisenbugs this can be hard, but even a fast way to reproduce it some percentage of the time is valuable.

2. Automate that test case. It’s best if the test case can be automated so that it can be run again and again. This also means that the test case can become part of your program’s test suite once the bug is eliminated. This’ll stop it coming back.

3. Debug until you find the root cause. The root cause is vital. Unless you fully understand why the bug occurred you can’t be sure that you’ve actually fixed it. It’s very easy to get fooled with heisenbugs into thinking that you’ve eliminated them, when all you’ve done is covered them up.

4. Fix it and verify using #2.

Read the entire piece.

Alfresco Links for February 7th, 2010

Alfresco Webcast: What’s New in 3.2?

Alfresco Webcast: WCM Roadmap Webinar – Versions 3.3 and 4.0

Key to Content: Alfresco WCM Roadmap (notes from the presentation)

Philip Kedy at CapTech: Alfresco WCM or DM: What is the best choice for your enterprise portal?

Jeff Potts: Understanding the differences between Alfresco’s repository implementations

Recent related event: Thanks to the folks at zivtech for hosting a great Alfresco lunch & learn event on January 27th.

The case for killing ‘WCM’ (Web Content Management)?

First, a disclaimer. The title refers to the term ‘WCM’, not the functionality implied by it.

WCM (Web Content Management) as defined by Wikipedia is a system that “allows non-technical users to make changes to a website with little training. A WCMS typically requires an experienced coder to set up and add features, but is primarily a Web-site maintenance tool for non-technical administrators.”

Sounds simple, but the definition is crazy expansive.

It’s so generic it enables a wide field of choices to claim they satisfy the need. Check out this list: Bricolage, Alfresco, Interwoven, ez Publish, Texpatten, MovableType, WordPress, Drupal, Jadu, Vignette, Day, Nuxeo, Radiant, typo, Fatwire, Clickability, Plone, SDL Tridion, ektron, it goes on and on. And the costs! From free to millions of dollars!

Couldn’t you consider page creation/site management tools like Dreamweaver in that definition? Sure you could. Many who think they want a CMS, really want one of these or a combination of one of these with a CMS. If you look at Google there are 2,370,000 hits for the combination of Dreamweaver and CMS.

WCM is thought of as a subset of ECM (Enterprise Content Management) concepts. ECM is defined by Wikipedia as “the technologies, strategies, methods and tools used to capture, manage, store, preserve, and deliver content and documents related to an organization and its processes. ECM tools allow the management of an enterprise level organization’s information.”

Among the list above are a few ECMs that have WCM functionality. Commonly mentioned are Alfresco, Interwoven, Vignette.

ECM is then considered a subset of CMS (Content Management System) concepts. A CMS is defined by Wikipedia as “a collection of procedures used to manage work flow in a collaborative environment.”

Referring to the above list, many can be called out as CMSes. In fact, all of them consider themselves such.

And then there are frameworks like Ruby on Rails, Grails, and Django. They just beg you to build your own.

With an alphabet soup like the above, no wonder so many get confused. WCM, in particular, is overloaded. So much so that there are some folks in the industry arguing to eliminate the acronym altogether!

Jon Marks says in “WCM is for Losers”:

I can already see the news headlines: LONDON, 2009 – SHOCK HORROR! WCM Geek Demands Death of term WCM. But it’s true. I’m of the camp that wished the term WCM would cease to exist.

Jon Marks concludes by saying:

But sadly, my prediction it isn’t going to happen. I’m just going to have to keep thinking of a WCMS as a tightly coupled hybrid of a content management system and a delivery framework. On the plus side, I’ll continue to make money out of poor customers that think a “WCM migration/replacement” doesn’t involve a complete site rewrite as they’re throwing the delivery baby out with the content bath water. Losers.

Deep within the comments on Jon Mark’s post, NPR’s Daniel Jacobson added:

In my posts about COPE, I tried to make a distinction between tools that capture content in a presentation-agnostic way and those that capture them for one (or more) specific presentation. I call the latter WPT (web-publishing tool), although Peter Monk’s Presentation Management System is in some ways a better term in that it is broader, covering systems that don’t just apply to the web.

For me, however, the future of the content management systems (CMS, or whatever acronym you want to give it) is in their ability to capture the content in a clean, presentation-independent way. Then, as Pie states, the content should be retrievable through a series of API’s, enabling the content to be distributed to any other platform. If available through an API in a truly portable, presentation-agnostic way, the system can then service any presentation layer.

Alfresco’s consulting lead for North America, Peter Monks, shares on his blog how difficult this is and looks for a new terminology in “The Case for Killing ‘WCM'”:

To start undoing the 15 years of mind share that the term “WCM” has enjoyed, it’s time to start thinking about new terminology that better describes these two functional categories. For several years I’ve been throwing around the terms “Content Production System” (CPS) and “Presentation Management System” (PMS), and in their COPE strategy NPR uses the terms “Content Management System” (CMS) and “Web Publishing Tool” (WPT).

Daniel Jacobson and Peter Monks are onto something. Jacobson wrote a piece for Programmable Web (“COPE: Create Once, Publish Everywhere”) I’ve linked to previously. The section “Build CMS, not WPT” makes some important distinctions:

COPE is the key difference between content management systems and web publishing tools, although these terms are often used interchangeably in our industry. The goal of any CMS should be to gather enough information to present the content on any platform, in any presentation, at any time. WPT’s capture content with the primary purpose of publishing web pages. As a result, they tend to manage the content in ways focused on delivering it to the web. Plug-ins are often available for distribution to other platforms, but applying tools on top of the native functions to manipulate the content for alternate destinations makes the system inherently unscalable. That is, for each new platform, WPT’s will need a new plug-in to tailor the presentation markup to that platform. CMS’s, on the other hand, store the content cleanly, enabling the presentation layers to worry about how to display the content not on how to transform the markup embedded within it.

True CMS’s are really just content capturing tools that are completely agnostic as to how or where the content will be viewed, whether it is a web page, mobile app, TV or radio display, etc. Additionally, platforms that don’t yet exist are able to be served by a true CMS in ways that WPT’s may not be able to (even with plug-ins).

COPE is an ecosystem and strategy. It is not an uber-CMS. Many of the vendors above claim their systems can provide you this almost-mythical beast. Indeed, many of them can, but it calls back to the point Jon Marks was making and a common mistake many trip into.

De-couple, break it down into separate systems systems (Presentation, API/Mashup/Data, and CMS) as Daniel Jacobson and Peter Monks suggest, and you gain much in flexibility. The trade off is flexibility’s evil twin – complexity. You have to be able to accept some complexity for the gains you draw in flexibility.

Lets take a look at a diagram Jacobson provided on NPR’s COPE:

Look at where the ‘CMS’ is in what NPR calls its Content Management Pipeline (what I call an ecosystem). Look at where the Presentation Layer is. Notice what feeds it. The API Layer capable of delivering content from multiple sources, including what the CMS feeds into, the Data Management Layer.

Take another look. This is an inkblot test.

Some see a diagram like this and see the whole thing as CMS or WCM. When these say ‘Content Management System’ or ‘Web Content Management System’ they are thinking of a singular application that performs all of the duties of all of the layers detailed in the diagram. In NPR’s case, this diagram shows the ‘Content Management System’ is part of a tier on the backend – where content is consumed, stored, maintained for reuse. It’s goals are as Jacobson points out. The CMS’s role constrained to a set of responsibilities and in order to do them it must integrate into a larger set of cooperating systems.

You might not need a system as comprehensive as NPR’s but you won’t know until you answer a few questions: “Where is it you want your business to go?”, “What is the Content Strategy?”, and importantly, “Show me how you do things now.” and “Let’s figure out a better way of getting this done.”. Starting from the simplest thing that can possibly work and allowing for evolution towards your end goal is always the way. For example, you could start with a combined Data Layer, API Layer, and Filtering Layer (what I call a “3 Box Content Management Ecosystem” below), and then decompose that into separate systems down the line. If you do have answers to these questions, and they resemble what NPR’s are, Jacobson has provided a great high level view of what this looks like. He deserves thanks for sharing it.

Related:

A friend at work passed along a great link, Blend Interactive’s “Thoughts on Content Management & Information Architecture”. I’ve linked to Gadgetopia.com, the official blog of Blend Interactive before, but this index is, as he suggests, “quite possibly the best single source of CMS-related questions, insights, etc. that I’ve ever found.” Bookmark it. Their “What Makes a Content Management System”? piece provides you with the best checklist of functionality to consider when looking at CMSes.

And lastly:

Sometimes I think I want to publish a series that describes the various layouts that define CMS systems in really, really simple terms because of the confusion. Here’s a first pass:

The ‘2 Box Content Management System’ – Presentation and content maintenance functionality using same software, with shared storage. (WordPress, Drupal)

The ‘3 Box Content Management Ecosystem’ – Presentation running its own software, content maintenance running its own software, with shared storage. (MovableType, WordPress and Drupal, ez Publish, Bricolage, Alfresco in some implementations)

The ‘4 Box Content Management Ecosystem’ – Presentation running its own software, content maintenance running its own software, a data tier for presentation, a data tier for content storage. (Alfresco in some implementations)

The ‘5 Box Content Management Ecosystem’ – Presentation running its own software, API/Mashup running its own software, data tier for presentation, content maintenance running its own software, a data tier for content storage. (Alfresco in some implementations, NPR’s COPE content pipeline).

And so on. Someday I might get around to it. The terminology soup is so oppressive and obscuring.