What is ETL and CMS?

You’re a programmer with a task to retrieve information from some source, manipulate and message it, and to deploy it somewhere.

Like all things in programming, there is an acronym for that: “ETL”.

ETL stands for Extract, Transform, and Load. The Wikipedia page is pretty thorough in its summary of the topic and reviews many of the typical functions an ETL process needs to take to accomplish its task.

The problem is ETL doesn’t roll off the tongue so easy. The acronym provides a weak set of metaphors for programmers to map familiar concepts to.

Rafe Colburn provides a great mental model to apply when developing ETL scripts and applications. It’s one I follow, but have lacked the words to describe. Go read his post.

Here’s a thought to challenge you if you are a CMS developer, now that you have read the above – are whatever forms you build to enable people to contribute and manage content in a CMS a kind of ETL process? Does the Wikipedia description for “Extract, Transform, and Load” contain functions there that you would expect a CMS to encompass?

And speaking of CMS, Gadgetopia has a terrific article on what a CMS system is. It is difficult to be clarifying in a world where hype and acronyms get thrown about so much (like this very post!) – but the Gadgetopia piece certainly is. It helps outline the functionality you should expect from a CMS implementation.

Some light reading (and research) on non-functional requirements in Agile/Scrum

Agile Modeling: Introduction to User Stories

Mike Cohn’s Blog: Non-functional Requirements as User Stories

aqris: Representing non-functional requirements with user stories

wikiwiki: Non Functional Requirements

Agile Coaching: Non-Functional Requirements: are user stories useful?

Artima: Johan Peeters: Dreams and Nightmares

Representing non-functional requirements is tricky. There are two kinds of non-functional requirement as mentioned in the aquis piece: independent, and distributed. When faced with a distributed non-functional requirement, adding it to your ‘definition of done‘ is called for. As you can tell by the links, there seems to be some difference in opinion in handling ‘independent’ non-functional requirements.

If you have pointers, please share in my comments. And thanks for the input in advance.

A solution to software maintenance from long ago?

Communications of the ACM: You Don’t Know Jack About Software Maintenance:

Software maintenance is not like hardware maintenance, which is the return of the item to its original state. Software maintenance involves moving an item away from its original state. It encompasses all activities associated with the process of changing software. That includes everything associated with “bug fixes,” functional and performance enhancements, providing backward compatibility, updating its algorithm, covering up hardware errors, creating user-interface access methods, and other cosmetic changes.

In software, adding a six-lane automobile expressway to a railroad bridge is considered maintenance–and it would be particularly valuable if you could do it without stopping the train traffic.

Related: Slashdot thread

Research: Software development roles and responsibilities

Trying to answer the elusive questions of:

What is a Software engineer?

What is a Lead Programmer?

What is a Tech Lead?

What is a Principal Engineer?

What is a Software Architect?

What is a Technical Project Manger?

What is a Scrum Master?

Links:

Wikipedia: Lead programmer

Wikipedia: Software engineer

Wikipedia: Software architect

IBM developerWorks: Characteristics of a software architect

Magpie Brain: A Tech Lead Manifesto

vanderbilt.edu: Project Roles and Responsibilities (Word .doc!)

it’s a delivery thing: Agile Project Roles and Responsibilities

Stack Overflow: Does a software architect have a role in agile, esp. Scrum?

Wikipedia: Scrum Roles

InfoQ: Mapping Traditional Software Development Roles to Scrum

Code Better:Classic Technical Lead Blunder

Atlassian: Tech Leads Talk

Two comparisons of different programming languages worth reading

Normally these kinds of pieces are worthless, but these two recently stood out to me:

Dennis M. Ritchie: Five Little Languages and How They Grew: Talk at HOPL* March 19, 2002

Michael Tsai: Perl vs. Python vs. Ruby – distinguished for the thoughtful replies in the discussion thread.

Clay Shirky lays out the issues confronting the future of news journalism

Read the whole thing. Nieman Journalism Lab: Clay Shirky at the Shorenstein Center on the Press, Politics and Public Policy:

…in the nightmare scenario that I’ve kind of been spinning at for the last couple years has been: Every town in this country of 500,000 or less just sinks into casual, endemic, civic corruption — that without somebody going down to the city council again today, just in case, that those places will simply revert to self-dealing. Not of epic, catastrophic sorts, but the sort that just takes five percent off the top. Newspapers have been our principal bulwark for that, and as they’re shrinking, that I think is where the threat is.

…So we don’t need another different kind of institution that does 85 percent of accountability journalism. We need a class of institutions or models, whether they’re endowments or crowdsourced or what have you — we need a model that produces five percent of accountability journalism. And we need to get that right 17 times in a row. That’s the issue before us. There will not be anything that replaces newspapers, because if you could write the list of stuff you needed and organizational characteristics and it looked like newspapers, newspapers would be able to fill that role, right?

It is really a shift from one class of institutions to the ecosystem as a whole where I think we have to situate the need of our society for accountability. I also want to distance myself — and I’ll end shortly. But I want to distance myself, with that observation I also want to distance myself from the utopians in my tribe, the web tribe, and even to some degree the optimists.

I think a bad thing is going to happen, right? And it’s amazing to me how much, in a conversation conducted by adults, the possibility that maybe things are just going to get a lot worse for a while does not seem to be something people are taking seriously. But I think this falling into relative corruption of moderate-sized cities and towns — I think that’s baked into the current environment. I don’t think there’s any way we can get out of that kind of thing. So I think we are headed into a long trough of decline in accountability journalism, because the old models are breaking faster than the new models can be put into place.

Again read the whole thing.

People tend to pick apart Shirky’s writings to find what supports their arguments. Which, I partially just did in fact, so don’t do that – absorb the nuance because the opportunities and problems at hand are far more complicated than the either naysayers or utopians would lead us believe.