Looks like a tutorial I want to take: “Flex, Spring and BlazeDS: the full stack! (Part 1)”
Tag Archives: programming
Using Disk to Scale
One of the largest concerns when developing an infrastructure for a site as large as Comcast.net is determining smart ways to scale. By smart, I mean requiring the least amount of effort to launch new channels or services. Each new channel or page can draw thousands, if not millions of page views. You need to plan for it.
When growing Cofax at Knight Ridder, we hit a nasty bump in the road after adding our 17th newspaper to the system. Performance wasn’t what it used to be and there were times when services were unresponsive.
A project was started to resolve the issue, to look for ‘the smoking gun’. The thought being that the database, being as well designed as it was, could not be of issue, even with our classic symptom being rapidly growing numbers of db connections right before a crash. So we concentrated on optimizing the application stack.
I disagreed and waged a number of arguments that it was our database that needed attention. We first needed to tune queries and indexes, and be willing to, if required, pre-calculate data upon writes and avoid joins by developing a set of denormalized tables. It was a hard pill for me to swallow since I was the original database designer. Turned out it was harder for everyone else! Consultants were called in. They declared the db design to be just right – that the problem must have been the application.
After two months of the team pushing numerous releases thought to resolve the issue, to no avail, we came back to my original arguments. The terrific thing was that restructuring the database was a no pain affair – we had a terrific service layer between the main web tier and the db that hid its schema. We were able to deliver a release of the database that did not require any code changes on the web tier.
There is no silver bullet here, for smaller sites you are adding a great degree of complexity taking this route and it is, most likely, not advisable. However, if you have a large site that is thrashing – dealing with the demands of growth – take a hard look.
Related – and supporting of this:
High Scalability: “How I Learned to Stop Worrying and Love Using a Lot of Disk Space to Scale”.
High Scalability: Scaling Secret #2: Denormalizing Your Way to Speed and Profit
Dare Obasanjo: When Not to Normalize your SQL Database
Java Application Servers Without J2EE
infoq: SpringSource Launches New Application Server without Java EE
SpringSource Team Blog: Introducing the SpringSource Application Platform
SpringSource: http://www.springsource.com/web/guest/products/suite/applicationplatform”>SpringSource Application Platform
Not directly related at all, but has everything to do with it:
The Daily Profeth: Google App Engine & eclipse (PyDev) – a nice getting started guide.
Great NYTimes article reveals a little about their Web production process
.It was both educational and fun to read the NYTimes interview with Khoi Vinh, their Design Director.
Got my Mac Book Pro at work
Many of us at work are migrating to OS-X. It’s logical since our deployment environment is a Unix variant, Solaris, and most of us on Windows run Cygwin to create a developer environment that resembles a Unix-like environment.
Now I’m not a stranger to OS-X. I’ve been convincing my family to switch for the past four years and now they mostly run iBooks and Mac Books, decreasing the time I used to spend helping fix problems. Fact of the matter is, if you are using a PC mostly to send email, surf the web, manage photos and video, it is a great all round choice.
The irony is that within minutes of getting my laptop I froze it! Turns out it isn’t all that smart to run Parallels, out of the box, the way I did, and run, oh, 8 or so programs simultaneously outside of it!
Anyways, in less than an hour I had my favorite web browser, Firefox, my organizer, Wikidpad (which required me to run it from the Python source – but it worked!), my encryption software TrueCrypt (Edit: TrueCrypt development was discontinued, see the link for background and alteratives), my IDE of choice Eclipse, my favorite OS-X free text editor, TextWrangler, all up and running. With Maven, SVN, Java and Python pre-installed made it easy to checkout my current work and get a build going. I won’t be needing Parallels all that much since so much of the work I do can be done in OS-X, but it will be convenient to be able to test websites in different browsers, on two of the three primary desktop OSes, with little effort.
I Upgraded to Movable Type 4.1 Open Source
I’m happy I finally got around to upgrading my personal blog to the latest and greatest Movable Type. It’s clear that open sourcing the software has been good for SixApart and that MT can again be considered a viable alternative to other blogging platforms like WordPress.
People at work like to ask me what blogging platform ‘is the best’. Honestly, after working with so many over the years, I have trouble identifying that. Feature for feature, you can make one do what the other does.
Someday I’ll put together a matrix that highlights the real differentiators as far as I am concerned, but I do have a shortlist I can share if I was doing a project as a consultant: Drupal, Movable Type, WordPress, and rolling something new with Django.
MT has a new beta release coming out with a few features I am looking forward to.
Interesting Programming Reads
Blog at trepca.si: Java, Python and defaults – Sure is true enough.
Code To Joy: Open-Source group announces jJavaM – It was an April fools, but a good one for the sarcasm.
Python-by-example – Will come in handy.
Better Programming With Java EE: A Conversation With Java Champion Adam Bien – Dispels some myths.
An Army of Solipsists: Blog Archive: Using Spring MVC Controllers in Grails – Might come in handy if I ever get around to experimenting with Grails.
Anil Dash: Atom Wins: The Unified Cloud Database API: “I want every program that thinks of itself today as a “blogging client” to reimagine their market as being a front-end to a database in the cloud. I want all the apps built on smart database abstractions to think about this new unified cloud API as an option they must support. And most of all, I want geeks to make something cool with this that we couldn’t do before.”
Hosting Java Web Applications: Why Is It Still So Hard? | Javalobby
“crippled by their own process”
Coding Horor: “Is Eeyore Designing Your Software?”:
Here’s my honest question: does open source software need all that process to be successful? Isn’t the radical lack of process baggage in open source software development not a weakness, but in fact an evolutionary advantage? What open source software lacks in formal process it makes up ten times over in ubiquity and community. In other words, if the Elbonians feel so strongly about localization, they can take that effort on themselves. Meanwhile, the developers have more time to implement features that delight the largest base of customers, instead of plowing through mountains of process for every miniscule five line code change.
Are large commercial software companies crippled by their own process?
I’d say that in large corporations, I’ve seen many internal projects beat down by the same.
The new portal architecture at CIM doesn’t suffer from this, but the old one certainly did. We’ve come a long way.
Design Patterns Aren’t (That) Evil
I agree with much of Jeff Atwood’s writing when it comes to programming and development. I’d say on any given subject 90% to damn near 100% (congrats to him on his new adventure). But I think his post on design patterns, unfortunately, falls into a line of argument that I disagree with strongly – that *identifying* design patterns leads to complexity. Complexity because some engineers leverage them right from the get go without thinking about simplicity first. He even flags Head First Design Patterns as a potential complexity creator.
I understand the concern. Every once and a while you get into an argument with someone who is telling you your code stinks because it doesn’t employ pattern “so-and-so” and every once and a while you come across some needlessly complicated code because the developer thought pattern “xyz” was the appropriate solution and implemented it without thinking a few minutes more about the problem and putting together something far simpler.
Replace the word “pattern” in the above sentence with “technology” or “API” or “archetecture”.
Give it a try. It leads to the same place. And I’d say the problem doesn’t start with patterns (or technologies, or APIs, or whatever). It starts with the developer.
Does that developer start from a KISS viewpoint, or one enamored by buzzwords?
So Jake Says: Music and Design Patterns:
Chord progressions are design patterns. They give a common framework musicians can use to communicate. However, the implementation is left to the musician. You can play classically or bluesy. You can shred the progression. You can take the most “outside” ideas of modern atonal theory and apply them to the song. There are elegant implementations, there are common implementations, there are “outside” implementations, and there are bad implementations.
Chord changes aren’t represented in the core notation/language of the music, but you can use musical notation to spell out changes. You can also use shorthand languages to design the music. The sentence “12-bar in Bb, 2-5-1 turnaround, on my lead” gives away none of the implementation details (voicings/melodies, etc.,) yet the song is written in a breath.
Design patterns act the exact same way for programmers. They are, at heart, a common framework by which programmers can discuss a design. They can spend less time focusing on minutiae and more time discussing design and code.
Even if you are using Python or Perl and you don’t have to explicitly define an Iterator to loop through arbitrary collections, you could easily point to a “for x in y” statement and say “iterate through y” to describe part of an algorithm. You will be correct, and a coder from any paradigm won’t have to give it two thoughts.
Design patterns always exist, but are sometimes invisible.
REST and Unix Pipes
http://www.linfo.org/pipe.html
http://www.xml.com/lpt/a/1644
http://www.sixapart.com/pronet/articles/shuffling_atom_.html
http://radar.oreilly.com/archives/2007/02/pipes-and-filters-for-the-inte.html