Getting HTML text with timeouts rediculously hard in Java

Posted on by

Creating a web crawler in Java is easy – if you don’t need to set timeouts shorter then the defaults. Here is a JavaWorld article and a Sun Developer Connection article on doing just that. But the moment you do require control over timeouts… ouch does it get confusing. Note that this post covers version earlier then 1.4.x.

Here goes some of the major issues I’ve found: 1. The HTTP enabled classes do not expose a timeout property setting method. 2. The lower level socket classes do expose that method, but require a crap load more programming to utilize. This great JavaWorld article covers that approach. Run it on this site though and watch what happens. 3. Trying to hack your way to a timeout with the HTTP classes and threads exposes a registered bug – the HTTP operation may not properly close.

Anyway, after much searching, jGuru pointed me towards Jakarta’s HTTP Client. I recall reading about it over at rebelutionary. But guess what? After who knows how long digging I discovered the timeout property was not exposed in the last release build. Ok I figure, I’ll go grab the latest nightly build and cross my fingers. Now some undocumented depencies are exposed. You will need to download and install the Jakarta Logging Component and Sun’s Java Secure Socket Extention to get it to work.

I could have written the low level Socket code…. but there is a strong part of me… the lazy programmer in me… that believes in never re-writing the wheel. I knew there had to be a set of packages that would allow me do this with as little as possible coding. By finding those packages, especially from a reliable source such as the Apache Jakarta project, I can have a higher degree of confidence in what I’m putting together. And oh yes… just write about 50 lines of code 🙂

Anyway… anyone else with these findings or did I just take a walk I didn’t need to?

This is the true joy in life

Posted on by

George Bernard Shaw

This is the true joy in life – that being used for a purpose recognized by yourself as a mighty one. That being a force of nature, instead of a feverish, selfish little clod of ailments and grievances complaining that the world will not devote itself to making you happy. I am of the opinion that my life belongs to the whole community and as long as I live it is my privilege to do for it whatever I can. I want to be thoroughly used up when I die. For the harder I work, the more I live. I rejoice in life for it’s own sake. Life is no brief candle to me. It is a sort of splendid torch I’ve got to hold up for the moment and I want to make it burn as brightly as possible before handing it to future generations.

That quote is scribbled on a piece of paper in my wallet. Been there for a long time. I havn’t found my purpose yet. But man do I feel blessed.

In that spirit I want to send some shout outs.

Please say a prayer for Garret and his family.

Send supportive e-mails to Rogi and Dave on their struggle with smokes. It’s way harder then most people realize.

Go to MP3.com and listen to a great band, a rocking song, and support one of my friends, Mark. The song is about to crack MP3.com’s top 50!

Send a supportive word to Meryl who survived Blogathon!. You can still donate to her very worthy cause.

Very importantly – The Sleeping Angels fund has placed it’s first headstone for a child who didn’t have one since February 2001! I am so proud of my brother and his family. How they are channeling their son’s passing is inspirational.

The world can be a horrible place – but this time a 7 year old inspires

Posted on by

Time’s deserving person of the week is 7 year old Erica Pratt. A girl that has inspired many in the city and out here in the blogging world. Her strength will be remembered for a long time by many.

There’s much more to this story. The Inquirer reports that the Suspected kidnappers, family have web of ties.

As Acel Moore says

Unlike recent kidnapping cases in Utah and California, this crime suits its surroundings.

She had to learn at a very young age how to watch for herself when away from her grandmother, and thank God for her grandmother’s love and care.

No, this posting is not a return to my past blogging velocity. Still lots more moving to do, and I like my time away from the web, with my friends and family.