On The Mark Technology

So I've got an application that requires some reporting. There are several different reports, and each report can be run with any number of input filters. Some of those filters are required, some are not. So the requirement is to select a report, and based on that selection, display the available filters - some of those filters even require some dynamic selections.

So here's what I did.

I built a reports table, and a report_filters table - a report may have 1 to many filters associated with it. When the user hits the reports page they see a drop down box of the reports available. Also on this page is a couple of hidden regions; "Filters" and "Results".

The drop down box has an "onChange" action that passes the selected report ID to a server procedure via Ajax that provides the HTML required for all the filters available for that report in the "Filters" section of the page. This is where the Native Dynamic Sql (NDS) comes in.

In my table of report filters I have a column (filter_input_procedure) where I put in an anonymus PL/SQL block. This block gets executed in a loop of all filters for for selected report with an EXECUTE IMMEDIATE command. When this fires, the pl/sql block interacts with the database and outputs the required HTML for the input filter.

procedure build_rpt_filters(p_rpt_id in number) is
cursor available_filters is
	   select * 
           from report_filters_table 
           where rpt_id=p_rpt_id 
           order by sort_seq;
begin
for xx in available_filters loop
	execute immediate xx.filter_input_procedure;
end loop;
end build_rpt_filters;

To build the start date input procedure show in "Report With Two Filters", here are the contents of the "filter_input_procedure" field.

declare
begin
htp.p('Provide the Start Date (mm/dd/yy): ');
htp.p('<input type="text" name="p_start_date" size="10" value="">');
end;

The next thing I had to do was accomodate the user entering a value for a filter, but show them some information that allows them to confirm they have entered the right value.

In the "Report That must do a lookup", the input filter is displaed as before - this time with an input box and a "Submit" button. However, the "submit" button has it's own AJAX call that executes another procedure on the server fires to get another batch of HTML, which is displayed in the "Results" section of the page.

EXAMPLE:

One of the last reasons for doing report filters in this way was a support requirement. The application I wrote this for is supported remotely and often without access to the server. So as much configuration as possible is defined in tables and a user interface built for it. Using Dynamic Sql allows me to control some coding aspects on the fly, and using AJAX allows me to craft a user interface that quick and intuitive.

Recently, this site has been hit with "Comment Spam". This is a phenomenon where commercial messages are pasted into comments on blogs. These comments are not related to the topic in the blog post, are typically for pharmaceutical products, are are done by programs, not people.

One approach to this spam is to turn off comments altogether. This is the nuculear option that is pretty unattractive. This site doesn't get to many comments, but if somebody has a question on a technical topic I post, then email becomes the method, and only the parties in question benefit from the results of the discussion.

Another approach is to moderate the comments. Hide the comments until a human (me) can review the comment and either approve or discard the comment. This is a hassle, especially since the spammers are typically code based and can throttle up the volume pretty easily. I was getting 30-40 per day.

Another approach is to put in a "CAPTCHA" ("completely automated public Turing test to tell computers and humans apart") test that puts an image on the screen that can't be read by a computer and ask the commenter to key in the value in the image. Accessibility is the issue here as those using screen readers can't participate in the commenting.

To prevent the message from providing value (most search engines like outbound links and boost site rankings when people link to the site) you could remove all the HTML or add the "ref=nofollow" attribute to anchor tags (this tells participating search engines to not index the link). But once again, this can limit functionality to real users.

Another approach is to change the comment form in a subtle way that prevents the spammers program from working properly. But much like security techniques, the smart spammer will eventually figure this out.

I've implemented a combination of the above methods, and so far I've been able to thwart their attempts. I think their programs are still hitting the site as my page views are abnormally high, but the spam actually be posted to the server has been eliminated.

What I'd really like to know though is 1) how did they find this site? 2) what makes them think that this site gets enough page views that spamming would have any success at reaching an audience (search engines or human eyeballs? 3) does it really work?

I wonder if there is some sort of blog "confessions of a blog spammer" that could provide those insights.

Ajax - also known as asynchronous JavaScript + XML, or even xmlHttpRequest is the newest flavor of the month with web developers. I have to admit that I was interested in using this to dynamically communicate with the server and craft the output of a page without doing a full page load.

I've documented several examples where people have explored the technology and written about it. Now I've finally done my own.

The trick for me is to make the code work in as many situations as possible, so it can be deployed with a minimum of thought. For my first implementation, I needed to build a drop down box with a list of profiles that are associated with a application selected in another drop down box. For example, the application "brand" has four profiles that are valid. When the user picks "brand" from the list of applicaitons, the profile list is updated to only show those four profiles.

Sounds simple enough.

What I ended up with is that a each deployment of the technology is deployed with 2 lines in a single function called by the "onChange" event of the application drop down list. Here is the select tag for the application list:

select name="selected_appl_name" size="1" onchange="updateProfileList(this[selectedIndex].text,document.appselect.profile_name_ms);"

Here is the profile select list (nothing really to see):

select name="profile_name_ms" size="1"

So when the user changes the application name, the "updateProfileList" function fires passing the value of the user's selection, and the drop down box that will be re-populated.

function updateProfileList(appl_name,lbToChange) {
    targetlb = lbToChange;
    loadXMLDoc("pkg.get_appl_profiles?p_appl_name=" + appl_name);
}

The first line assigns the targeted select box to be changed to a variable that is global and available to all procedures. The second line is where the interaction begins. The loadXMLDoc procedure accepts a URL that returns a result set of option names and values. Here is a sample of the output:

<?xml version="1.0" encoding="UTF-8" >
<response>
   <option>
      <optionvalue>admin</optionvalue>
      <optiontext>admin</optiontext>
   </option>
   <option>
      <optionvalue>americas</optionvalue>
      <optiontext>americas</optiontext>
   </option>
   <option>
      <optionvalue>international</optionvalue>
      <optiontext>international</optiontext>
   </option>
</response>

As a developer, all I have to do is write the two html tags, the function called on the onChange event, and the server side procedure to generate the proper XML.

Piece of cake.

The real work is being done by three functions. The first one, initiates the xmlHTTPRequest object. Basically, it checks to see which type of XmlHTTP object it can create (the browser makers implement it differently, so the code needs to figure out which is correct).

function getXMLHTTP(){
  var A=null;
  try{
    A=new ActiveXObject("Msxml2.XMLHTTP")
  }catch(e){
    try{
      A=new ActiveXObject("Microsoft.XMLHTTP")
    } catch(oc){
      A=null
    }
  }
  if(!A && typeof XMLHttpRequest != "undefined") {
    A=new XMLHttpRequest()
  }
  return A
}

The second function is the one that actually sends the request to the server. The tricky part about this functionality is that the object is Asynchronus, meaning that the code does not wait for the function to finish before continuing on. So the "onreadystatechange" property is an event handler which is triggered whenever the state of the request object changes - notice how it is assigned a function. This function is run each time the state of the request changes, which means it's not in your control.

function loadXMLDoc(url) {
  req=getXMLHTTP();
  if(req){
      req.onreadystatechange = processReqChange;
      req.open("GET", url, true);
      req.send(null);
  }
}

The last function is the "processReqChange" function that is being run. This is where all the real work is being done to build the new list box. The first if statement that checks the readyState and the status is really testing if the HTTP request is complete, and there is data to process. If the server does not respond, then nothing will happen. Otherwise, it parses the XML document, truncates the targeted list box of any previous entries, and then rebuilds it with the values in the XML package. If there are no entries, then a single entry in the drop down list is built "No Value", and a null value will be passed as the selected value.

var req;
var targetlb;
function processReqChange () {
  if ((req.readyState == 4) && (req.status == 200)) {
     response  = req.responseXML.documentElement;
     options = response.getElementsByTagName("optiontext");
     targetlb.length=0;
     if (options.length > 0) {	    
	for (i=0; i < options.length; i++) { 
		opttext   = response.getElementsByTagName("optiontext")[i].firstChild.data;
		optvalue    = response.getElementsByTagName("optionvalue")[i].firstChild.data;
		targetlb.options[i]  = new Option(opttext,optvalue );
	} 
     } else {
	targetlb.options[0]  = new Option("No Value","");
     }
  }
}

A couple things to note.

1. There are two variables that are defined that must be available to all four of the functions - put them after the "script" tag.
2. This code will only work when populating a select box.
3. The XML packaged must be formatted with the two elements - optiontext and optionvalue..

A couple of opportunities for improvement.

1. How about adding support for other types of form elements - e.g enter value in a text box and it populates another box, or boxes.
2. Support some type of indicator that would tell the script if a null value should be even available. If not, then should the application drop down be set back to what it was? Is there an error message?
3. The value provided for the null - "No Value" should be customizable.

I relied on these two articles heavily when building my implementation - the very good primer at XML.com "Very Dynamic Web Interfaces" and the great job Chris Justus did dissecting the Google Suggest page.I put together my first working implementation in about 3-4 hours, and the finishing touches to generalize it in about another 3-4 (including BDM -Big Dumb Mark-time chasing dumb Javascript errors where I should have known better).

UPDATE: - I've posted a few examples of this technique.

37 Signals is a web development, design and consulting firm based out of Chicago. They are a classic example of how a company sponsored blog ("Signals vs. Noise) can be used as a marketing vehicle. Jason Fried, who looks to be the primary writer on the blog writes smart and interesting posts about web stuff almost daily. The blog has quite a following and the comment section of each post always has some interesting discussion.

Recently, 37signals launched Basecamp, a hosted, online project management application. I'd bet that the tool was developed in house to help 37signals manage their own business. One day, somebody said "other people should use this, and they might even pay for it." So they likely invested some time and money, turned it out to the public and now have people paying anywhere from $12 to $99 per month to use it. A wonderful idea, and something I'd love to do also.

I signed up for a free Basecamp account that allows me to manage a single project. I have to admit, the application works really well. It takes a collaborative blog concept, adds a few widgets here and there and now they have a online application that people love. Congratulations to them.

But when I look at it from a developers perspective, the product is not really that deep. In some respects, my project "Reparté Collaborative" is looking at the same target market and has lots more conceptual functionality. I was focusing the application to web based development/support teams, using a blog with comments as the center, and adding the repository of other stuff around it - database info, URL, user names, passwords, document management, file upload/download etc would all be stored in the application. Adding to do lists and milestone tracking to the application would be pretty trivial. All of this would be contained within a central, available anywhere with a login, website. My concept is solid, and I believe there is a market out there who would even pay for it. It would not be that hard to develop.

Two things make finishing the build difficult: finding your customers who have money to pay, and time. 37Signals used their blog to develop a respected "brand" concept. The blog is also frequented by their target audience - web savy people who need an easy way to get organized. So when the product was ready, (I'm making an assumption here) they said "here, try this" and probably got lots of feedback - both in the determination of what functions should be included, an what someone might pay for it.

The other barrier is time for development. You see, I have this thing called a "day job", and I already have a "night job" too (daddy/husband). So other than day or night, when could I finish the build out?

A recent blog entry on SvN that would probably help me was titled Less is... More satisfying.

...I think we just cut down the feature list by about 2/3. Wow does that feel good. Faster to market, easier to design, more attention to detail on the 1/3 that remains...

Probably wise words, but then I'd have to do that marketing thing..

It's frustrating sometimes to see products coming to market that I could do using my skills. I guess I just need to quit my job and do all these software projects that rattle around in my head. Are there any Venture Capitalists out there with some money to burn (I don't need that much) on a smart web developer with lots of ideas that everybody else seems to get done first?

I try. I really do. But I don't think I'm very good with colors. I'm more of a coder at heart. I can make programs that sing, that are extensible, and are easy to maintain. But color design, working with "the wheel", mixing matching hues and saturations, understanding what works and what doesn't... I tend to rely upon others.

Here are two sites, recently redone and are so impressive with color, I have to call them out.

1. Sparkplug, by Firewheel Design simply rocks with powerful orange and black background and a funny greenish on the right navigation. I'm not sure how those work together... but they do.
2. SimpleBits is home to web design luminary Dan Cederholm and uses gradients of cool blue, green, and ... orange (again) to create a new bright look.

Does this mean a redesign is coming? No. I've got too much work that is already behind schedule. But, the RotoMojo site, which is still in early development, may see some new colors. It currently relies on design variations from Neil Turner - which are used on two other sites I visit regularly. So, while those designs were good to get started, it needs to get re-visited before it becomes "ready for primetime".

Firefox is my browser of choice. Ever since I discovered it's availability at about version .6, I've been hooked. Mostly due to tabbed browsing. Having a single window open with multiple tabs is truly, the best way to navigate the Interweb.

But second on my list of "Best Things About Firefox" is the ability to add to install "extensions." Extensions add to the functionality of Firefox in some of the most meaningful ways. Here is a list of my favorites:

• Web Developer - This extension adds a toolbar at the top of the window and allows you to truly examine how a website is constructed. From viewing the underlying CSS, to resizing your windw to emulate different size screens, this extension is a developer's dream and essential tool.
• AdBlock - I've recently just discovered the wonders of this extension. Built in to Firefox is the ability to block images from a site. I've used this ever since I found Firefox. But what ad block does is two fold.
  • First, it allows you to further specify individual directories that contain images you want blocked. This was especially helpful as a site I read daily has ads - the wildly annoying blinking animated ads - served up from their own server. I couldn't use Firefox's built in blocking as some of their navigation elements would have been blocked too. But their ads are served from an "ad" directory. AdBlock is now configured to not display those ads and my browsing is safe from those annoyances.
  • Second, Adblock also detects images it thinks might be ads, and gives you a transparent graphic attached to the ad which allows you to add it's definition to its blocking file. Usually, I edit the definition to specify a wildcard, so everything from that site, is blocked.
  
  I now browse in peace.
• IE View - This extension allows you to quickly display a site using through the eyes of the evil empire. It will immediately open up Internet Explorer and go to the same URL you are currently viewing in Firefox. I use this primarily for browser testing, but sadly, some sites, including some of my employers intranet sites, do not work with Firefox.

If you don't use Firefox, download it today. It's free, blocks popups automatically, displays websites faster, imports all your bookmarks from Internet Explorer, and will easily improve your Interweb experience.

Being in the computer industry, dealing with users and customers all the time, I know the value of good customer service. Customer Relationship Management software, or CRM, is what companies use to keep track of how they have done business with you, the customer. Sometimes, they just don't get it right.

Here's an example.

I'm blessed with a post-40 year male pattern baldness. My hair is thick, lush and ever growing around the sides and back, while thin and wispy on top. Eventually the top will all be gone, and I'll look like my beloved Grandpa Sickler: I'm ok with that. I simply hope that I don't have a horribly disfigured dome hiding under what remains of my thin wispy hair.

These days when the sides and back get bushy, I head off to the Great Clips down the street and pay my $12 bucks. This gets me some beauty school dropout to work the #2 buzzer around the sides and back, then a little trimmage on the top. Buzz, Buzz, Snip Snip, done - I usually spend more time waiting than in the chair itself.

Now at Great Clips, my hair cutting history is in the computer; this is a CRM system, albeit kinda low-tech. They ask for my phone number and the computer then tells them my name as well as pertinent details of prior visits - how long, what kind of cut, etc. This way, the newest beauty school dropout who is lucky enough to draw my name from the waiting room hat, doesn't have to have a long discussion with me about my "style." I simply say "#2 Blend, it's been a long time since my last cut, so don't be shy". This says that I acknowledge that lots of hair will be cut.

(If you are not familar, a #2 is short enough that when my hair stands on end, you can't tell the difference - it's military in its shortness, and after a shower, it's dry before I get out.)

So last night I get get "styled" and go to pay. Usually on the receipt, it says something like, "This receipt is good for $3 off your next visit" - and being the responsible person I am, I promptly lose it and never bring it back.

Last night, after my #2 Buzzcut, the receipt said:

"This receipt is good for $10 off your next perm."

I don't think that coupon has much value for me. Maybe their CRM system needs a little improvement.