Processing Long Forms in Servlets/JSP – An easy way out

After downloading and watching “Omo Ghetto” aka Ghetto Child (A nigerian home video), and listening and re-listening to a wonderful, soulful, inspiring sermon “Put your trust in Allah” by brother Belal Asaad (watch here – its just 7 minutes), I decided to write this post to round off the weekend.

One of the modules of the EMIS (Education Management Information System) platform I am working on presently is a capacity assessment system that enables member states (countries) submit data on their current state of education. To cut the long English short, its a QUESTIONNAIRE. . .a very long, boring and annoying one. Its too sophisticated for all the online survey builders I checked out before I decided to attack with my fingers. It’s the first time I will be processing forms of this nature with more than a hundred check boxes, input boxes, combos, file uploads etc.

For a lazy programmer like myself, the the shear terror and fright of having to setup database tables, design the forms, validate, submit entered data, retrieve later for updates could make my face start to age at a speed of 5 years by minute. Good news is that Allah in his infinite mercies after giving us life, water, food and other things, he gave us Objected Oriented Programming to make our lives easy 🙂

Form Designs and Validation

I use JotForm. . .First Web Based WYSIWYG Form Builder. Its simple to use, Dnd (drag ‘n’ drop), Ajax validation, ready made custom components etc. The beauty of it all is that you can download your entire source files as zip archive. . .! Now that’s gangster. . .! Visit www.jotform.com to get started.

Apache Commons To The Rescue

Yap. . .I checked out several java form processing frameworks and Apache Commons FileUpload would be just fine for me. This API is designed to handle file uploads in Servlets/JSP but studying the documentation (click here to view I discovered two lovely classes.

FileItemStream: This interface provides access to a file or form item that was received within a multipart/form-data POST request.

FileItemIterator: An iterator, as returned by FileUploadBase.getItemIterator(RequestContext).

So with these classes, I could interate through my forms using the FileItemIterator then retrieve the element property using the FileItemStream. . .

I donloaded the Apache Commons library and the FileUpload library here http://commons.apache.org/

Database Setup

While designing the forms, I named every component as I would have named the columns that would hold their data in the database table. Then I wrote a JSP that handles table creation. Basically, the JSP loops  through every component of the form and retrieves the name of each component, builds a ‘CREATE TABLE’ query and executes it. I repeated this process for each form. Below is the JSP

[sourcecode language=”java”]
<%@ page import="java.io.File" %>
<%@ page import="java.io.FileOutputStream" %>
<%@ page import="java.io.InputStream" %>
<%@ page import="org.apache.commons.fileupload.servlet.ServletFileUpload"%>
<%@ page import="org.apache.commons.fileupload.disk.DiskFileItemFactory"%>
<%@ page import="org.apache.commons.fileupload.*"%>
<%@page import="org.apache.commons.fileupload.util.Streams;" %>

<%
String sql="CREATE TABLE `"+request.getParameter("table")+"` ( `RDate` datetime NOT NULL, `ID` varchar (4) NOT NULL,";
InputStream stream = null;
ServletFileUpload uploads = new ServletFileUpload();

// Parse the request
FileItemIterator iter = uploads.getItemIterator(request);
while (iter.hasNext())
{
FileItemStream item = iter.next();
String name = item.getFieldName();
stream = item.openStream();

if (item.isFormField())
{
String val=""+ Streams.asString(stream).toString();
sql=sql+"`"+name+"` LONGTEXT NOT NULL default ”,";

}
else
{
//Then its a FileUpload field. You can handle it as you wish.
}

}

boolean done = mainBean.executeUpdate(sql);

%>
[/sourcecode]

The code above is self explainatory even without comments. It loops through the form and builds and sql query using the form element names. The above JSP will work with any form. To use do this this. . .

<form method=”post” action=”createTable.jsp?table=section1″ enctype=”multipart/form-data”>

Form Submission

Next, I wrote another JSP that handles submissions to the database. Again, I looped through the form, retrieve form elements name and value, build an insert or update query then execute. Did I say an update query? Yes, because before any form is submitted, I check to see if the user has not filled the particular section and the query is built accordingly.

Here we go. . .

[sourcecode language=”java”]
<%@ page import="java.util.Iterator" %>
<%@ page import="java.util.ArrayList" %>
<%@ page import="java.io.File" %>
<%@ page import="java.io.FileOutputStream" %>
<%@ page import="java.io.InputStream" %>
<%@ page import="org.apache.commons.fileupload.servlet.ServletFileUpload"%>
<%@ page import="org.apache.commons.fileupload.disk.DiskFileItemFactory"%>
<%@ page import="org.apache.commons.fileupload.*"%>
<%@page import="org.apache.commons.fileupload.util.Streams;" %>

<%

String sql="";
String section =request.getParameter("f");
String id=request.getParameter("id");
String isUpdate = mainBean.checkSectionCompleted(id,section);  //Check if the user has completed this section

InputStream stream = null;

ServletFileUpload uploads = new ServletFileUpload();

// Parse the request
FileItemIterator iter = uploads.getItemIterator(request);
while (iter.hasNext())
{
FileItemStream item = iter.next();
String name = item.getFieldName();
stream = item.openStream();
if (item.isFormField())
{
String val=""+ Streams.asString(stream).toString();
if(name.startsWith("Check")) //Handling checkboxes
{
if(val != null)
{
val = "OFF";
}
else
{
val = "ON";
}
}

sql=sql+name+"=’"+val+"’, ";
}

}

sql=sql+"@";
sql=sql.replace(", @", "");  // "Ojoro" (trick) to remove the last comma appended to the query. . .hahaha

String m = "";

if(isUpdate.equals("false"))
{
m="insert into section_"+section+" set ID=’"+id+"’, RDate=Now(), "+sql;
}
else
{
m="update section_"+section+" set "+sql+" where ID=’"+id+"’";
}

boolean done = mx.updateDoneSection(id,section,m);

%>
[/sourcecode]

Finally, I changed the destination JSP in all my <form> tags to saveSection.jsp.

Now to retrieve the entered data, I placed some javascript snippet at the bottom of every form that talks to my java bean. This snippet checks if the user has completed a particular section. If yes, it fetches the entered data as a JSON, parses it and sets each form element value. The data entered for this section is then retrieved alog with the column names using ResultSetMetaData. Remember the column are derived from our form element names, so setting values for them should be pretty easy.

[sourcecode language=”javascript”]
<script>

var isDone = "<%=mainBean.checkisDone("9",id)%>";  //Notice how the java bean is accessed from javascript code. The check isDone accepts the section number and the user ID as parameter
if(isDone == "yes")
{
var elementValues = "<%=m1.getElementValues("9",id)%>";
var words = elementValues.trim().split("%%%_");
for (i = 0; i < words.length; i++)
{
var e = words[i].split("-:::-");
var l=e[0];  //column name (form element name)
var v=e[1]; // value
//alert(l);

var p = document.getElementsByName(l)[0].type;
switch(p)
{
case ‘text’:
case ‘textarea’:
case ‘radio’:
case ‘select-one’:
document.getElementsByName(l)[0].value = v;
break;
case ‘checkbox’:
if(v == "OFF")
document.getElementsByName(l)[0].checked = true;

}
}

}

</script>
[/sourcecode]

Ces’t fiini… I completed the entire application in two days because all I focused on was designing the forms. The  DRY – Don’t Repeat Yourself principle that OOP preaches has been applied here and it worked really well.

If you feel there is a better way I could have achieved my goal, please feel free to share. I am open minded. . .:)