A Flexible AJAX Framework

AJAX = Asyncronous (but not always), JavaScript, XML (Text or JSON). These technologies have been around almost as long as the Internet itself, but have only recently been used together to change the way that web applications are built.

XMLHTTPRequest is an API used mainly by browser-based Javascript applications to send and retrieve data from servers. It was developed originally by Microsoft for Exchange Server’s Outlook Web Access, but it has since been widely supported in browsers and is the heart of AJAX dynamic Web applications.

I’ve messed with many of the frameworks available online, and while they are very good, I’ve routinely found issues or features that they lack. Over time I’ve cobbled together my own framework, I fully expect this to change over time, but publish it here for your consideration!

The XSS methods (and “otherHost” variable) are provided as MSIE6 allows for Cross-site Scripting here. MSIE7 and Mozilla will indicate errors if connections are attempted to a ‘non-originating host’. I’ve exploited this vulnerability in a few applications, as such I provide the test for you too!

NOTE: The example response program is written in PHP, but should be easily ported to any other language.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html>

<head>

<title>AJAX Example</title>

<script type=”text/javascript”>

var xbusy = true;

var otherHost = ‘http://www.giantgeek.com’; // DO NOT USE the ending slash!

var xhr = null;

/**

* Check for existance on DOM browsers (Mozilla, etc.)

* @return xhr

*/

function ajaxCheckDOM(){

var myxhr = null;

if(window.XMLHttpRequest) {

try {

myxhr = new XMLHttpRequest();

}

catch(e) {}

}

return myxhr;

}

/**

* Check for existance on Windows/MSIE (prior to MSIE7 which is now DOM)

* Evaluate using – new ActiveXObject(“Microsoft.XMLDOM”);

* @return xhr

*/

function ajaxCheckActiveX(){

var myxhr = null;

//if(window.ActiveXObject){

try {

myxhr = new ActiveXObject(“Msxml2.XMLHTTP”);

}

catch(e) {

try {

myxhr = new ActiveXObject(“Microsoft.XMLHTTP”);

}

catch(e) {}

}

//}

return myxhr;

}

/**

* This is a default response hook for AJAX, you SHOULD create your own as it’s only for DEMO

* @param obj – the clicked item

* @param customObj – user defined

*/

function ajaxResponseHook(obj,customObj){

var txt = xhr.responseText;// NOTE: xhr.responseXML is also valid

popStatus(‘AJAX Response value [‘ + txt + ‘|’ + customObj +’]’,”);

}

/**

* This is a common Response handler AJAX, you SHOULD NOT need to modify, use a ‘custom’ ajaxhook function to process the responses.

* @param obj – the clicked item

* @param ajaxhook Function (default will be used if undefined)

* @param customObj (optional – left for developer implementation, passed to the ajaxhook)

*/

function ajaxResponse(obj,ajaxhook,customObj){

var status=”;

if(xhr!=undefined){

var state=xhr.readyState;

if(state!=undefined){

if(state==4){

var code = xhr.status;

status = xhr.statusText;

if(code==200){

popStatus(‘AJAX Response Received.’,”);

if(ajaxhook==undefined){

ajaxhook = function(){ ajaxResponseHook(obj,customObj); }

}

ajaxhook(obj,customObj);

ajaxBusy(obj,false);

} else {

popStatus(‘AJAX Error ‘ + code + ‘ ‘ + status + ‘.’,”);

}

ajaxBusy(obj,false);

xhr=null; /* MSIE6 leak fix */

}

}

}

}

/**

* NOTE: MSIE6-7 supports XSS url’s (be careful!)

* @param obj – the clicked item

* @param url – the GET url params (FQDN)

* @param async (true or false) – false will LOCK browser until response – use cautiously!

* @param callback Function – allows for customized response handling

*/

function ajaxObj(obj,url,async,callback){

if(xbusy == true){

popStatus(‘AJAX BUSY, Please Retry.’,”);

} else {

if(callback==undefined){

callback = function(){ ajaxResponse(obj,hook,”); }

}

ajaxInit(obj,url,async,callback);

}

}

/**

* This is a Non-Caching implementation of ajaxObj() to show how you can avoid the MSIE caching issue.

* NOTE: You can use similar approaches to cache per page or session.

* @param obj – the clicked item

* @param url – the GET url params (FQDN)

* @param async (true or false) – false will LOCK browser until response – use cautiously!

* @param callback Function – allows for customized response handling

*/

function ajaxObjNoCache(obj,url,async,callback){

var cacheBuster=uniqueUrl(url);// damn MSIE!

ajaxObj(obj,cacheBuster,async,callback)

}

/**

* Initializes the AJAX operation

* @param obj – item clicked

* @param url – FQDN

* @param async (true/false) – for locking

* @param callback Function – provided for customization.

*/

function ajaxInit(obj,url,async,callback){

xhr=ajaxCheckDOM();

if(!xhr){

xhr=ajaxCheckActiveX();

}

if(xhr){

ajaxBusy(obj,true);

popStatus(‘AJAX Start.’,”);

xhr.onreadystatechange = callback; // method call

try {

xhr.open(‘GET’,url,async); /* POST */

//xhr.setRequestHeader(‘Content-Type’,’application/x-www-form-urlencoded’);

xhr.send(”); /* null */

}

catch(e)

{

if(xhr){

var code = xhr.status;

var status = xhr.statusText;

popStatus(‘AJAX Client SECURITY ‘ + navigator.appName +’ ‘+ navigator.appVersion + ‘ ‘ + code + ‘ ‘ + status + ‘.’,”);

}

}

ajaxBusy(obj,false);

} else {

popStatus(‘AJAX Client ERROR.’,”);

}

}

function ajaxBusy(obj,yn){

if(yn){

swapStyleObj(obj,’idle’,’busy’);

} else {

swapStyleObj(obj,’busy’,’idle’);

}

xbusyInd(yn,”);

}

function xmillis(){

return new Date().getTime();

}

function swapStyleObj(obj,oldCSS,newCSS){

if(obj!=undefined) {

var current=obj.className;

if(current!=undefined){

var txtOld = current.replace(newCSS,’ ‘);//no doubles

var txtMid = txtOld.replace(oldCSS, ‘ ‘);

var txtNew = (txtMid + ‘ ‘ + newCSS);

obj.className = txtNew;

} else {

obj.className = newCSS;

}

}

}

function xbusyInd(valnew, msg){

xbusy = valnew;

if(xbusy==true){

document.body.style.cursor=’wait’;

if(msg != ”){

top.window.defaultStatus=msg;

popStatus(msg,”);

}

showDiv(‘throbber’);

//blockScreen();

} else {

document.body.style.cursor=’default’;

hideDiv(‘throbber’);

//unBlockScreen();

}

}

function showDiv(id) { //show a div

var obj=xgetHelper(id);

if(obj!=null){ obj.style.display=”block”;}

//xrestart();

}

function hideDiv(id) { //hide a div

var obj=xgetHelper(id);

if(obj!=null){ obj.style.display=”none”;}

//xrestart();

}

function popStatus(txt,title){

popit(‘statusdyn’,’statusarrow’,’statusul’,’statusdiv’,txt,false,title,”);

}

function popit(dynId,arrowId,ulId,divId,txt,expand,title,cssCls){

popText(ulId,txt,title,cssCls);

showDiv(divId);

if(expand == true){

var arrowObj=xgetHelper(arrowId); if(arrowObj!=null){ arrowObj.style.backgroundPosition=’2px -106px’; }

showDiv(dynId);

}

}

function popText(id,txt,title,cssCls){

var obj = xgetHelper(id);

if(obj != null){

var oldHTML = obj.innerHTML;

var cls=”; if(cssCls!=”){ cls=’ class=”‘ + cls +'”‘; }

var htm = ‘<‘+’li’+ cls +’>’ + txt + ‘<‘+’/’+’li’+’>’ + oldHTML;

obj.innerHTML = htm;

}

}

function xgetHelper(id){

var obj = null;

try {

obj = document.getElementById(id);

} catch(z) {

var dummy=alert(“Error:” + z);

}

return obj;

}

function arrowTog(objectID,arrow) {

var obj = xgetHelper(objectID);

if(obj!=null){

if (obj.style.display ==’block’) {

arrow.style.backgroundPosition = ‘2px -21px’;}

else {arrow.style.backgroundPosition = ‘2px -106px’;}

objTog(objectID);

}

return false;

}

function objTog(objectID) {

var obj = xgetHelper(objectID);

if(obj!=null){

if (obj.style.display ==’block’) obj.style.display=’none’;

else {obj.style.display=’block’;}

}

return false;

}

function headTog(objectID,arrow) {

var obj = xgetHelper(objectID);

if(obj!=null){

if (obj.style.display ==’block’) {

arrow.style.borderBottomWidth = ‘1px’;}

else {arrow.style.borderBottomWidth = ‘0px’;}

arrowTog(objectID,arrow);

}

return false;

}

/*

* adds timestamp to URLs to make them unique

* @param URL String

*/

function uniqueUrl(x){

return urlAppender(x,’.cache’,xmillis());

}

/*

* helps to add parms to the url

* @param URL String

* @param aname String

* @param avalue String

*/

function urlAppender(x,aname,avalue){

var delim = “?”;

if(x.indexOf(“?”) >=0) { delim = “&”; }

return x + delim + aname + ‘=’ + avalue;

}

function xload(){

hideDiv(‘throbber’);

xbusy=false;

}

function testAjax(obj){

var callback=function(){ ajaxResponse(obj,null,’testAjax’); }

var x = ajaxObjNoCache(obj,’/ajax.php’,true,callback);

}

function testAjaxParms(obj){

var callback=function(){ ajaxResponse(obj,null,’testAjaxParms’); }

var x = ajaxObjNoCache(obj,’/ajax.php?testing=Y’,true,callback);

}

function testAjaxXSS(obj){

var callback=function(){ ajaxResponse(obj,null,’testAjaxXSS’); }

var x = ajaxObjNoCache(obj,otherHost+’/ajax.php’,true,callback,otherHost);

}

function testAjaxHook(obj){

var hook=function(){ customAjaxHook(obj,’ajaxhookTestMessageObect’); }

var callback=function(){ ajaxResponse(obj,hook,’testAjaxHook’); }

var x = ajaxObjNoCache(obj,’/ajax.php’,true,callback);

}

function testAjaxHookXSS(obj){

var hook=function(){ customAjaxHook(obj,’ajaxhookXSSMessageObect’); }

var callback=function(){ ajaxResponse(obj,hook,’testAjaxHookXSS’); }

var x = ajaxObjNoCache(obj,otherHost+’/ajax.php’,true,callback);

}

/**

* This is a custom implemenation, the customObj COULD be used for anything (perhaps delay measurement!)

* @param obj – the clicked item

* @param customObj – user defined

*/

function customAjaxHook(obj,customObj){

var xml = xhr.responseXML;// NOTE: xhr.responseText is also valid

var tmp = ‘DEMO customAjaxHook [‘ + customObj + ‘]\n’ + xml;

alert(tmp);

}

</script>

<style type=”text/css”>

.busy { color:red; }

.idle { color:black; }

</style>

</head>

<body onload=”xload();”><!– onbeforeunload=”alert(‘before’);” onunload=”alert(‘after’);” –>

<div id=”throbber”>WORKING!</div>

<div id=”statusdiv” class=”dyn” style=”display:none;”>

<h3><a id=”statusarrow” onclick=”headTog(‘statusdyn’,this);” href=”javascript:void(0);”>Status</a></h3>

<fieldset id=”statusdyn” style=”display:block;background-color:#ffffcc;”>

<div id=”statusBox” class=”box”>

<ul id=”statusul” class=”error”>

<li id=”ajaxStatus” style=”list-style:none;display:none;”>FILLER</li>

</ul>

</div>

</fieldset>

</div>

<h3><a href=”javascript:void(0);” onclick=”testAjax(this);”>TEST</a></h3>

<h3><a href=”javascript:void(0);” onclick=”testAjaxParms(this);”>TESTPARMS</a></h3>

<h3><a href=”javascript:void(0);” onclick=”testAjaxXSS(this);”>TESTXSS</a></h3>

<h3><a href=”javascript:void(0);” onclick=”testAjaxHook(this);”>TESTHOOK</a></h3>

<h3><a href=”javascript:void(0);” onclick=”testAjaxHookXSS(this);”>TESTHOOKXSS</a></h3>

<p><a href=”index.php”>RELOAD</a></p>

</body>

</html>

Response program (ajax.php)

<?php

header(“Cache-Control: no-store”);

header(“Charset: UTF-8”);

header(“Content-Type: text/xml”);

echo(“<?xml version=\”1.0\” encoding=\”UTF-8\”?>\n”);

?>

<test><?php echo( gmdate(“D, d M Y H:i:s”) ) ?> </test>

What are the MSIE Bugs you might ask… these are all in the ActiveX implementation of XmlHttpRequest, they are:

  1. Caching – for some reason, unless you define a ‘unique’ URL for each request, MSIE6 will respond from the cache, even if the server is unreachable!
  2. XSS – I discussed this above, MSIE6 will allow you to connect to ANY HOST with AJAX!
  3. Memory leak – if you don’t nullify the XHR object (for MSIE6), the application will leak memory.

Cheers!

Mozilla networking configuration

Here’s another, albeit awkward configuration change for Mozilla Firefox for networking.

Enter about:config in the URL of the browser and manipulate the following,¬† I’ve shown the defaults in parethesis to aid in reverting them if you encounter problems.

network.http.pipelining=true (def:false)
network.http.proxy.pipelining (def:false – only required if you use proxies that support)
network.http.pipelining.maxrequests=8 (def:4, max is 8)

References:

Cheers!

NOTE: this process is obsolete, from what I can gather it was only supported in MSIE6, and possibly MSIE7.

Use of this tag will disable the Image Toolbar (normally accessed via right-click) within MSIE. Typically it is enabled whenever an image larger than 130×130 is displayed.

Implementation:
Add the following to the <head> section of your webpage(s):
<meta http-equiv="imagetoolbar" content="false" />

Alternately, you COULD use some proprietary MSIE attributes on the <img /> tag.
<img src="..." galleryimg="false" />

Even when you use the META tag to disable this feature for all images, you can explicitly re-enable it with the following proprietary tag…
<img src="..." galleryimg="true" />

References:

PICS Implementation Guide

Platform for Internet Content Selection (PICS)

This was originally designed to help parents and teachers control what children access on the Internet, but it also facilitates other uses for labels, including code signing and privacy. The PICS platform is one on which other rating services and filtering software have been built.’

References:

HTTP Headers (optional):


Protocol: {PICS-1.1 {headers PICS-Label}}
PICS-Label: (PICS-1.1 'http://www.weburbia.com/safe/ratings.htm' l r (s 0))

Example HTML:


<html>
<head>
<title>example</title>
<meta http-equiv="PICS-Label" content="(PICS-1.1 'http://www.weburbia.com/safe/ratings.htm' l r (s 0))" />
</head>
<body>
...
</body>
</html>

P3P 1.0 Implementation guide

Standards documentation is available from W3C at:

NOTES:

  1. Version P3P 1.1 is currently in the works.
  2. Throughout the specifications you’ll see references to “Well-Known Location”, this refers to the default path and naming of these files in the /w3c/ folder.
  3. In my examples below, I have left MOST data empty, the “

xxx” indicates a field that must match between these files.
HTML:


<html>
<head>
<link type="text/xml" rel="P3Pv1" href="/w3c/p3p.xml" />
</head>
<body>
...
</body>
</html>

HTTP Header:

p3p: policyref="/w3c/p3p.xml", CP="TST"

/w3c/p3p.xml:


<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<META xmlns="http://www.w3.org/2002/01/P3Pv1">
<POLICY-REFERENCES>
<POLICY-REF about="/w3c/privacy.xml#xxx">
<INCLUDE>/*</INCLUDE>
<COOKIE-INCLUDE name="*" value="*" domain="*" path="*" />
</POLICY-REF>
</POLICY-REFERENCES>
</META>

/w3c/prixacy.xml


<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<POLICIES xmlns="http://www.w3.org/2002/01/P3Pv1">
<POLICY name="xxx" discuri="/index.html" xml:lang="en">
<ENTITY>
<DATA-GROUP>
<DATA ref="#business.name"></DATA>
<DATA ref="#business.department"></DATA>
<DATA ref="#business.contact-info.postal.name.given"></DATA>
<DATA ref="#business.contact-info.postal.street"></DATA>
<DATA ref="#business.contact-info.postal.city"></DATA>
<DATA ref="#business.contact-info.postal.stateprov"></DATA>
<DATA ref="#business.contact-info.postal.postalcode"></DATA>
<DATA ref="#business.contact-info.postal.country"></DATA>
<DATA ref="#business.contact-info.online.email"></DATA>
<DATA ref="#business.contact-info.telecom.telephone.intcode"></DATA>
<DATA ref="#business.contact-info.telecom.telephone.loccode"></DATA>
<DATA ref="#business.contact-info.telecom.telephone.number"></DATA>
<DATA ref="#business.contact-info.online.uri"></DATA>
</DATA-GROUP>
</ENTITY>
<ACCESS><nonident/></ACCESS>
<DISPUTES-GROUP>
<DISPUTES resolution-type="service" service="/index.html" short-description="Customer Service">
<LONG-DESCRIPTION></LONG-DESCRIPTION>
<REMEDIES><correct/></REMEDIES>
</DISPUTES>
</DISPUTES-GROUP>
<STATEMENT>
<CONSEQUENCE>We record some information in order to serve your request and to secure and improve our Web site.</CONSEQUENCE>
<PURPOSE><current/><develop/><admin/></PURPOSE>
<RECIPIENT><ours/></RECIPIENT>
<RETENTION><stated-purpose/></RETENTION>
<DATA-GROUP>
<DATA ref="#dynamic.clickstream"/>
<DATA ref="#dynamic.http"/>
</DATA-GROUP>
</STATEMENT>
</POLICY>
</POLICIES>

REFERENCES:

  • http://www.w3.org/TR/2000/CR-P3P-20001215/
  • http://msdn.microsoft.com/en-us/library/ie/ms537343%28v=vs.85%29.aspx#unsatisfactory_cookies

ROBOTS.TXT

I’ve been asked about this file in many projects i’ve worked on. It resides in the root of the website, and has no external references to it, however, there is usually a lot of requests for it in the server logs. (Or… “404 Not Found” Errors if it doesn’t exist).

Additionally, automated security audit software will often indicate that this file itself is a possible security problem as it can expose hidden areas of your website (more on this later).

Here’s what it’s all about….

ROBOTS.TXT is used by spiders and robots, primarily so that they can index your website for search engines (which is usually a good thing). However…. there are times when you don’t want this to occur. Some spiders/robots can be too agressive on your servers, consuming precious bandwidth and CPU utilization, or they can dig too deep into your content. As such you might want to control their access.

The Robots Exclusion Protocol sets out several ways to accomplish this goal. Of course the spider must comply with this convention.
1. ROBOTS.TXT can be used to limit the access:

Example that limits only the images, javascript and css folders:

#robots.txt - for info see http://www.robotstxt.org/wc/robots.html
User-agent: *
Disallow: /images/
Disallow: /js/
Disallow: /css/

2. A <meta> tag on each webpage indicating spider actions to take.

<html>
<head>
<title>example</title>
<meta name="robots" content="INDEX, FOLLOW, ALL" />
</head>
<body>
...
</body>
</html>

Values, there are a few other attributes, but these are the most common….
INDEX -index this page
NOINDEX – do not index this page
FOLLOW -follow links from this page
NOFOLLOW -do not follow links from this page
ALL – same as INDEX, FOLLOW
NONE – same as NOINDEX, NOFOLLOW

In most cases, a spider/robot will first request the ROBOTS.TXT file, and then start indexing the site. You can exclude all or specific spiders from individual files or directories.

As for the earlier bit on security, since this file is available to anyone, you should never indicate sensitive areas of your website in this file as it would be an easy way to find those areas.